CN111143361B - Characteristic data writing method and system for online training - Google Patents
Characteristic data writing method and system for online training Download PDFInfo
- Publication number
- CN111143361B CN111143361B CN201911319666.2A CN201911319666A CN111143361B CN 111143361 B CN111143361 B CN 111143361B CN 201911319666 A CN201911319666 A CN 201911319666A CN 111143361 B CN111143361 B CN 111143361B
- Authority
- CN
- China
- Prior art keywords
- online training
- preset script
- characteristic data
- redis
- writing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012549 training Methods 0.000 title claims abstract description 171
- 238000000034 method Methods 0.000 title claims abstract description 44
- 238000013515 script Methods 0.000 claims abstract description 179
- 238000012545 processing Methods 0.000 claims abstract description 35
- 238000004590 computer program Methods 0.000 claims description 16
- 230000001960 triggered effect Effects 0.000 abstract description 20
- 238000010586 diagram Methods 0.000 description 9
- 230000008569 process Effects 0.000 description 6
- 230000006870 function Effects 0.000 description 3
- 230000006978 adaptation Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 2
- 230000006399 behavior Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2471—Distributed queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
Abstract
The application provides a characteristic data writing method, a characteristic data writing system, computer equipment and a storage medium for online training, wherein the characteristic data writing method comprises the following steps: receiving a plurality of write operations for feature data of the online training; sequentially summarizing each writing operation into a preset script, wherein the preset script is pre-configured with processing instructions written in on-line training characteristic data; requesting the redis server to run a preset script to sequentially execute each writing operation and write the characteristic data of the online training in the redis cluster through the redis server. According to the method, the plurality of requests are sent once through the preset script and are requested to the redis server once, so that network overhead and network time delay are reduced, writing of the characteristic data of the plurality of online training and online training of the characteristic data can be triggered, and the efficiency of online training of the special data is improved.
Description
Technical Field
The application relates to the technical field of data processing, in particular to a characteristic data writing method, a characteristic data writing system, computer equipment and a storage medium for online training.
Background
Redis (Remote Dictionary Server, remote dictionary service) has very excellent performance, can support read/write operations of hundreds of thousands of operations per second, has far beyond a database, and also supports configuration such as clustering, distribution, master-slave synchronization and the like, and can be expanded infinitely in principle, so that more data can be stored in a memory. Machine learning has many training features, and feature data of many dimensions is stored for online data. A large number of online features need to be invoked during the training process and generated after the training are stored in the redis cluster.
The read-write operation is separated in the redis, the online feature is written into the redis cluster through a writer service, and corresponding feature data is summarized or read from the redis cluster by a reader service.
The method has the advantages that the writer service and the reader service in the redis operation are respectively and directly called in each online training, the operation frequency of the redis is high, and the online training efficiency is affected.
Disclosure of Invention
Based on this, it is necessary to provide a characteristic data writing method, system, computer device and storage medium for online training, aiming at the technical defects, especially the technical defects of high operating redis frequency and low online training efficiency.
A characteristic data writing method for online training comprises the following steps:
receiving a plurality of write operations for feature data of the online training;
sequentially summarizing each writing operation into a preset script, wherein the preset script is pre-configured with a processing instruction written in the processing instruction for training the characteristic data on line;
requesting the redis server to run the preset script so as to sequentially execute each writing operation and write the characteristic data of the online training in the redis cluster through the redis server.
In one embodiment, the preset script is a script written in the Lua scripting language.
In one embodiment, before the step of sequentially summarizing the write operations into the preset script, the method further includes:
and writing the reading operation of each online training characteristic data into the preset script so that the processing instruction calls the online training characteristic data.
In one embodiment, the redis server is further configured to start executing the preset script when the constraint condition is satisfied.
In one embodiment, the step of requesting the redis server to run the preset script includes:
uploading the preset script to the redis server;
and sending an execution request to the redis server to request the redis server to run the preset script.
In one embodiment, the redis server is further configured to receive execution requests sent by other redis clients to run the preset script.
In one embodiment, the redis server is further configured to execute the updated preset script when it is detected that the preset script is updated.
A feature data writing system for online training, comprising:
a receiving module for receiving write operations for a plurality of feature data for online training;
the summarizing module is used for summarizing each writing operation into a preset script in sequence, wherein the preset script is pre-configured with a processing instruction written into the on-line training characteristic data;
the operation module is used for requesting the redis server to operate the preset script so as to sequentially execute each writing operation and write the characteristic data of the online training in the redis cluster through the redis server.
A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the feature data writing method of online training of any of the embodiments described above when the computer program is executed.
A computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of the feature data writing method of online training of any of the above embodiments.
According to the characteristic data writing method, system, computer equipment and storage medium for online training, through receiving a plurality of writing operations aiming at the characteristic data of online training, each writing operation is sequentially summarized into the preset script, wherein the preset script is pre-configured with a processing instruction for writing the characteristic data of online training, a redis server is requested to run the preset script, each writing operation is sequentially executed through the redis server, and the characteristic data of online training is written in a redis cluster; only by sending a plurality of requests once through a preset script and requesting the redis server once, network overhead and network time delay are reduced, writing of the characteristic data of a plurality of online training and online training of the characteristic data can be triggered, and the efficiency of online training of special data is improved.
Additional aspects and advantages of the application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice.
Drawings
The foregoing and/or additional aspects and advantages will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:
FIG. 1 is a diagram of an environment in which a feature data writing method for online training is implemented, as provided in one embodiment;
FIG. 2 is a schematic diagram of the principle of redis operation;
FIG. 3 is a flow chart of a feature data writing method for online training in one embodiment;
FIG. 4 is a schematic diagram of a feature data writing system for online training in one embodiment;
fig. 5 is a schematic diagram showing an internal structure of the computer device in one embodiment.
Detailed Description
Embodiments of the present application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative only and are not to be construed as limiting the application.
As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless expressly stated otherwise, as understood by those skilled in the art. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. The term "and/or" as used herein includes all or any element and all combination of one or more of the associated listed items.
It will be understood by those skilled in the art that all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless defined otherwise. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
As shown in fig. 1, fig. 1 is a diagram of an implementation environment of a feature data writing method for online training provided in one embodiment, in which a computer device 110 and a server device 120 are included.
The server device 120 is configured with a redis database and is provided with a redis server, and the redis server can perform read-write operations on the redis cluster, and the computer device 110 is provided with a redis client. The computer device 110 may configure a script file by sending a request to a redis client and to a database of the redis server, the redis server responding to the received request, and executing the configured script file.
It should be noted that the computer device 110 may be a desktop computer, a notebook computer, a tablet computer, or the like, but is not limited thereto. The computer device 110 and the server device 120 may be connected through a network, and the present application is not limited herein.
As shown in fig. 2, fig. 2 is a schematic diagram of the redis operation, in which a large number of online features are stored in a redis cluster. The read-write separation in the redis operation requires that the feature data stream be written to the redis cluster by a writer, and that the reader is responsible for summarizing or reading the corresponding feature data from the redis cluster.
Because the reading and writing of the redis cluster storing the characteristic data are separated, in the process of training the characteristic data on line, the redis client requests to retrieve the characteristic data from the redis database, and triggers the redis server to invoke the reader to read the characteristic data from the corresponding redis cluster, and the redis database returns the characteristic data to the redis client. After the feature data is acquired, the feature data is operated according to the processing of the online training, and the trained feature data is generated. After online training, the characteristic data is required to be written into the redis database through the redis client, and the redis server is triggered to call the writer to write the characteristic data into the corresponding redis cluster. In each online training, the writers and readers for redis operation are respectively and directly called for a plurality of times, so that the operation frequency of redis is high, and the online training efficiency is affected.
In order to solve the technical problems, the application provides a feature data writing method for online training, which is used for receiving a plurality of writing operations aiming at feature data for online training, and sequentially summarizing each writing operation into a preset script, wherein the preset script is pre-configured with a processing instruction for writing the feature data for online training, requesting a redis server to run the preset script, and sequentially executing each writing operation through the redis server and writing the feature data for online training in a redis cluster. Therefore, a plurality of requests are sent once through a preset script and are requested to the redis server once, and the efficiency of online training of the feature data is improved.
In one embodiment, as shown in fig. 3, fig. 3 is a flowchart of a feature data writing method of online training in one embodiment, and in this embodiment, an online training feature data writing method is provided, where the online training feature data writing method may be applied to the redis client of the computer device 110, and may specifically include the following steps:
step S310: a plurality of write operations are received for the online trained feature data.
The write operation is used for requesting the redis server to write the characteristic data into the redis cluster, and in the step, a plurality of write operations aiming at the characteristic data of the online training are received, so that a plurality of characteristic data obtained through the online training are written into the redis cluster.
Step S320: and sequentially summarizing each writing operation into a preset script, wherein the preset script is pre-configured with processing instructions written in the online training characteristic data.
In this step, a processing instruction for performing online training on the feature data is configured in the preset script, and a plurality of writing operations are sequentially summarized to the preset script, so that the preset script can be used for starting online training and updating the feature data after training. The summarized preset script can be sent to redis; or the preset script can be stored in redis in advance, and write operations are summarized into the preset script by modifying the preset script.
Step S330: requesting the redis server to run a preset script to sequentially execute each writing operation and write the characteristic data of the online training in the redis cluster through the redis server.
In this step, the redis server is requested to run a preset script, the redis server is triggered to execute a processing instruction of online training feature data in the preset script, the redis server is triggered to realize the online training feature data, and the redis server is triggered to sequentially write a plurality of feature data into the redis cluster.
According to the characteristic data writing method for online training, by receiving a plurality of writing operations aiming at the characteristic data of online training, each writing operation is sequentially summarized into the preset script, wherein the preset script is pre-configured with a processing instruction for writing the characteristic data of online training, a redis server is requested to run the preset script, each writing operation is sequentially executed through the redis server, and the characteristic data of online training is written in a redis cluster; only by sending a plurality of requests once through a preset script and requesting the redis server once, network overhead and network time delay are reduced, writing of the characteristic data of a plurality of online training and online training of the characteristic data can be triggered, and the efficiency of online training of special data is improved.
In addition, the redis server side can execute the preset script as a whole, and other commands cannot be inserted in the execution process, so that race conditions are not required to be worried about in the writing process of the preset script, the execution of the preset script cannot be interrupted by a thread scheduling mechanism, and atomic operation is realized.
In one embodiment, the preset script is a script written in the Lua scripting language.
Lua is a lightweight and compact scripting language written in standard C language and open in source code, and Lua script written in Lua scripting language is embedded in an application program, thereby providing flexible extension and customization functions for the application program. The Lua script has the characteristics of light weight and expandability, and supports process-oriented programming and functional programming. Furthermore, a general type of table is provided in the automatic memory management of Lua, with which storage of arrays, hash tables, collections, objects, etc. can be achieved, which is suitable for storage and processing of feature data.
In one embodiment, before the step of sequentially summarizing the write operations into the preset script in step S320, the method further includes:
step S300: and writing the reading operation of each online training characteristic data into a preset script so that the processing instruction calls the online training characteristic data.
According to the characteristic data writing method for online training, the reading operation of each online training characteristic data is written into the preset script, and the preset script can call the characteristic data stored in the redis cluster through the reading operation so as to facilitate the subsequent processing instruction to retrain the characteristic data.
Particularly, after a certain time, the requirement of online training is changed, the training conditions or parameters of online training are changed, the processing instruction of the online training feature data can be modified through a redis client to meet the updated online training requirement, and the existing online feature data is retrained according to the current requirement of online training.
Taking online training feature data related to a live broadcast room as an example, obtaining time length feature data through online training of the watching time length of the live broadcast room, and carrying out online training on payment behaviors of a user to obtain payment feature data. Before the change of the demand, the score of the live broadcasting room is set in the preset script and is obtained only through online training of the duration characteristic data; however, after the demand changes, the processing instructions in the preset script can be modified by modifying the preset script to train the live room score online through the duration characteristic data and the payment characteristic data. At this time, not only the complicated operation instruction of requesting reading and writing is avoided, but also the flexibility of adjusting the training conditions and parameters of the online training can be improved.
The above embodiments illustrate the implementation of online training by executing a preset script, and the following embodiments illustrate the triggering execution of the preset script.
In one embodiment, the redis server is further configured to execute the updated preset script when it is detected that the preset script is updated.
According to the characteristic data writing method for online training, when the redis client uploads the new version of the preset script or directly modifies the preset script to obtain the updated preset script, the redis server can be directly triggered to directly execute the updated preset script, and the efficiency of online training is improved.
In one embodiment, the redis server is further configured to initiate execution of the preset script when the constraint condition is satisfied.
Constraint conditions can be configured in the redis server side, and when the constraint conditions are met, the redis server side is triggered to execute the preset script. The constraint may be a preset point in time or time interval, or a trigger condition. The triggering condition may be that an event occurs, for example, a preset script is updated, for example, when the constraint condition is that the preset script is updated, the updated preset script is automatically executed, and when the redis client uploads the updated preset script to cover the original stored preset script, the redis server is triggered to execute the covered preset script. Or the triggering condition is that the monitored characteristic data or parameter reaches a specified threshold value to trigger the redis server to execute the preset script. Taking feature data of online training of a live broadcasting room as an example, when the number of persons in the live broadcasting room is started to reach a monitoring threshold, executing a preset script by the redis server to perform online training under the condition.
According to the characteristic data writing method for online training, the constraint conditions are set, execution of the preset script is automatically triggered when the constraint conditions are met, the online training is triggered, and the efficiency of the online training is improved.
In one embodiment, the step of requesting the redis server to run the preset script includes:
step S331: uploading a preset script to a redis server;
step S332: and sending an execution request to the redis server to request the redis server to run the preset script.
According to the characteristic data writing method for online training, the preset script is uploaded to the redis server, and the preset script can be called only by sending an execution request, so that the redis server is triggered to run the preset script.
In one embodiment, the redis server is further configured to receive execution requests sent by other redis clients to run the preset script.
Other redis clients connected with the redis server may also run preset scripts by sending execution requests to trigger the redis server.
According to the characteristic data writing method for online training, the preset scripts stored in the redis server can be called by other redis clients, the preset scripts are allowed to be reused, the other redis clients can complete online training and writing of a plurality of characteristic data without summarizing a plurality of operation requests again, and the efficiency of online training is improved.
As shown in fig. 4, fig. 4 is a schematic structural diagram of an online training feature data writing system, which specifically includes a receiving module 410, a summarizing module 420, and an operating module 430, where:
a receiving module 410 for receiving a plurality of write operations for the online trained feature data.
The write operation is used for requesting the redis server to write the feature data into the redis cluster, and the receiving module 410 receives a plurality of write operations for the feature data trained online, so as to write a plurality of feature data obtained through online training into the redis cluster.
And the summarizing module 420 is configured to summarize each writing operation into a preset script in sequence, where the preset script configures in advance a processing instruction written into the online training feature data.
The processing instructions for online training of the feature data are configured in the preset script, and a plurality of writing operations are sequentially summarized to the preset script, so that the preset script can be used for starting the online training and updating the trained feature data. The summarized preset script can be sent to redis; or the preset script can be stored in redis in advance, and write operations are summarized into the preset script by modifying the preset script.
And the running module 430 is configured to request the redis server to run a preset script, so as to sequentially execute each write operation through the redis server and write the feature data of the online training in the redis cluster.
Requesting the redis server to run a preset script, triggering the redis server to execute a processing instruction of the online training feature data in the preset script, realizing the online training feature data by the redis server, and triggering the redis server to sequentially write a plurality of feature data into the redis cluster.
According to the characteristic data writing system for online training, through receiving a plurality of writing operations aiming at the characteristic data of online training, each writing operation is sequentially summarized into the preset script, wherein the preset script is pre-configured with a processing instruction for writing the characteristic data of online training, a redis server is requested to run the preset script, each writing operation is sequentially executed through the redis server, and the characteristic data of online training is written in a redis cluster; only by sending a plurality of requests once through a preset script and requesting the redis server once, network overhead and network time delay are reduced, writing of the characteristic data of a plurality of online training and online training of the characteristic data can be triggered, and the efficiency of online training of special data is improved.
In addition, the redis server side can execute the preset script as a whole, and other commands cannot be inserted in the execution process, so that race conditions are not required to be worried about in the writing process of the preset script, the execution of the preset script cannot be interrupted by a thread scheduling mechanism, and atomic operation is realized.
In one embodiment, the preset script is a script written in the Lua scripting language. Lua is a lightweight and compact scripting language written in standard C language and open in source code, and Lua script written in Lua scripting language is embedded in an application program, thereby providing flexible extension and customization functions for the application program. The Lua script has the characteristics of light weight and expandability, and supports process-oriented programming and functional programming. Furthermore, a general type of table is provided in the automatic memory management of Lua, with which storage of arrays, hash tables, collections, objects, etc. can be achieved, which is suitable for storage and processing of feature data.
The summarization module 420 is further configured to write the read operation of each online training feature data into a preset script, so that the processing instruction invokes the online training feature data.
According to the characteristic data writing system for online training, the reading operation of each online training characteristic data is written into the preset script, and the preset script can call the characteristic data stored in the redis cluster through the reading operation so as to facilitate the subsequent processing instruction to retrain the characteristic data.
Particularly, after a certain time, the requirement of online training is changed, the training conditions or parameters of online training are changed, the processing instruction of the online training feature data can be modified through a redis client to meet the updated online training requirement, and the existing online feature data is retrained according to the current requirement of online training. Taking online training feature data related to a live broadcast room as an example, obtaining time length feature data through online training of the watching time length of the live broadcast room, and carrying out online training on payment behaviors of a user to obtain payment feature data. Before the change of the demand, the score of the live broadcasting room is set in the preset script and is obtained only through online training of the duration characteristic data; however, after the demand changes, the processing instructions in the preset script can be modified by modifying the preset script to train the live room score online through the duration characteristic data and the payment characteristic data. At this time, not only the complicated operation instruction of requesting reading and writing is avoided, but also the flexibility of adjusting the training conditions and parameters of the online training can be improved.
In one embodiment, the redis server is further configured to execute the updated preset script when it is detected that the preset script is updated. When the redis client uploads the new version of the preset script or directly modifies the preset script to obtain the updated preset script, the redis server can be directly triggered to directly execute the updated preset script, and the efficiency of online training is improved.
In one embodiment, the redis server is further configured to initiate execution of the preset script when the constraint condition is satisfied. By setting the constraint conditions, the execution of the preset script is automatically triggered when the constraint conditions are met, the online training is triggered, and the efficiency of the online training is improved.
Constraint conditions can be configured in the redis server side, and when the constraint conditions are met, the redis server side is triggered to execute the preset script. The constraint may be a preset point in time or time interval, or a trigger condition. The triggering condition may be that an event occurs, for example, a preset script is updated, for example, when the constraint condition is that the preset script is updated, the updated preset script is automatically executed, and when the redis client uploads the updated preset script to cover the original stored preset script, the redis server is triggered to execute the covered preset script. Or the triggering condition is that the monitored characteristic data or parameter reaches a specified threshold value to trigger the redis server to execute the preset script. Taking feature data of online training of a live broadcasting room as an example, when the number of persons in the live broadcasting room is started to reach a monitoring threshold, executing a preset script by the redis server to perform online training under the condition.
In one embodiment, the running module 430 is further configured to upload a preset script to the redis server; and sending an execution request to the redis server to request the redis server to run the preset script.
According to the characteristic data writing system for online training, the preset script is uploaded to the redis server, and the preset script can be called only by sending an execution request, so that the redis server is triggered to run the preset script.
In one embodiment, the redis server is further configured to receive execution requests sent by other redis clients to run the preset script. Other redis clients connected with the redis server may also run preset scripts by sending execution requests to trigger the redis server.
According to the characteristic data writing system for online training, the preset scripts stored in the redis server can be called by other redis clients, the preset scripts are allowed to be reused, the other redis clients can complete online training and writing of a plurality of characteristic data without summarizing a plurality of operation requests again, and the efficiency of online training is improved.
For specific limitations regarding the online training feature data writing system, reference may be made to the above limitation of the online training feature data writing method, and no further description is given here. The various modules in the online training feature data writing system described above may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
As shown in fig. 5, fig. 5 is a schematic diagram of an internal structure of the computer device in one embodiment. The computer device includes a processor, a non-volatile storage medium, a memory, and a network interface connected by a system bus. The nonvolatile storage medium of the computer equipment stores an operating system, a database and a computer program, wherein the database can store a control information sequence, and the computer program can enable the processor to realize a characteristic data writing method of online training when being executed by the processor. The processor of the computer device is used to provide computing and control capabilities, supporting the operation of the entire computer device. The memory of the computer device may have stored therein a computer program which, when executed by the processor, causes the processor to perform a method of writing feature data for online training. The network interface of the computer device is for communicating with a terminal connection. It will be appreciated by those skilled in the art that the structure shown in FIG. 5 is merely a block diagram of some of the structures associated with the present inventive arrangements and is not limiting of the computer device to which the present inventive arrangements may be applied, and that a particular computer device may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.
In one embodiment, a computer device is presented, the computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the following steps when executing the computer program: receiving a plurality of write operations for feature data of the online training; sequentially summarizing each writing operation into a preset script, wherein the preset script is pre-configured with a processing instruction written in the processing instruction for training the characteristic data on line; requesting the redis server to run the preset script so as to sequentially execute each writing operation and write the characteristic data of the online training in the redis cluster through the redis server.
In one embodiment, the processor when executing the computer program further performs the steps of: and writing the reading operation of each online training characteristic data into the preset script so that the processing instruction calls the online training characteristic data.
In one embodiment, the step of requesting the redis server to run the preset script performed by the processor includes: uploading the preset script to the redis server;
and sending an execution request to the redis server to request the redis server to run the preset script.
In one embodiment, a computer readable storage medium is provided having a computer program stored thereon, which when executed by a processor, performs the steps of: receiving a plurality of write operations for feature data of the online training; sequentially summarizing each writing operation into a preset script, wherein the preset script is pre-configured with a processing instruction written in the processing instruction for training the characteristic data on line; requesting the redis server to run the preset script so as to sequentially execute each writing operation and write the characteristic data of the online training in the redis cluster through the redis server.
In one embodiment, the computer program when executed by the processor further performs the steps of: and writing the reading operation of each online training characteristic data into the preset script so that the processing instruction calls the online training characteristic data.
In one embodiment, the step of requesting the redis server to run the preset script when the computer program is executed by the processor includes: uploading the preset script to the redis server;
and sending an execution request to the redis server to request the redis server to run the preset script.
It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, these steps are not necessarily performed in order as indicated by the arrows. The steps are not strictly limited in order and may be performed in other orders, unless explicitly stated herein. Moreover, at least some of the steps in the flowcharts of the figures may include a plurality of sub-steps or stages that are not necessarily performed at the same time, but may be performed at different times, the order of their execution not necessarily being sequential, but may be performed in turn or alternately with other steps or at least a portion of the other steps or stages.
The foregoing is only a partial embodiment of the present application, and it should be noted that it will be apparent to those skilled in the art that modifications and adaptations can be made without departing from the principles of the present application, and such modifications and adaptations should and are intended to be comprehended within the scope of the present application.
Claims (9)
1. The characteristic data writing method for online training is characterized by comprising the following steps of:
receiving a plurality of write operations for feature data of the online training;
writing the reading operation of each online training characteristic data into a preset script, so that the preset script pre-configures the processing instruction written into the online training characteristic data to call the online training characteristic data, and sequentially summarizing each writing operation into the preset script;
requesting a redis server to run the preset script to sequentially execute each writing operation and write the characteristic data of online training in a redis cluster through the redis server, wherein the redis server executes the preset script as a whole.
2. The method for writing feature data for online training according to claim 1, wherein the preset script is a script written in Lua script language.
3. The method for writing feature data in online training according to claim 1, wherein the redis server is further configured to start executing the preset script when a constraint condition is satisfied.
4. The method for writing feature data of online training according to claim 1, wherein the step of requesting the redis server to run the preset script comprises:
uploading the preset script to the redis server;
and sending an execution request to the redis server to request the redis server to run the preset script.
5. The method for writing feature data for online training according to claim 4, wherein the redis server is further configured to receive an execution request sent by another redis client to run the preset script.
6. The method for writing feature data in online training according to claim 1, wherein the redis server is further configured to execute the updated preset script when it is detected that the preset script is updated.
7. A feature data writing system for on-line training, comprising:
a receiving module for receiving write operations for a plurality of feature data for online training;
the summarizing module is used for writing the reading operation of each online training characteristic data into a preset script so that the preset script is preconfigured with a processing instruction written into the online training characteristic data to call the online training characteristic data, and sequentially summarizing each writing operation into the preset script, wherein a redis server side executes the preset script as a whole;
the operation module is used for requesting the redis server to operate the preset script so as to sequentially execute each writing operation and write the characteristic data of the online training in the redis cluster through the redis server.
8. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the on-line training characteristic data writing method of any of claims 1 to 6 when the computer program is executed by the processor.
9. A computer readable storage medium having stored thereon a computer program, characterized in that the computer program when executed by a processor realizes the steps of the characteristic data writing method of online training of any of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911319666.2A CN111143361B (en) | 2019-12-19 | 2019-12-19 | Characteristic data writing method and system for online training |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911319666.2A CN111143361B (en) | 2019-12-19 | 2019-12-19 | Characteristic data writing method and system for online training |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111143361A CN111143361A (en) | 2020-05-12 |
| CN111143361B true CN111143361B (en) | 2023-11-14 |
Family
ID=70518961
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911319666.2A Active CN111143361B (en) | 2019-12-19 | 2019-12-19 | Characteristic data writing method and system for online training |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111143361B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2006352845A (en) * | 2005-05-20 | 2006-12-28 | Ricoh Co Ltd | Image handling apparatus, image processing system, image processing control method, and image processing control program program |
| EP2833282A1 (en) * | 2013-07-31 | 2015-02-04 | Sap Se | Data aggregation in an in-memory DBMS |
| CN108491227A (en) * | 2018-03-09 | 2018-09-04 | 深圳市富途网络科技有限公司 | A kind of security configuration system and method based on lua and redis |
| CN108958948A (en) * | 2018-06-19 | 2018-12-07 | 北京云枢网络科技有限公司 | A kind of timer event processing method, database and timer |
| CN109814478A (en) * | 2019-01-14 | 2019-05-28 | 浙江工业大学 | Virtual Debugging system based on iOpenWorks |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9785489B1 (en) * | 2014-12-18 | 2017-10-10 | Amazon Technologies, Inc. | Secure script execution using sandboxed environments |
-
2019
- 2019-12-19 CN CN201911319666.2A patent/CN111143361B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2006352845A (en) * | 2005-05-20 | 2006-12-28 | Ricoh Co Ltd | Image handling apparatus, image processing system, image processing control method, and image processing control program program |
| EP2833282A1 (en) * | 2013-07-31 | 2015-02-04 | Sap Se | Data aggregation in an in-memory DBMS |
| CN108491227A (en) * | 2018-03-09 | 2018-09-04 | 深圳市富途网络科技有限公司 | A kind of security configuration system and method based on lua and redis |
| CN108958948A (en) * | 2018-06-19 | 2018-12-07 | 北京云枢网络科技有限公司 | A kind of timer event processing method, database and timer |
| CN109814478A (en) * | 2019-01-14 | 2019-05-28 | 浙江工业大学 | Virtual Debugging system based on iOpenWorks |
Non-Patent Citations (1)
| Title |
|---|
| 一种使用Node.js构建的分布式数据流日志服务系统;张煜;;计算机系统应用(02);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111143361A (en) | 2020-05-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111625452B (en) | Flow playback method and system | |
| US9489189B2 (en) | Dynamically generate and execute a context-specific patch installation procedure on a computing system | |
| CN106557470B (en) | Data extraction method and device | |
| CN112000741B (en) | Internal and external network data exchange system, method, device, computer equipment and medium | |
| US11206316B2 (en) | Multiple model injection for a deployment cluster | |
| US10078509B2 (en) | Method and system for processing lifelong learning of terminal and apparatus | |
| CN109862101B (en) | Cross-platform application starting method and device, computer equipment and storage medium | |
| CN106874281B (en) | Method and device for realizing database read-write separation | |
| US10942771B2 (en) | Method, apparatus and system for multi-module scheduling | |
| CN111866099A (en) | Method, device, system, equipment and storage medium for downloading mirror image file | |
| CN108255735B (en) | Associated environment testing method, electronic device and computer readable storage medium | |
| US9325576B2 (en) | Dynamic server to server configuration and initialization | |
| US8539048B2 (en) | Electronic device and method for loading configuration files using the same | |
| US20140380094A1 (en) | Avoiding restart on error in data integration | |
| CN114979029A (en) | Control method, device, equipment and storage medium of virtual robot | |
| CN111143361B (en) | Characteristic data writing method and system for online training | |
| US20230409295A1 (en) | Building base applications with user interface micro frontends | |
| US20150222485A1 (en) | Dynamic server configuration and initialization | |
| US20210004387A1 (en) | Changing the number of replicas of a pod based on size of message queue | |
| CN111045787B (en) | Rapid continuous experiment method and system | |
| CN113360558A (en) | Data processing method, data processing device, electronic device, and storage medium | |
| CN112433749A (en) | Application gray level publishing method and device, server and client | |
| CN108073638B (en) | Data diagnosis method and device | |
| CN111600949B (en) | Data transmission method, device, equipment and computer readable storage medium | |
| CN111182012B (en) | Intelligent equipment management method, device and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20210111 Address after: 511442 3108, 79 Wanbo 2nd Road, Nancun Town, Panyu District, Guangzhou City, Guangdong Province Applicant after: GUANGZHOU CUBESILI INFORMATION TECHNOLOGY Co.,Ltd. Address before: 29th floor, building B-1, Wanda Plaza, Wanbo business district, Nancun Town, Panyu District, Guangzhou City, Guangdong Province Applicant before: GUANGZHOU HUADUO NETWORK TECHNOLOGY Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20200512 Assignee: GUANGZHOU HUADUO NETWORK TECHNOLOGY Co.,Ltd. Assignor: GUANGZHOU CUBESILI INFORMATION TECHNOLOGY Co.,Ltd. Contract record no.: X2021440000054 Denomination of invention: Writing method and system of characteristic data for online training License type: Common License Record date: 20210208 |
|
| EE01 | Entry into force of recordation of patent licensing contract | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |