CN110704468A - Data updating method and device and controller - Google Patents
Data updating method and device and controller Download PDFInfo
- Publication number
- CN110704468A CN110704468A CN201910988305.0A CN201910988305A CN110704468A CN 110704468 A CN110704468 A CN 110704468A CN 201910988305 A CN201910988305 A CN 201910988305A CN 110704468 A CN110704468 A CN 110704468A
- Authority
- CN
- China
- Prior art keywords
- data
- updating
- user
- dirty
- storing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/60—Generating or modifying game content before or while executing the game program, e.g. authoring tools specially adapted for game development or game-integrated level editor
- A63F13/69—Generating or modifying game content before or while executing the game program, e.g. authoring tools specially adapted for game development or game-integrated level editor by enabling or updating specific game elements, e.g. unlocking hidden features, items, levels or versions
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the application provides a data updating method, a data updating device and a controller, which relate to the technical field of communication and are used for marking dirty data generated in the process of using a service by a user; and responding to the triggering operation of the updating event, and storing the marked dirty data updating to the target storage position so as to improve the speed of data updating.
Description
Technical Field
The application belongs to the technical field of communication, and particularly relates to a data updating method, a data updating device and a controller.
Background
With the development of game technology, users have higher and higher requirements on timeliness of data updating, but at the same time, the data volume of interaction between the client and the server is increased.
In the prior art, a full-volume updating model is generally adopted when data updating is performed, namely, all data are refreshed based on timing or an updating trigger event, but only part of the updated data is substantially modified, so that a large amount of unnecessary data is updated and stored, and the problems of long updating time, large updated data volume and low updating efficiency are caused. Therefore, the contradiction between the requirement of storing a large amount of data and the requirement of completing game data updating in a relatively short time is increasingly prominent, and great technical difficulty is brought to timely updating and storing of the data.
Disclosure of Invention
In order to solve the contradiction between the large data storage between the client and the server and the requirement for completing data updating in a lower time in the prior art at least to a certain extent, the application provides a data updating method, a data updating device and a controller, which are used for improving the data updating speed.
In order to achieve the purpose, the following technical scheme is adopted in the application:
in a first aspect, a data updating method is provided, including:
marking dirty data generated in the process of using the service by the user;
and responding to the trigger operation of the updating event, and storing the marked dirty data updating to the target storage position.
In a second aspect, there is provided a data updating apparatus comprising:
the data marking module is used for marking dirty data generated in the process of using the service by the user;
and the data updating module is used for responding to the triggering operation of the updating event and storing the marked dirty data update to the target storage position.
The third aspect provides a controller for executing the data updating method.
According to the data updating method, the data updating device and the controller provided by the embodiment of the invention, in the process that a user logs in a client to use a service, dirty data generated in the process that the user uses the service is marked to distinguish the subsequent data needing to be updated and stored from other data needing not to be updated and stored, and after an updating event is triggered, the marked dirty data can be updated and stored to a target storage position without updating and storing full data, so that the data updating speed is increased.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments or related technologies of the present application, the drawings needed to be used in the description of the embodiments or related technologies are briefly introduced below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a flow chart of a data update method in an embodiment of the present application;
fig. 2 is a schematic structural diagram of a data updating apparatus in an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be described in detail below. It is to be understood that the embodiments described are only a few embodiments of the present application and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the examples given herein without making any creative effort, shall fall within the protection scope of the present application.
To facilitate understanding of the technical solutions of the present invention, the related terms in the technical solutions are explained as follows:
dirty data: data that has been modified in memory but not stored in disk in time.
Updating the dirty data: dirty data is stored to a database or disk.
And Map: map is an association container. It provides one-to-one (where the first may be referred to as a key, each key can only appear once in the map, and the second may be referred to as the value of the key) data processing capability.
And (4) incremental updating: when updating, only the data needing to be changed is updated, and the data which does not need to be updated or is updated does not need to be updated repeatedly, and incremental updating is opposite to full updating.
Redis: the novel non-relational database (NoSQL) supports various data types and has extremely high reading and writing speed.
Example one
In one embodiment of the present application, the present application provides a data updating method, and an execution subject of the method may be a Server in a Client/Server (C/S) architecture. As shown in fig. 1, the method comprises the steps of:
and S110, marking dirty data generated in the process of using the service by the user.
The user performs service interaction with the server by logging in the client, and in the process of using the service by the user, because the user modifies data at the client or the server modifies the data, the data is cached in the memory of the server to form dirty data. And the data modified by the user at the client can be fed back to the server through service interaction.
In order to distinguish these dirty data from user data that has completed a data update, these dirty data may be specially marked.
In one embodiment, the data fields corresponding to dirty data may be stored to the first data table of the user.
Specifically, to mark dirty data generated during the process of using a service by each user, a first data table may be created for the user when the user uses the service for the first time. For example, in units of service types specifically used by a user, when a corresponding service entity is created for the user, the first data table is created for each service entity of the user. The server can store data fields corresponding to dirty data generated in the user service process to a first data table of the user. For example as shown in table 1:
TABLE 1 first data sheet
In table 1, dirtyMapFlag is a Map-type data structure, and each row is a < key, value > key-value pair. Where dirtyField may be of string type, may represent some property for the user-created game entity, all modified fields (dirty data) are stored in dirtyMapFlag, and the key value is "fire". Value may be of the pool type, representing whether the key field is modified.
And S120, responding to the trigger operation of the updating event, and updating and storing the marked dirty data to the target storage position.
The update event is any event that can trigger the implementation of updating dirty data, and may include, but is not limited to: and at least one of time period arrival and service version number change. Wherein, the time period reaching refers to periodically triggering the operation of updating the dirty data with a fixed time length; the service version number change means that when a user enters a new service operation progress stage, the corresponding service version number is changed, and when the service version number is changed, an operation of updating dirty data can be triggered once.
Storing the marked dirty data update to the target storage location may be performed once after each update event is triggered.
In one embodiment, the process of updating dirty data may be implemented by performing the following steps:
step one, constructing a second data table of a user;
for example, as shown in table 2, dirtyMap is a Map-type data structure, and each row is a < key, value > key-value pair. Where dirtyField may be of string type and may represent some property for the user-created game entity, all modified field data (dirty data) will be stored in dirtyMap (i.e., value). Value may be an interface type representing the Value of the key field after it has been modified.
TABLE 2 second data Table
And step two, extracting the data of the data field recorded in the first data table from the full data and storing the data in the second data table.
The total data can be all data generated in the process of using the service by the user and needing to be updated and stored in a database or a disk for permanence, and the total data at least comprises dirty data.
The dirty data may be looked up from the full amount of data based on data fields of the dirty data recorded in the first data table. The found dirty data may be stored in the second data table first.
After the data of the data field recorded in the first data table is extracted from the full data and stored in the second data table, the data recorded in the first data table can be cleared. The first data table may then start recording data fields of dirty data newly generated by the user during the service usage.
And step three, updating and storing the data in the second data table to a database.
In a specific embodiment, the found dirty data may be stored in the second data table, and after all the dirty data are found, the dirty data are updated and stored together from the second data table to the database, so that the efficiency of data storage may be improved. Wherein, the database can include: a remote dictionary service database (Redis).
After the data in the second data table is updated and stored in the database, the second data table can be deleted, so that the occupation of system resources is reduced. In this way, the second data table is only created when the update dirty data operation is performed and is deleted immediately after the update is completed, thereby reducing the maintenance cost of the server for the second data table.
In addition, in practical application scenarios, sometimes new dirty data may not actually be generated after an update event is triggered. For example, the data field recorded in the first data table is empty. Then the operation of updating the dirty data need not be performed at this point. Therefore, after the update event is triggered, whether the data field recorded in the first data table is empty or not can be judged; if not, indicating that dirty data is generated, executing an operation of updating and storing the marked dirty data to a target storage position, wherein the specific updating and storing process can refer to the steps; if so, indicating that no dirty data is generated, ending the dirty data updating operation.
According to the data updating method provided by the embodiment of the invention, in the process that the user logs in the client to use the service, dirty data generated in the process that the user uses the service is marked to distinguish the data which needs to be updated and stored subsequently from other data which does not need to be updated and stored, and after an updating event is triggered, the marked dirty data can be updated and stored to a target storage position without updating and storing the whole amount of data, so that the data updating speed is increased.
Example two
In order to implement the data updating method in a coordinated manner, an embodiment of the present invention provides a data updating apparatus. As shown in fig. 2, the data updating apparatus includes:
a data marking module 210, configured to mark dirty data generated during a service using process of a user;
and a data update module 220, configured to store the marked dirty data update to the target storage location in response to a trigger operation of the update event.
In an embodiment, the data marking module 210 is specifically configured to store the data field corresponding to the dirty data to the first data table of the user.
In an embodiment, the data updating module 220 is specifically configured to construct a second data table of the user; extracting data of data fields recorded in a first data table from the full data and storing the data in a second data table; and updating and storing the data in the second data table to the database.
In a specific embodiment, the data updating apparatus shown in fig. 2 may further include: the first data table processing module is used for creating the first data table for the user when the user uses the service for the first time; and after the data of the data fields recorded in the first data table is extracted from the full data and stored in the second data table, clearing the data recorded in the first data table.
In an embodiment, in the data updating apparatus shown in fig. 2, the data updating module 220 may be further configured to delete the second data table after the data in the second data table is updated and stored in the database.
In an embodiment, in the data updating apparatus shown in fig. 2, the data updating module 220 may be further configured to respond to a triggering operation of an updating event, and determine whether a data field recorded in the first data table is empty; if not, executing the operation of updating and storing the marked dirty data to the target storage position; if so, ending the dirty data updating operation.
In one embodiment, the update event may include: and at least one of time period arrival and service version number change.
In one embodiment, the database may include: a remote dictionary service database.
According to the data updating device provided by the embodiment of the invention, in the process that a user logs in a client to use a service, dirty data generated in the process that the user uses the service is marked so as to distinguish the subsequent data needing to be updated and stored from other data needing not to be updated and stored, and after an updating event is triggered, the marked dirty data can be updated and stored to a target storage position without updating and storing full data, so that the data updating speed is increased.
Further, the present embodiment also provides a controller, configured to execute any one of the data updating methods described above.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.
It is understood that the same or similar parts in the above embodiments may be mutually referred to, and the same or similar parts in other embodiments may be referred to for the content which is not described in detail in some embodiments.
It should be noted that, in the description of the present application, the terms "first", "second", etc. are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. Further, in the description of the present application, the meaning of "a plurality" means at least two unless otherwise specified.
Any process or method descriptions in flow charts or otherwise described herein may be understood as: represents modules, segments or portions of code which include one or more executable instructions for implementing specific logical functions or steps of a process, and the scope of the preferred embodiments of the present application includes other implementations in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the embodiments of the present application.
It should be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.
In addition, functional units in the embodiments of the present application may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium.
The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc.
In the description herein, reference to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
Although embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application, and that variations, modifications, substitutions and alterations may be made to the above embodiments by those of ordinary skill in the art within the scope of the present application.
Claims (10)
1. A method for updating data, comprising:
marking dirty data generated in the process of using the service by the user;
and responding to the trigger operation of the updating event, and storing the marked dirty data updating to the target storage position.
2. The method of claim 1, wherein the marking dirty data generated during the usage of the service by the user comprises:
and storing the data field corresponding to the dirty data to the first data table of the user.
3. The method of claim 2, wherein the storing the marked dirty data update to a target storage location comprises:
constructing a second data table of the user;
extracting data of data fields recorded in the first data table from the full data and storing the data in the second data table;
and updating and storing the data in the second data table to a database.
4. The method of claim 3, further comprising:
when the user uses the service for the first time, creating the first data table for the user;
and after the data of the data field recorded in the first data table is extracted from the full data and stored in the second data table, clearing the data recorded in the first data table.
5. The method of claim 3, wherein after storing the data update in the second data table to a database, further comprising:
deleting the second data table.
6. The method according to any of claims 3-5, wherein the triggering operation in response to an update event further comprises:
judging whether a data field recorded in the first data table is empty or not;
if not, performing an operation of updating and storing the marked dirty data to a target storage position;
if so, ending the dirty data updating operation.
7. The method of claim 1, wherein the update event comprises: and at least one of time period arrival and service version number change.
8. The method of claim 3, wherein the database comprises: a remote dictionary service database.
9. A data update apparatus, comprising:
the data marking module is used for marking dirty data generated in the process of using the service by the user;
and the data updating module is used for responding to the triggering operation of the updating event and storing the marked dirty data update to the target storage position.
10. A controller for performing the data update method of any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910988305.0A CN110704468A (en) | 2019-10-17 | 2019-10-17 | Data updating method and device and controller |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910988305.0A CN110704468A (en) | 2019-10-17 | 2019-10-17 | Data updating method and device and controller |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110704468A true CN110704468A (en) | 2020-01-17 |
Family
ID=69200397
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910988305.0A Pending CN110704468A (en) | 2019-10-17 | 2019-10-17 | Data updating method and device and controller |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110704468A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112131240A (en) * | 2020-09-30 | 2020-12-25 | 腾讯科技（深圳）有限公司 | Dirty data processing method and device, storage medium and electronic equipment |
| CN112473129A (en) * | 2020-11-24 | 2021-03-12 | 杭州电魂网络科技股份有限公司 | Method, system, electronic device and storage medium for incremental data acquisition |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102725752A (en) * | 2011-10-20 | 2012-10-10 | 华为技术有限公司 | Method and device for processing dirty data |
| CN108399047A (en) * | 2017-02-06 | 2018-08-14 | 中兴通讯股份有限公司 | A kind of flash memory file system and its data managing method |
| CN109408522A (en) * | 2018-09-28 | 2019-03-01 | 阿里巴巴集团控股有限公司 | A kind of update method and device of user characteristic data |
| CN110019340A (en) * | 2017-12-08 | 2019-07-16 | 北京新媒传信科技有限公司 | Data storage/read method, apparatus and system |
-
2019
- 2019-10-17 CN CN201910988305.0A patent/CN110704468A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102725752A (en) * | 2011-10-20 | 2012-10-10 | 华为技术有限公司 | Method and device for processing dirty data |
| CN108399047A (en) * | 2017-02-06 | 2018-08-14 | 中兴通讯股份有限公司 | A kind of flash memory file system and its data managing method |
| CN110019340A (en) * | 2017-12-08 | 2019-07-16 | 北京新媒传信科技有限公司 | Data storage/read method, apparatus and system |
| CN109408522A (en) * | 2018-09-28 | 2019-03-01 | 阿里巴巴集团控股有限公司 | A kind of update method and device of user characteristic data |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112131240A (en) * | 2020-09-30 | 2020-12-25 | 腾讯科技（深圳）有限公司 | Dirty data processing method and device, storage medium and electronic equipment |
| CN112131240B (en) * | 2020-09-30 | 2023-09-26 | 腾讯科技（深圳）有限公司 | Dirty data processing method and device, storage medium and electronic equipment |
| CN112473129A (en) * | 2020-11-24 | 2021-03-12 | 杭州电魂网络科技股份有限公司 | Method, system, electronic device and storage medium for incremental data acquisition |
| CN112473129B (en) * | 2020-11-24 | 2024-03-26 | 杭州电魂网络科技股份有限公司 | Incremental data acquisition method, system, electronic device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9189506B2 (en) | Database index management | |
| CN103136243B (en) | File system duplicate removal method based on cloud storage and device | |
| US11232071B2 (en) | Regressable differential data structures | |
| CN108319654B (en) | Computing system, cold and hot data separation method and device, and computer readable storage medium | |
| CN105373541B (en) | The processing method and system of the data operation request of database | |
| CN111046034A (en) | Method and system for managing memory data and maintaining data in memory | |
| CN105989160A (en) | Memory data persistence method and apparatus for Redis database | |
| CN106844089B (en) | Method and equipment for recovering tree data storage | |
| CN110597821B (en) | Method and device for changing database table structure | |
| CN103164525A (en) | Method and device for WEB application release | |
| CN105900093A (en) | Keyvalue database data table updating method and data table updating device | |
| CN110704468A (en) | Data updating method and device and controller | |
| CN108241676A (en) | Realize the method and apparatus that data synchronize | |
| CN105447168A (en) | Method for restoring and recombining fragmented files in MP4 format | |
| US10394844B2 (en) | Integrating co-deployed databases for data analytics | |
| CN103377292B (en) | Database result set caching method and device | |
| CN104281673A (en) | Cache building system and method for database | |
| CN104268097B (en) | A kind of metadata processing method and system | |
| CN102937923B (en) | A kind of method of automatic preservation user interface state | |
| CN109471860B (en) | Large-scale charging pile data processing method and device for electric vehicle charging network | |
| CN112783927B (en) | Database query method and system | |
| KR101588375B1 (en) | Method and system for managing database | |
| CN109710454A (en) | A kind of cloud host snapshot method and device | |
| CN116339643B (en) | Formatting method, formatting device, formatting equipment and formatting medium for disk array | |
| CN114518848B (en) | Method, device, equipment and medium for processing stored data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200117 |