CN111782667B - MongoDB updated data driving method, system and storage medium - Google Patents
MongoDB updated data driving method, system and storage medium Download PDFInfo
- Publication number
- CN111782667B CN111782667B CN202010574553.3A CN202010574553A CN111782667B CN 111782667 B CN111782667 B CN 111782667B CN 202010574553 A CN202010574553 A CN 202010574553A CN 111782667 B CN111782667 B CN 111782667B
- Authority
- CN
- China
- Prior art keywords
- data
- module
- field
- database
- updating
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 26
- 230000010076 replication Effects 0.000 claims description 3
- 230000001960 triggered effect Effects 0.000 claims description 3
- 238000004891 communication Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 241000282375 Herpestidae Species 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- BQJCRHHNABKAKU-KBQPJGBKSA-N morphine Chemical compound O([C@H]1[C@H](C=C[C@H]23)O)C4=C5[C@@]12CCN(C)[C@@H]3CC5=CC=C4O BQJCRHHNABKAKU-KBQPJGBKSA-N 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a driving method, a system and a storage medium for MongoDB updated data, wherein in the inquiring stage, original data in a database is directly adopted, so that the memory space is saved, in the inquiring stage, in order to facilitate the operation of the data in the prior art, a layer of object array is packaged for the data in the database, and once the inquiring result is more, a larger memory is occupied; meanwhile, in the driving method of MongoDB updated data, in the updating stage, the combined sentences are directly updated by the original bulk operation, so that the using cost of CPU is greatly reduced.
Description
Technical Field
The present invention relates to database technologies, and in particular, to a method, a system, and a storage medium for driving MongoDB update data.
Background
With the popularity of the non-relational database MongoDB in the industry, more and more ODMs such as Morphia, pyMongo, mongoose also provide gradually perfected functions so that developers can manipulate documents stored in the database like normal objects. Taking Monoose as an example, when a certain document in a database needs to be updated, a target object is found through a Monoose query method, and because the object inherits a Model class in Monoose, the object can be smoothly operated, and then a save method of the object is called to be saved in the database.
While updating data with ODM is straightforward at the code level, these are based on sacrificing performance. In the scene of updating a plurality of documents, each object has a plurality of default example methods of Monoose, which leads to a significant increase in memory overhead, when a single document needs to modify data in an embedded array, monoose directly covers an original array with a new array, and when the array is large, the overwriting operation brings huge cpu overhead.
Therefore, it is necessary to design a data update scheme that is applicable to the non-relational database MongoDB and can save memory and cpu overhead.
Disclosure of Invention
In view of the above-described deficiencies of the prior art, the present invention aims to: provided is a driving method of MongoDB update data capable of saving memory and cpu overhead.
In order to achieve the above object, the present invention provides the following technical solutions:
A driving method of MongoDB update data, comprising the steps of:
S1: inquiring the data which needs to be updated into target data in the database, and caching the inquired data into a memory;
S2: checking whether each object in an outermost object array of data in a memory contains an_id field;
If the_id field is included, directly entering step S3; if the _id field is not contained, assigning a value to the _id field, combining a statement for assigning the _id field, and updating the data which needs to be updated in the database according to the statement for assigning the _id field, so that the data is consistent with the data in the memory;
S3: copying data in a part of memory, and carrying out updating operation on one part of data so that the data is updated into the target data;
s4: comparing the two data in the memory and combining sentences for updating the database;
s5: and updating corresponding data in the database into the target data according to the statement used for updating the database.
According to a specific embodiment, in the driving method of the MongoDB update data of the present invention, a statement for updating a database is executed as a parameter and is transferred into a bulk handle, so as to update corresponding data in the database to the target data.
According to a specific embodiment, in the method for driving MongoDB update data of the present invention, corresponding operators are selected according to the data type and operation type of an object, and the selected operators are added to a combined sentence.
In a specific implementation aspect, the present invention further provides a driving system for updating data of MongoDB, which includes:
the memory module is used for caching data;
the query module is used for querying the data which needs to be updated into the target data in the database, and caching the queried data into the memory module;
A field checking module, configured to determine whether each object in the outermost object array of the data in the memory module includes an_id field; if the_id field is included, triggering a copy module to work; if the _id field is not contained, a field assignment module, a statement combination module, a database update module and a replication module are triggered to work in sequence;
The copying module is used for copying data in one copy of the memory module according to the triggering of the field checking module and triggering the statement combination module to work;
a field assignment module, configured to assign a value to the_id field according to the triggering of the field checking module;
The statement combination module is used for combining the statement for assigning the_id field according to the triggering of the field checking module and combining the statement for updating the database according to the triggering of the copying module;
The data updating module is used for updating one copied data so that the data is updated into the target data;
And the database updating module is used for updating corresponding data in the database according to the statement for assigning the_id field and the statement for updating the database.
In an aspect of the present invention, there is also provided a readable storage medium having stored thereon one or more programs which when executed by one or more processors implement the driving method of MongoDB update data of the present invention.
Compared with the prior art, the invention has the beneficial effects that:
in the method for driving MongoDB updated data, original data in a database is directly adopted in a query stage, so that the memory space is saved, a layer of object array is packaged for the data in the database in order to facilitate the operation of the data in the query stage in the prior art, and once the query result is more, larger memory is occupied; in the driving method of MongoDB updated data, in the updating stage, the combined sentences are directly updated by the original bulk operation, so that the using cost of CPU is greatly reduced.
Drawings
FIG. 1 is a flow chart of the method of the present invention;
Fig. 2 is a schematic diagram of the system of the present invention.
Detailed Description
Other advantages and effects of the present invention will become apparent to those skilled in the art from the following disclosure, which describes the embodiments of the present invention with reference to specific examples. The invention may be practiced or carried out in other embodiments that depart from the specific details, and the details of the present description may be modified or varied from the spirit and scope of the present invention.
As shown in fig. 1, the driving method of MongoDB update data of the present invention includes the steps of:
S1: inquiring the data which needs to be updated into target data in the database, and caching the inquired data into a memory;
S2: checking whether each object in an outermost object array of data in a memory contains an_id field;
If the_id field is included, directly entering step S3; if the _id field is not contained, assigning a value to the _id field, combining a statement for assigning the _id field, and updating the data which needs to be updated in the database according to the statement for assigning the _id field, so that the data is consistent with the data in the memory;
S3: copying data in a part of memory, and carrying out updating operation on one part of data so that the data is updated into the target data;
s4: comparing the two data in the memory and combining sentences for updating the database;
s5: and updating corresponding data in the database into the target data according to the statement used for updating the database.
Specifically, in the driving method of MongoDB updated data, sentences used for updating a database are transmitted into a bulk handle as parameters to be executed, so that corresponding data in the database are updated into the target data.
In addition, in the driving method of MongoDB updated data, the mode of combining corresponding sentences is as follows: according to the data type and the operation type of the object, selecting a corresponding operator, and adding the selected operator to the combined sentence.
Taking node. Js as an example, the original data A1, A2, A3 … in MongoDB need to be updated to B1, B2, B3 … according to a certain rule. Then, the driving method of the MongoDB updating data based on the invention comprises the following specific implementation steps:
(1) And inquiring the data A1, A2 and A3 … which need to be updated at the time to the memory, wherein the data is an Object corresponding to the document in the JSON format in the database.
(2) For each data A1, A2, A3 … in the memory that needs to be updated, the outermost object array, that is, the first level key and the value, are checked, and the value characterizes the condition of the object array. Specifically, for each object array in the value, if the_id field is not included, a_id field is assigned to the value, which is ObjectId in the MongoDB, and update statements of the_id are set in combination, and at this time, the data in the memory becomes C1, C2, and C3 ….
(3) The data A1, A2 and A3 … in the database are updated by the statement of the set_id in the last step, so that the data A1, A2 and A3 are consistent with the data C1, C2 and C3 … in the memory, namely, the data in the database are updated to be C1, C2 and C3 … at the moment.
(4) And copying the data in the memory, performing updating operation on the data, and updating one data C1, C2 and C3 … into target data B1, B2 and B3 …, so that two data are stored in the memory, namely the data C1, C2 and C3 … before updating and the final data B1, B2 and B3 … after updating.
(5) And comparing the two data in the memory to combine the update statement of the database.
(6) The database is updated with the update statement of the last step so that the data C1, C2, C3 … in the database is also updated to B1, B2, B3 ….
In practice, the set_id field operation in step (2) (3) aims at: providing prompt information for combining the update sentences in the step (5), and when comparing the values of a certain object array of new and old data, wherein the objects in the original database are provided with an_id field, and the new objects have no_id field, then a push operation is adopted. Because there are only String, number, boolean, array, object data types in the Javascript, for the value of the Array type, we can update its value by using $push and $pull operators, and for the other four types we can update its value by using $set and $unset operators, so that in the case of two data sets, the corresponding update statement can be completely combined, so that the old data is updated into new data, and the purpose of step (5) is to obtain an Array of such update statement containing these 4 update operators, and to transfer the Array into bulk handle as parameters for execution, thus completing the update of the data in the database.
As shown in fig. 2, in an aspect of the present invention, there is further provided a driving system for MongoDB update data, including:
the memory module is used for caching data;
the query module is used for querying the data which needs to be updated into the target data in the database, and caching the queried data into the memory module;
A field checking module, configured to determine whether each object in the outermost object array of the data in the memory module includes an_id field; if the_id field is included, triggering a copy module to work; if the _id field is not contained, a field assignment module, a statement combination module, a database update module and a replication module are triggered to work in sequence;
The copying module is used for copying data in one copy of the memory module according to the triggering of the field checking module and triggering the statement combination module to work;
a field assignment module, configured to assign a value to the_id field according to the triggering of the field checking module;
The statement combination module is used for combining the statement for assigning the_id field according to the triggering of the field checking module and combining the statement for updating the database according to the triggering of the copying module;
The data updating module is used for updating one copied data so that the data is updated into the target data;
And the database updating module is used for updating corresponding data in the database according to the statement for assigning the_id field and the statement for updating the database.
In an aspect of the present invention, there is further provided a readable storage medium having one or more programs stored thereon, wherein the one or more programs, when executed by one or more processors, implement the method for driving MongoDB update data of the present invention.
It should be understood that the system disclosed in the present invention may be implemented in other manners. For example, the modules may be divided into only one logic function, and there may be other manners of dividing the modules when actually implemented, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted or not performed. In addition, the communication connection between modules may be an indirect coupling or communication connection via some interfaces, devices or units, and may be in electrical or other forms.
In addition, each functional module in the embodiments of the present invention may be integrated in one processing unit, or each module may exist alone physically, or two or more modules may be integrated in one processing unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied essentially or in part or all of the technical solution or in part in the form of a software product stored in a storage medium, including instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a usb disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a removable hard disk, a magnetic disk, or an optical disk, or other various media capable of storing program codes.
Claims (5)
1. A driving method of MongoDB update data, comprising the steps of:
S1: inquiring the data which needs to be updated into target data in the database, and caching the inquired data into a memory;
S2: checking whether each object in an outermost object array of data in a memory contains an_id field;
If the_id field is included, directly entering step S3; if the _id field is not contained, assigning a value to the _id field, combining a statement for assigning the _id field, and updating the data which needs to be updated in the database according to the statement for assigning the _id field, so that the data is consistent with the data in the memory;
S3: copying data in a part of memory, and carrying out updating operation on one part of data so that the data is updated into the target data;
s4: comparing the two data in the memory and combining sentences for updating the database;
s5: and updating corresponding data in the database into the target data according to the statement used for updating the database.
2. The driving method of MongoDB update data according to claim 1, wherein a statement for updating a database is executed as a parameter incoming bulk handle to update corresponding data in the database to the target data.
3. The driving method of MongoDB update data according to claim 2, wherein a corresponding operator is selected according to a data type and an operation type of an object, and the selected operator is added to the combined sentence.
4. A driving system of MongoDB update data, comprising:
the memory module is used for caching data;
the query module is used for querying the data which needs to be updated into the target data in the database, and caching the queried data into the memory module;
A field checking module, configured to determine whether each object in the outermost object array of the data in the memory module includes an_id field; if the_id field is included, triggering a copy module to work; if the _id field is not contained, a field assignment module, a statement combination module, a database update module and a replication module are triggered to work in sequence;
The copying module is used for copying data in one copy of the memory module according to the triggering of the field checking module and triggering the statement combination module to work;
a field assignment module, configured to assign a value to the_id field according to the triggering of the field checking module;
The statement combination module is used for combining the statement for assigning the_id field according to the triggering of the field checking module and combining the statement for updating the database according to the triggering of the copying module;
The data updating module is used for updating one copied data so that the data is updated into the target data;
And the database updating module is used for updating corresponding data in the database according to the statement for assigning the_id field and the statement for updating the database.
5. A readable storage medium having one or more programs stored thereon, wherein the one or more programs, when executed by one or more processors, implement the method of driving MongoDB update data according to any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010574553.3A CN111782667B (en) | 2020-06-22 | 2020-06-22 | MongoDB updated data driving method, system and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010574553.3A CN111782667B (en) | 2020-06-22 | 2020-06-22 | MongoDB updated data driving method, system and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111782667A CN111782667A (en) | 2020-10-16 |
| CN111782667B true CN111782667B (en) | 2024-05-17 |
Family
ID=72756103
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010574553.3A Active CN111782667B (en) | 2020-06-22 | 2020-06-22 | MongoDB updated data driving method, system and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111782667B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101499069A (en) * | 2008-02-02 | 2009-08-05 | 中兴通讯股份有限公司 | Internal memory database file updating method and apparatus |
| CN108536798A (en) * | 2018-04-02 | 2018-09-14 | 携程旅游网络技术（上海）有限公司 | The restoration methods and system of the other database data of order level |
| CN109597818A (en) * | 2018-11-28 | 2019-04-09 | 优刻得科技股份有限公司 | Data-updating method, device, storage medium and equipment |
| CN110795494A (en) * | 2018-07-17 | 2020-02-14 | 北京京东尚科信息技术有限公司 | Automatic testing method and device for synchronous and asynchronous cache data |
| CN110874364A (en) * | 2019-11-19 | 2020-03-10 | 北京启迪区块链科技发展有限公司 | Query statement processing method, device, equipment and storage medium |
| CN111008244A (en) * | 2019-11-22 | 2020-04-14 | 厦门安胜网络科技有限公司 | Database synchronization and analysis method and system |
| CN111143326A (en) * | 2019-12-26 | 2020-05-12 | 深圳前海环融联易信息科技服务有限公司 | Method and device for reducing database operation, computer equipment and storage medium |
-
2020
- 2020-06-22 CN CN202010574553.3A patent/CN111782667B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101499069A (en) * | 2008-02-02 | 2009-08-05 | 中兴通讯股份有限公司 | Internal memory database file updating method and apparatus |
| CN108536798A (en) * | 2018-04-02 | 2018-09-14 | 携程旅游网络技术（上海）有限公司 | The restoration methods and system of the other database data of order level |
| CN110795494A (en) * | 2018-07-17 | 2020-02-14 | 北京京东尚科信息技术有限公司 | Automatic testing method and device for synchronous and asynchronous cache data |
| CN109597818A (en) * | 2018-11-28 | 2019-04-09 | 优刻得科技股份有限公司 | Data-updating method, device, storage medium and equipment |
| CN110874364A (en) * | 2019-11-19 | 2020-03-10 | 北京启迪区块链科技发展有限公司 | Query statement processing method, device, equipment and storage medium |
| CN111008244A (en) * | 2019-11-22 | 2020-04-14 | 厦门安胜网络科技有限公司 | Database synchronization and analysis method and system |
| CN111143326A (en) * | 2019-12-26 | 2020-05-12 | 深圳前海环融联易信息科技服务有限公司 | Method and device for reducing database operation, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111782667A (en) | 2020-10-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20230306135A1 (en) | Masking sensitive information in records of filtered accesses to unstructured data | |
| CN111104392B (en) | Database migration method and device, electronic equipment and storage medium | |
| WO2017019879A1 (en) | Multi-query optimization | |
| US9659108B2 (en) | Pluggable architecture for embedding analytics in clustered in-memory databases | |
| US10650032B1 (en) | Filtering pipeline optimizations for unstructured data | |
| US7650346B2 (en) | User-defined type consistency checker | |
| US9772865B2 (en) | On-demand loading of dynamic scripting language code for reduced memory usage | |
| US10664508B1 (en) | Server-side filtering of unstructured data items at object storage services | |
| CN104252501A (en) | Computing equipment and method for executing database operating command | |
| CN111914020A (en) | Data synchronization method and device and data query method and device | |
| CN112912870A (en) | Tenant identifier conversion | |
| CN112182637A (en) | Safety control system, method, device and storage medium | |
| CN111782667B (en) | MongoDB updated data driving method, system and storage medium | |
| EP3605353B1 (en) | Method and system for data transfer between databases | |
| US8286144B2 (en) | Administering a process filesystem with respect to program code conversion | |
| US20090164197A1 (en) | Method for transforming overlapping paths in a logical model to their physical equivalent based on transformation rules and limited traceability | |
| WO2008085359A1 (en) | Accelerating queries using delayed value projection of enumerated storage | |
| US7290085B2 (en) | Method and system for flexible and efficient protocol table implementation | |
| CN111400404A (en) | Node initialization method, device, equipment and storage medium | |
| CN111222833A (en) | Algorithm configuration combination platform based on data lake server | |
| JP7328884B2 (en) | Data management computer and data management method | |
| CN113806504B (en) | Multi-dimensional report data calculation method and device and computer equipment | |
| WO2024017100A1 (en) | Code snippet obtaining method and device, and storage medium | |
| US20210365416A1 (en) | Mount parameter in file systems | |
| CN116644085A (en) | Data query method and device, storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |