CN110209677A - The method and apparatus of more new data - Google Patents
The method and apparatus of more new data Download PDFInfo
- Publication number
- CN110209677A CN110209677A CN201810117125.0A CN201810117125A CN110209677A CN 110209677 A CN110209677 A CN 110209677A CN 201810117125 A CN201810117125 A CN 201810117125A CN 110209677 A CN110209677 A CN 110209677A
- Authority
- CN
- China
- Prior art keywords
- business
- major key
- bar line
- data
- associate field
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000003860 storage Methods 0.000 claims abstract description 40
- 238000004590 computer program Methods 0.000 claims description 7
- 238000012545 processing Methods 0.000 abstract description 24
- 238000005516 engineering process Methods 0.000 abstract description 4
- 238000010586 diagram Methods 0.000 description 13
- 230000006854 communication Effects 0.000 description 8
- 230000006870 function Effects 0.000 description 8
- 238000004891 communication Methods 0.000 description 7
- 238000004519 manufacturing process Methods 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 230000005291 magnetic effect Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 210000003127 knee Anatomy 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000002688 persistence Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
Abstract
The invention discloses the method and apparatus of more new data, are related to field of computer technology.One specific embodiment of this method includes: the demand according to more new data, obtains associate field；According to associate field, the business major key of the corresponding main business bar line of associate field is searched in caching system；According to business major key, data are updated into the data source of main business bar line.The embodiment obtains business major key and then more new data from buffer storage, reduces business major key query time, promotes business processing efficiency, guarantees the real-time and accuracy of business datum.
Description
Technical field
The present invention relates to field of computer technology, more particularly to a kind of method and apparatus of more new data.
Background technique
In internet and software field, message queue is the use of many software companys, its Decoupling Characteristics, asynchronous communication machine
System and peak value processing capacity are the important leverages of real-time in software data product.However message-oriented middleware and business procedure knot
When being used in combination, once the traffic handing capacity inefficiency of software program, can generate data and overstock, how solve this and ask
Topic is of great significance in today of IT trade fast development.Wherein, message queue is in the transmission process of message
The container of message is saved, message-oriented middleware is to carry out platform-independent data exchange using the message passing mechanism of high efficient and reliable
A kind of technological means.
In the method for the prior art more new data, calling interface inquires the business major key of main business bar line, then updates master
The respective field data of business bar line.Main business bar line refers to the line of current data processing links, data source be need by
What the data of external business bar line updated.External business bar line refers to the line of non-current processing links, its message count
According to the field that will be updated main business bar line.Business major key refers to the field in data source with service logic meaning.
In realizing process of the present invention, at least there are the following problems in the prior art for inventor's discovery: current more new data
Method in, in the case where business complexity, frequent calling interface inquiry database or wide table, the time of cost constantly increase
Add, worse and worse, the speed of processing message can slow down the processing capacity of business procedure in the unit time, lead to data severe product
Pressure, seriously affects the real-time and accuracy of data.
Summary of the invention
In view of this, the embodiment of the present invention provides a kind of method and apparatus of more new data, business major key can be reduced and looked into
The time is ask, business processing efficiency is promoted, guarantees the real-time and accuracy of business datum.
To achieve the above object, according to an aspect of an embodiment of the present invention, a kind of method of more new data is provided.
A kind of method of more new data of the embodiment of the present invention includes: the demand according to more new data, obtains associate field；
According to the associate field, the business major key of the corresponding main business bar line of the associate field is searched in caching system；According to
The business major key updates data into the data source of main business bar line.
Optionally, the corresponding main business item of the associate field according to the associate field, is being searched in caching system
Before the business major key of line, the method also includes: the business major key of the main business bar line is stored into the data source,
And in a manner of preset by the storage of the business major key of the associate field and the main business bar line into the caching system.
Optionally, the preset mode is the mode of key-value pair.
Optionally, the data source includes: at least one of wide table and database.
To achieve the above object, according to another aspect of an embodiment of the present invention, a kind of device of more new data is provided.
The device of a kind of more new data of the embodiment of the present invention, comprising: module is obtained, for the need according to more new data
It asks, obtains associate field；Searching module, for searching the associate field pair in caching system according to the associate field
The business major key for the main business bar line answered；Update module, for according to the business major key, data to be updated to main business bar line
Data source in.
Optionally, described device further include: memory module, for storing the business major key of the main business bar line to institute
It states in data source, and is stored the business major key of the associate field and the main business bar line to described in a manner of preset
In caching system.
Optionally, the preset mode is the mode of key-value pair.
Optionally, the data source includes: at least one of wide table and database.
To achieve the above object, according to an embodiment of the present invention in another aspect, providing a kind of electronic equipment.
The a kind of electronic equipment of the embodiment of the present invention includes: one or more processors；Storage device, for storing one
Or multiple programs, when one or more programs are executed by one or more processors, so that one or more processors realize this
The method of the more new data of inventive embodiments.
To achieve the above object, another aspect according to an embodiment of the present invention, provides a kind of computer-readable medium.
A kind of computer-readable medium of the embodiment of the present invention, is stored thereon with computer program, and program is held by processor
The method of the more new data of the embodiment of the present invention is realized when row.
One embodiment in foregoing invention has the following advantages that or the utility model has the advantages that can obtain business from buffer storage
Major key and then more new data promote business processing efficiency, guarantee business datum so as to reduce business major key query time
Real-time and accuracy；The business major key of main business bar line is stored into data source in the embodiment of the present invention, so as to root
According to the data in business major key more source of new data；In the business major key of storage main business bar line to data source in the embodiment of the present invention
While, in a manner of preset by the storage of the business major key of associate field and main business bar line into caching system, so as to
Its corresponding business major key is inquired by associate field in caching system, interface polls is avoided, reduces the exploitation of interface
Amount, further reduces the query time of business major key；Preset mode can be the mode of key-value pair in the embodiment of the present invention,
So as to utilize the characteristic of key assignments storage mode；Data source can be database and be also possible to wide table in the embodiment of the present invention,
So as to the storage mode of flexible choice data.
Further effect possessed by above-mentioned non-usual optional way adds hereinafter in conjunction with specific embodiment
With explanation.
Detailed description of the invention
Attached drawing for a better understanding of the present invention, does not constitute an undue limitation on the present invention.Wherein:
Fig. 1 is the schematic diagram of the key step of the method for more new data according to an embodiment of the present invention；
Fig. 2 is the schematic diagram of the main business link of the method for more new data according to an embodiment of the present invention；
Fig. 3 is the schematic diagram of the external business link of the method for more new data according to an embodiment of the present invention；
Fig. 4 is the schematic diagram of the main modular of the device of more new data according to an embodiment of the present invention；
Fig. 5 is that the embodiment of the present invention can be applied to exemplary system architecture figure therein；
Fig. 6 is adapted for the structural representation of the computer system for the terminal device or server of realizing the embodiment of the present invention
Figure.
Specific embodiment
Below in conjunction with attached drawing, an exemplary embodiment of the present invention will be described, including the various of the embodiment of the present invention
Details should think them only exemplary to help understanding.Therefore, those of ordinary skill in the art should recognize
It arrives, it can be with various changes and modifications are made to the embodiments described herein, without departing from scope and spirit of the present invention.Together
Sample, for clarity and conciseness, descriptions of well-known functions and structures are omitted from the following description.
It will lead to frequent calling interface by the data that calling interface updates main business bar line, take a significant amount of time, reduce
The processing capacity and treatment effeciency of business influence the real-time and accuracy of data.Industry is obtained in the present invention from buffer storage
Business major key and then more new data, avoid interface polls, reduce the exploitation quantity of interface, reduce business major key query time, are promoted
Business processing efficiency guarantees the real-time and accuracy of business datum.In the present invention, current system is data processing system first
System, main business bar line and external business bar line are the component parts of data processing system, and either main business bar line is also
It is external business bar line, requires to obtain creation data from production system.In order to be decoupled with production system, main business bar line and
External business bar line directly passes through message-oriented middleware and obtains message data from production system, carries out data processing processing.Wherein, it adopts
Message queue technology is the means of communication of the asynchronous message subscribing of one kind, production and consumption.Existing message queue technology
Compare it is more, as ActiveMQ (ActiveMQ be Apache produce, most popular, the powerful open source messaging bus of ability), Kafka
(Kafka is that a kind of distributed post of high-throughput subscribes to message system, it can handle the institute in the website of consumer's scale
Have movement flow data), (zeroMQ is a Message Processing bank of queues to zeroMQ, can be between multiple threads, kernel and host box
Elastic telescopic) etc..
Fig. 1 is the schematic diagram of the key step of the method for more new data according to an embodiment of the present invention, as shown in Figure 1, this
The key step of the method for the more new data of inventive embodiments may include:
Step S101: according to the demand of more new data, associate field is obtained.When the message data of external business bar line updates
When influencing the data of the data source under main business bar line, external business main line will obtain associate field.Associate field is exactly
In data processing system, the field that main business bar line and external business bar line are associated, for example, the field of the main table of order is
Order odd numbers, customer name and customer address, order are order odd numbers, order goods and ordering price from the field of table, then order
It only is number the main table of order and order from the associate field of table.
Step S102: according to associate field, the business of the corresponding main business bar line of associate field is searched in caching system
Major key.According to the associate field that step S101 is obtained, external business bar line searches the corresponding master of associate field in caching system
The business major key of business bar line.Wherein, caching system can be Redis caching in the present invention, be also possible to other caching forms,
This is not limited by the present invention.Wherein, Redis caching is being write, propped up using standard computer programming language for an open source
Hold network, it is memory-based also can persistence log type, key assignments type database, and provide the application programming of multilingual
Interface.
Step S103: according to business major key, data are updated into the data source of main business bar line.External business bar line obtains
After getting business major key, its more new data is updated into the data source of main business bar line, completes external business data to master
The data of business bar line update.
In the embodiment of the present invention, according to associate field, the corresponding main business item of associate field is searched in caching system
Before the business major key of line, the method for more new data can also include: by the business major key storage of main business bar line to data source
In, and stored the business major key of associate field and main business bar line into caching system in a manner of preset.In this step
Being by the purpose that the business major key of main business bar line is stored into data source can be according to the number in business major key more source of new data
According to.And in this step while storing the business major key of main business bar line to data source, by associate field in a manner of preset
Storing the purpose into caching system with the business major key of main business bar line is that can be looked into caching system by associate field
Its corresponding business major key is ask, interface polls are avoided, reduces the exploitation amount of interface, further reduces looking into for business major key
Ask the time.
In the embodiment of the present invention, preset mode can be the mode of key-value pair.Wherein, key-value pair is programming language logarithm
The realization mapped in concept is learned, key (Key) is used as the index of element, and value (value) then indicates the data for storing and reading.This
In invention, it is value with the business major key of main business bar line that the mode of key-value pair, which can be with associate field as Key,.
In the embodiment of the present invention, data source may include: at least one of wide table and database.Wherein, wide table refers to word
Section is more, comprising dimension hierarchy also more table, can be ES wide table in the present invention.ES, i.e. ElasticSearch are
One is based on the search server of Lucene (Lucene is a set of open source library for full-text search and search), it is provided
The full-text search engine of one distributed multi-user ability.Database is to come tissue, storage and management number according to data structure
According to warehouse of the foundation on computer memory device.
In order to facilitate understanding, in the subsequent descriptions of the embodiment of the present invention, " preset mode " is taken as " side of key-value pair
Formula ", " data source of main business bar line " are taken as " the wide table of main business ", and " caching system " is taken as " Redis caching ", certain " key assignments
Pair mode ", " the wide table of main business " and " Redis caching " be not used to limit the protection scope of technical solution of the present invention
Fixed, " preset mode ", " data source of main business bar line " and " caching system " can be according to specific business field in the present invention
Scape is adaptively adjusted.The method of the present invention more new data includes main business link and external business link, and Fig. 2 is according to this
The schematic diagram of the main business link of the method for the more new data of inventive embodiments.As shown in Fig. 2, the method for the present invention more new data
Main business link in, when the message data of main business bar line transmits, main business bar line save business major key data
It is Key with associate field while table wide to main business, business major key is that value arrives associate field and business major key storage
In Redis caching.
Fig. 3 is the schematic diagram of the external business link of the method for more new data according to an embodiment of the present invention.Such as Fig. 3 institute
Show, in the external business link of the method for the present invention more new data, when the data update message of external business bar line transmits
When, the business major key for the main business bar line that its Key is stored is inquired into Redis caching by associate field.Then it is obtaining
To after the business major key of main business bar line, the data of external business bar line are updated into the wide table of main business, complete external business
Update of the line to the wide table data of main business.
By taking logistics distribution as an example, the method that explains in detail more new data of the invention.Wherein, main business bar line can be point
Business bar line is picked, external business bar line can be waybill business bar line, and associate field can be Air Way Bill No., and business major key can be
The unique identification of package and sortation hubs, more new data can be waybill and delete label.When sorting link, when sorting business
When the data-message of bar line transmits, in the unique identification for saving package and sortation hubs to while sorting ES wide table,
It is Key with Air Way Bill No., the unique identification with package and sortation hubs is value, by Air Way Bill No., package and sortation hubs
Unique identification storage is into Redis caching.It, by Air Way Bill No., is arrived in waybill link when waybill, which deletes flag transmission, to come
The unique identification of its corresponding package and sortation hubs is inquired in Redis caching.Then waybill is deleted and marks this information
It updates in sorting ES wide table.
The technical solution of more new data according to an embodiment of the present invention can be seen that can obtain business from buffer storage
Major key and then more new data promote business processing efficiency, guarantee business datum so as to reduce business major key query time
Real-time and accuracy；The business major key of main business bar line is stored into data source in the embodiment of the present invention, so as to root
According to the data in business major key more source of new data；In the business major key of storage main business bar line to data source in the embodiment of the present invention
While, in a manner of preset by the storage of the business major key of associate field and main business bar line into caching system, so as to
Its corresponding business major key is inquired by associate field in caching system, interface polls is avoided, reduces the exploitation of interface
Amount, further reduces the query time of business major key；Preset mode can be the mode of key-value pair in the embodiment of the present invention,
So as to utilize the characteristic of key assignments storage mode；Data source can be database and be also possible to wide table in the embodiment of the present invention,
So as to the storage mode of flexible choice data.
Fig. 4 is the schematic diagram of the main modular of the device of more new data according to an embodiment of the present invention.As shown in figure 4, this
The device 400 of the more new data of inventive embodiments mainly comprises the following modules: obtaining module 401, searching module 402 and updates mould
Block 403.
Wherein, obtaining module 401 can be used for the demand of basis more new data, obtain associate field.Searching module 402 can be used
According to associate field, the business major key of the corresponding main business bar line of associate field is searched in caching system.Update module 403
It can be used for being updated data into the data source of main business bar line according to business major key.
In the embodiment of the present invention, the device of more new data can also include: memory module (not shown).Wherein, it deposits
Storage module can be used for storing the business major key of main business bar line into data source, and in a manner of preset by associate field and
The business major key of main business bar line is stored into caching system.
In the embodiment of the present invention, preset mode can be the mode of key-value pair.
In the embodiment of the present invention, data source may include at least one of wide table and database.
From the above, it can be seen that business major key and then more new data can be obtained from buffer storage, so as to
Reduction business major key query time promotes business processing efficiency, guarantees the real-time and accuracy of business datum；The present invention is implemented
The business major key of main business bar line is stored into data source in example, so as to according to the number in business major key more source of new data
According to；In the embodiment of the present invention store main business bar line business major key arrive data source while, in a manner of preset will be associated with
The business major key of field and main business bar line is stored into caching system, so as to be looked into caching system by associate field
Its corresponding business major key is ask, interface polls are avoided, reduces the exploitation amount of interface, further reduces looking into for business major key
Ask the time；Preset mode can be the mode of key-value pair in the embodiment of the present invention, so as to utilize key assignments storage mode
Characteristic；Data source can be database and be also possible to wide table in the embodiment of the present invention, so as to the storage of flexible choice data
Mode.
Fig. 5 shows the exemplary of the method for more new data or the device of more new data that can apply the embodiment of the present invention
System architecture 500.
As shown in figure 5, system architecture 500 may include terminal device 501,502,503, network 504 and server 505.
Network 504 between terminal device 501,502,503 and server 505 to provide the medium of communication link.Network 504 can be with
Including various connection types, such as wired, wireless communication link or fiber optic cables etc..
User can be used terminal device 501,502,503 and be interacted by network 504 with server 505, to receive or send out
Send message etc..Various telecommunication customer end applications, such as the application of shopping class, net can be installed on terminal device 501,502,503
(merely illustrative) such as the application of page browsing device, searching class application, instant messaging tools, mailbox client, social platform softwares.
Terminal device 501,502,503 can be the various electronic equipments with display screen and supported web page browsing, packet
Include but be not limited to smart phone, tablet computer, pocket computer on knee and desktop computer etc..
Server 505 can be to provide the server of various services, such as utilize terminal device 501,502,503 to user
The shopping class website browsed provides the back-stage management server (merely illustrative) supported.Back-stage management server can be to reception
To the data such as information query request analyze etc. processing, and by processing result (such as target push information, product letter
Breath -- merely illustrative) feed back to terminal device.
It should be noted that the method for more new data provided by the embodiment of the present invention is generally executed by server 505, phase
Ying Di, the device of more new data are generally positioned in server 505.
It should be understood that the number of terminal device, network and server in Fig. 5 is only schematical.According to realization need
It wants, can have any number of terminal device, network and server.
Below with reference to Fig. 6, it illustrates the computer systems 600 for the terminal device for being suitable for being used to realize the embodiment of the present invention
Structural schematic diagram.Terminal device shown in Fig. 6 is only an example, function to the embodiment of the present invention and should not use model
Shroud carrys out any restrictions.
As shown in fig. 6, computer system 600 includes central processing unit (CPU) 601, it can be read-only according to being stored in
Program in memory (ROM) 602 or be loaded into the program in random access storage device (RAM) 603 from storage section 608 and
Execute various movements appropriate and processing.In RAM 603, also it is stored with system 600 and operates required various programs and data.
CPU 601, ROM 602 and RAM 603 are connected with each other by bus 604.Input/output (I/O) interface 605 is also connected to always
Line 604.
I/O interface 605 is connected to lower component: the importation 606 including keyboard, mouse etc.；It is penetrated including such as cathode
The output par, c 607 of spool (CRT), liquid crystal display (LCD) etc. and loudspeaker etc.；Storage section 608 including hard disk etc.；
And the communications portion 609 of the network interface card including LAN card, modem etc..Communications portion 609 via such as because
The network of spy's net executes communication process.Driver 610 is also connected to I/O interface 605 as needed.Detachable media 611, such as
Disk, CD, magneto-optic disk, semiconductor memory etc. are mounted on as needed on driver 610, in order to read from thereon
Computer program be mounted into storage section 608 as needed.
Particularly, disclosed embodiment, the process described above with reference to flow chart may be implemented as counting according to the present invention
Calculation machine software program.For example, embodiment disclosed by the invention includes a kind of computer program product comprising be carried on computer
Computer program on readable medium, the computer program include the program code for method shown in execution flow chart.?
In such embodiment, which can be downloaded and installed from network by communications portion 609, and/or from can
Medium 611 is dismantled to be mounted.When the computer program is executed by central processing unit (CPU) 601, system of the invention is executed
The above-mentioned function of middle restriction.
It should be noted that computer-readable medium shown in the present invention can be computer-readable signal media or meter
Calculation machine readable storage medium storing program for executing either the two any combination.Computer readable storage medium for example can be --- but not
Be limited to --- electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor system, device or device, or any above combination.Meter
The more specific example of calculation machine readable storage medium storing program for executing can include but is not limited to: have the electrical connection, just of one or more conducting wires
Taking formula computer disk, hard disk, random access storage device (RAM), read-only memory (ROM), erasable type may be programmed read-only storage
Device (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), light storage device, magnetic memory device,
Or above-mentioned any appropriate combination.In the present invention, computer readable storage medium can be it is any include or storage journey
The tangible medium of sequence, the program can be commanded execution system, device or device use or in connection.And at this
In invention, computer-readable signal media may include in a base band or as carrier wave a part propagate data-signal,
Wherein carry computer-readable program code.The data-signal of this propagation can take various forms, including but unlimited
In electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media can also be that computer can
Any computer-readable medium other than storage medium is read, which can send, propagates or transmit and be used for
By the use of instruction execution system, device or device or program in connection.Include on computer-readable medium
Program code can transmit with any suitable medium, including but not limited to: wireless, electric wire, optical cable, RF etc. are above-mentioned
Any appropriate combination.
Flow chart and block diagram in attached drawing are illustrated according to the system of various embodiments of the invention, method and computer journey
The architecture, function and operation in the cards of sequence product.In this regard, each box in flowchart or block diagram can generation
A part of one module, program segment or code of table, a part of above-mentioned module, program segment or code include one or more
Executable instruction for implementing the specified logical function.It should also be noted that in some implementations as replacements, institute in box
The function of mark can also occur in a different order than that indicated in the drawings.For example, two boxes succeedingly indicated are practical
On can be basically executed in parallel, they can also be executed in the opposite order sometimes, and this depends on the function involved.Also it wants
It is noted that the combination of each box in block diagram or flow chart and the box in block diagram or flow chart, can use and execute rule
The dedicated hardware based systems of fixed functions or operations is realized, or can use the group of specialized hardware and computer instruction
It closes to realize.
Being described in module involved in the embodiment of the present invention can be realized by way of software, can also be by hard
The mode of part is realized.Described module also can be set in the processor, for example, can be described as: a kind of processor packet
It includes and obtains module, searching module and update module.Wherein, the title of these modules is not constituted under certain conditions to the module
The restriction of itself is also described as " according to the demand of more new data, obtaining the mould of associate field for example, obtaining module
Block ".
As on the other hand, the present invention also provides a kind of computer-readable medium, which be can be
Included in equipment described in above-described embodiment；It is also possible to individualism, and without in the supplying equipment.Above-mentioned calculating
Machine readable medium carries one or more program, when said one or multiple programs are executed by the equipment, makes
Obtaining the equipment includes: the demand according to more new data, obtains associate field；According to associate field, searches and close in caching system
Join the business major key of the corresponding main business bar line of field；According to business major key, data are updated into the data source to main business bar line
In.
Technical solution according to an embodiment of the present invention can obtain business major key and then more new data from buffer storage,
So as to reduce business major key query time, business processing efficiency is promoted, guarantees the real-time and accuracy of business datum；This
The business major key of main business bar line is stored into data source in inventive embodiments, so as to according to business major key more new data
Data in source；In the embodiment of the present invention while storing the business major key of main business bar line to data source, with preset side
Formula stores the business major key of associate field and main business bar line into caching system, so as to pass through pass in caching system
Join its corresponding business major key of Field Inquiry, avoids interface polls, reduce the exploitation amount of interface, further reduce business
The query time of major key；Preset mode can be the mode of key-value pair in the embodiment of the present invention, so as to be deposited using key assignments
The characteristic of storage mode；Data source can be database and be also possible to wide table in the embodiment of the present invention, so as to flexible choice number
According to storage mode.
Above-mentioned specific embodiment, does not constitute a limitation on the scope of protection of the present invention.Those skilled in the art should be bright
It is white, design requirement and other factors are depended on, various modifications, combination, sub-portfolio and substitution can occur.It is any
Made modifications, equivalent substitutions and improvements etc. within the spirit and principles in the present invention, should be included in the scope of the present invention
Within.
Claims (10)
1. a kind of method of more new data characterized by comprising
According to the demand of more new data, associate field is obtained；
According to the associate field, the business major key of the corresponding main business bar line of the associate field is searched in caching system；
According to the business major key, data are updated into the data source of main business bar line.
2. the method according to claim 1, wherein being searched in caching system according to the associate field
Before the business major key of the corresponding main business bar line of the associate field, the method also includes: by the main business bar line
Business major key is stored into the data source, and by the industry of the associate field and the main business bar line in a manner of preset
Major key storage be engaged in into the caching system.
3. according to the method described in claim 2, it is characterized in that, the preset mode is the mode of key-value pair.
4. the method according to claim 1, wherein the data source includes: at least one in wide table and database
Kind.
5. a kind of device of more new data characterized by comprising
Module is obtained, for the demand according to more new data, obtains associate field；
Searching module, for searching the corresponding main business item of the associate field in caching system according to the associate field
The business major key of line；
Update module, for according to the business major key, data to be updated into the data source of main business bar line.
6. device according to claim 5, which is characterized in that described device further include: memory module is used for the master
The business major key of business bar line is stored into the data source, and by the associate field and the main business in a manner of preset
The business major key of business line is stored into the caching system.
7. device according to claim 6, which is characterized in that the preset mode is the mode of key-value pair.
8. device according to claim 5, which is characterized in that the data source includes: at least one in wide table and database
Kind.
9. a kind of electronic equipment characterized by comprising
One or more processors；
Storage device, for storing one or more programs,
When one or more of programs are executed by one or more of processors, so that one or more of processors are real
The now method as described in any in claim 1-4.
10. a kind of computer-readable medium, is stored thereon with computer program, which is characterized in that described program is held by processor
The method as described in any in claim 1-4 is realized when row.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810117125.0A CN110209677A (en) | 2018-02-06 | 2018-02-06 | The method and apparatus of more new data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810117125.0A CN110209677A (en) | 2018-02-06 | 2018-02-06 | The method and apparatus of more new data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110209677A true CN110209677A (en) | 2019-09-06 |
Family
ID=67778569
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810117125.0A Pending CN110209677A (en) | 2018-02-06 | 2018-02-06 | The method and apparatus of more new data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110209677A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111339561A (en) * | 2020-02-28 | 2020-06-26 | 北京字节跳动网络技术有限公司 | Data processing method, electronic device and storage medium |
| CN112000816A (en) * | 2020-08-07 | 2020-11-27 | 中国建设银行股份有限公司 | Data processing method and device |
| CN112084219A (en) * | 2020-09-16 | 2020-12-15 | 京东数字科技控股股份有限公司 | Method, apparatus, electronic device, and medium for processing data |
| CN112685191A (en) * | 2019-10-18 | 2021-04-20 | 北京京东振世信息技术有限公司 | Message processing method and device |
| CN112749233A (en) * | 2021-01-15 | 2021-05-04 | 京东数字科技控股股份有限公司 | Data synchronization method and device, electronic equipment and storage medium |
| CN113157734A (en) * | 2021-04-20 | 2021-07-23 | 平安银行股份有限公司 | Data processing method, device and equipment based on search framework and storage medium |
| CN116501715A (en) * | 2023-04-28 | 2023-07-28 | 成都赛力斯科技有限公司 | Real-time association updating method and device for multi-table full data |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130275476A1 (en) * | 2012-04-12 | 2013-10-17 | International Business Machines Corporation | Managing record location lookup caching in a relational database |
| CN103473267A (en) * | 2013-08-09 | 2013-12-25 | 深圳市中科新业信息科技发展有限公司 | Data storage query method and system |
| CN105260475A (en) * | 2015-10-30 | 2016-01-20 | 努比亚技术有限公司 | Data searching method, data saving method and related equipment |
| CN105653566A (en) * | 2014-12-03 | 2016-06-08 | 阿里巴巴集团控股有限公司 | Method and apparatus realizing database writing and visiting |
| CN105843895A (en) * | 2016-03-22 | 2016-08-10 | 北京建飞无限科技有限公司 | EhCache-based data querying and synchronizing method, device and system |
| CN106997378A (en) * | 2017-03-13 | 2017-08-01 | 摩贝（上海）生物科技有限公司 | The synchronous method of database data polymerization based on Redis |
-
2018
- 2018-02-06 CN CN201810117125.0A patent/CN110209677A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130275476A1 (en) * | 2012-04-12 | 2013-10-17 | International Business Machines Corporation | Managing record location lookup caching in a relational database |
| CN103473267A (en) * | 2013-08-09 | 2013-12-25 | 深圳市中科新业信息科技发展有限公司 | Data storage query method and system |
| CN105653566A (en) * | 2014-12-03 | 2016-06-08 | 阿里巴巴集团控股有限公司 | Method and apparatus realizing database writing and visiting |
| CN105260475A (en) * | 2015-10-30 | 2016-01-20 | 努比亚技术有限公司 | Data searching method, data saving method and related equipment |
| CN105843895A (en) * | 2016-03-22 | 2016-08-10 | 北京建飞无限科技有限公司 | EhCache-based data querying and synchronizing method, device and system |
| CN106997378A (en) * | 2017-03-13 | 2017-08-01 | 摩贝（上海）生物科技有限公司 | The synchronous method of database data polymerization based on Redis |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112685191A (en) * | 2019-10-18 | 2021-04-20 | 北京京东振世信息技术有限公司 | Message processing method and device |
| CN112685191B (en) * | 2019-10-18 | 2023-09-22 | 北京京东振世信息技术有限公司 | Message processing method and device |
| CN111339561A (en) * | 2020-02-28 | 2020-06-26 | 北京字节跳动网络技术有限公司 | Data processing method, electronic device and storage medium |
| CN111339561B (en) * | 2020-02-28 | 2023-04-07 | 北京字节跳动网络技术有限公司 | Data processing method, electronic device and storage medium |
| CN112000816A (en) * | 2020-08-07 | 2020-11-27 | 中国建设银行股份有限公司 | Data processing method and device |
| CN112084219A (en) * | 2020-09-16 | 2020-12-15 | 京东数字科技控股股份有限公司 | Method, apparatus, electronic device, and medium for processing data |
| CN112749233A (en) * | 2021-01-15 | 2021-05-04 | 京东数字科技控股股份有限公司 | Data synchronization method and device, electronic equipment and storage medium |
| CN113157734A (en) * | 2021-04-20 | 2021-07-23 | 平安银行股份有限公司 | Data processing method, device and equipment based on search framework and storage medium |
| CN113157734B (en) * | 2021-04-20 | 2022-10-11 | 平安银行股份有限公司 | Data processing method, device and equipment based on search framework and storage medium |
| CN116501715A (en) * | 2023-04-28 | 2023-07-28 | 成都赛力斯科技有限公司 | Real-time association updating method and device for multi-table full data |
| CN116501715B (en) * | 2023-04-28 | 2024-03-12 | 重庆赛力斯凤凰智创科技有限公司 | Real-time association updating method and device for multi-table full data |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110209677A (en) | The method and apparatus of more new data | |
| CN109189835A (en) | The method and apparatus of the wide table of data are generated in real time | |
| CN110019211A (en) | The methods, devices and systems of association index | |
| CN110371560A (en) | Automatically the method and apparatus made an inventory | |
| CN110019258A (en) | The method and apparatus for handling order data | |
| CN109034988A (en) | A kind of accounting entry generation method and device | |
| CN110019087A (en) | Data processing method and its system | |
| CN107844324A (en) | Customer terminal webpage redirects treating method and apparatus | |
| CN109614402A (en) | Multidimensional data query method and device | |
| CN109388654A (en) | A kind of method and apparatus for inquiring tables of data | |
| CN110443531A (en) | The method and apparatus for determining the bit information of article in order | |
| CN110348771A (en) | The method and apparatus that a kind of pair of order carries out group list | |
| CN110019367A (en) | A kind of method and apparatus of statistical data feature | |
| CN109241033A (en) | The method and apparatus for creating real-time data warehouse | |
| CN110119445A (en) | The method and apparatus for generating feature vector and text classification being carried out based on feature vector | |
| CN109961198A (en) | Related information generation method and device | |
| CN110135770A (en) | The generation method and device of outbound scheme | |
| CN113095893A (en) | Method and device for determining sales of articles | |
| CN110304385A (en) | A kind of warehouse restocking method and apparatus | |
| CN110389812A (en) | For managing the method, equipment and computer readable storage medium of virtual machine | |
| CN109960212A (en) | Task sending method and device | |
| CN110309142A (en) | The method and apparatus of regulation management | |
| CN109961306A (en) | A kind of inventory allocation method and apparatus of article | |
| CN109961299A (en) | The method and apparatus of data analysis | |
| CN109977139A (en) | Data processing method and device based on type of structured query statement |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |