CN109766392A - Data processing method and device - Google Patents
Data processing method and device Download PDFInfo
- Publication number
- CN109766392A CN109766392A CN201910053403.5A CN201910053403A CN109766392A CN 109766392 A CN109766392 A CN 109766392A CN 201910053403 A CN201910053403 A CN 201910053403A CN 109766392 A CN109766392 A CN 109766392A
- Authority
- CN
- China
- Prior art keywords
- information
- data
- state
- target
- indicated
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Stored Programmes (AREA)
Abstract
Embodiment of the disclosure discloses data processing method and device.One specific embodiment of this method includes: to obtain target data information set；It is the data information of armed state in target data information set, state indicated by current state information, the source storage location indicated by the source storage location information in the data information reads data, target storage position indicated by data deposit target storage position information by reading, and the current state information of the data information is updated to characterize the information of storage state；It is the data information of storage state in target data information set, state indicated by current state information, it is identical in response to the data that determine that source storage location information and the position indicated by target storage position information of the data information are stored, the current state information of the data information is updated to the information that state is completed in characterization.The embodiment, which is realized, is stored in target storage position from source storage location for data.
Description
Technical field
Embodiment of the disclosure is related to field of computer technology, and in particular to data processing method and device.
Background technique
With the fast development of Internet technology, a large amount of data can be all generated all the time.These mass datas, especially
It is basic data, is typically stored in distributed database.When need to certain data of these data or certain dimensions into
When row is checked, need to inquire these data.But the problem of due to database self performance, store the data of mass data
Library some do not support real-time query.Therefore need as needed, by partial data import support real-time query query engine or
In platform.
Summary of the invention
Embodiment of the disclosure proposes data processing method and device.
In a first aspect, embodiment of the disclosure provides a kind of data processing method, this method comprises: obtaining target data
Information aggregate, data information include source storage location information, target storage position information and current state information；For number of targets
It is the data information of armed state according in information aggregate, state indicated by current state information, from the data information
Source storage location information indicated by source storage location read data, the data of reading are stored in target storage position information institute
The target storage position of instruction, and the current state information of the data information is updated to characterize the information of storage state；
It is the data information of storage state, response in target data information set, state indicated by current state information
In the data phase for determining that position indicated by the source storage location information and target storage position information of the data information is stored
Together, the current state information of the data information is updated to the information that state is completed in characterization.
In some embodiments, data information further include: the priority level information.
In some embodiments, this method further include: in response to determining that source storage location information and target storage position are believed
The data that the indicated position of breath is stored are not identical, at preset time point, push for characterize source storage location information and
The different prompt information of the data that position indicated by target storage position information is stored.
In some embodiments, method further include: in response to determining each data information in target data information set
Current state information indicated by state be to be completed state, push for characterizing each number in target data information set
It is believed that breath it is processed at prompt information.
In some embodiments, for state indicated by target data information set, current state information be to
The data information of processing status, the source storage location indicated by the source storage location information in the data information read data,
By target storage position indicated by the data of reading deposit target storage position information, and by the current shape of the data information
State information update is the information of characterization storage state, comprising: according to priority level by high sequence on earth, from number of targets it is believed that
The data information that state indicated by current state information is armed state is chosen in breath set, executes following processing step:
The source storage location indicated by the source storage location information in the data information reads data, and the data of reading are stored in target
Target storage position indicated by storage location information, and the current state information of the data information is updated to characterization and has been deposited
The information of storage state.
Second aspect, embodiment of the disclosure provide a kind of data processing equipment, which includes: acquiring unit, quilt
It is configured to obtain target data information set, data information includes source storage location information, target storage position information and current
Status information；Storage unit, be configured to be for state indicated by target data information set, current state information
The data information of armed state, the source storage location indicated by the source storage location information in the data information read number
According to, target storage position indicated by the data deposit target storage position information by reading, and working as the data information
Preceding state information updating is the information for characterizing storage state；Updating unit is configured in target data information set
, the data information that state indicated by current state information is storage state, in response to determining that the source of the data information is deposited
The data that storage location information and position indicated by target storage position information are stored are identical, by the current shape of the data information
State information update is the information that state is completed in characterization.
In some embodiments, data information further include: the priority level information.
In some embodiments, device further include: the first push unit is configured in response to determine source storage location
The data that position indicated by information and target storage position information is stored are not identical, and at preset time point, push is used for
The different prompt information of data that position indicated by characterization source storage location information and target storage position information is stored.
In some embodiments, the device further include: the second push unit, be configured in response to determine number of targets it is believed that
State indicated by the current state information of each data information in breath set is the state that is completed, and is pushed for characterizing target
Each data information in data information set it is processed at prompt information.
In some embodiments, storage unit is further configured to: according to priority level by high sequence on earth, from mesh
The data information for choosing that state indicated by current state information is armed state in data information set is marked, following place is executed
Manage step: the source storage location indicated by the source storage location information in the data information reads data, by the data of reading
It is stored in target storage position indicated by target storage position information, and the current state information of the data information is updated to
Characterize the information of storage state.
The third aspect, embodiment of the disclosure provide a kind of server, which includes: one or more processing
Device；Storage device is stored thereon with one or more programs；When said one or multiple programs are by said one or multiple processing
Device executes, so that said one or multiple processors realize the method as described in implementation any in first aspect.
Fourth aspect, embodiment of the disclosure provide a kind of computer-readable medium, are stored thereon with computer program,
The method as described in implementation any in first aspect is realized when above procedure is executed by processor.
The method and apparatus that embodiment of the disclosure provides, can obtain target data information set first.Later, for
In target data information set, state indicated by current state information is the data information of armed state, from the data
Source storage location indicated by source storage location information in information reads data, and the data of reading are stored in target storage position
Target storage position indicated by information, and the current state information of the data information is updated to characterization storage state
Information.It is the data information of storage state in target data information set, state indicated by current state information,
The number stored in response to determining position indicated by the source storage location information and target storage position information of the data information
According to identical, the current state information of the data information is updated to the information that state is completed in characterization.Thus, it is possible to by data from
Source storage location is stored in target storage position.Meanwhile guaranteeing the data and source storage location data phase of deposit target storage position
Together.
Detailed description of the invention
By reading a detailed description of non-restrictive embodiments in the light of the attached drawings below, the disclosure is other
Feature, objects and advantages will become more apparent upon:
Fig. 1 is that one embodiment of the disclosure can be applied to exemplary system architecture figure therein；
Fig. 2 is the flow chart according to one embodiment of the data processing method of the disclosure；
Fig. 3 is the schematic diagram of an application scenarios of data processing method according to an embodiment of the present disclosure；
Fig. 4 is the flow chart according to another embodiment of the data processing method of the disclosure；
Fig. 5 is the structural schematic diagram according to one embodiment of the data processing equipment of the disclosure；
Fig. 6 is adapted for the structural schematic diagram for realizing the electronic equipment of embodiment of the disclosure.
Specific embodiment
The disclosure is described in further detail with reference to the accompanying drawings and examples.It is understood that this place
The specific embodiment of description is used only for explaining that correlation is open, rather than the restriction to the disclosure.It also should be noted that being
Convenient for description, is illustrated only in attached drawing and to related disclose relevant part.
It should be noted that in the absence of conflict, the feature in embodiment and embodiment in the disclosure can phase
Mutually combination.The disclosure is described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.
Fig. 1 is shown can be using the data processing method of embodiment of the disclosure or the exemplary system of data processing equipment
System framework 100.
As shown in Figure 1, system architecture 100 may include terminal device 101,102,103, network 104 and server 105.
Network 104 between terminal device 101,102,103 and server 105 to provide the medium of communication link.Network 104 can be with
Including various connection types, such as wired, wireless communication link or fiber optic cables etc..
User can be used terminal device 101,102,103 and be interacted by network 104 with server 105, to receive or send out
Send message etc..Various client applications can be installed, such as data show class application, number on terminal device 101,102,103
According to analysis classes application etc..
Terminal device 101,102,103 can be hardware, be also possible to software.When terminal device 101,102,103 is hard
When part, it can be the various electronic equipments supporting data storage and showing.It, can when terminal device 101,102,103 is software
To be mounted in above-mentioned electronic equipment.Multiple softwares or software module may be implemented into (such as providing distributed clothes in it
Business), single software or software module also may be implemented into.It is not specifically limited herein.
Server 105 can be to provide the server of various services, such as to the application on terminal device 101,102,103
The background server supported is provided.The available target data information set of background process server, and carry out data storage
Deng processing.As needed, user can inquire it by the client application on terminal device.It should be noted that
Data processing method provided by embodiment of the disclosure is generally executed by server 105, and correspondingly, data processing equipment is general
It is set in server 105.
It should be noted that server can be hardware, it is also possible to software.When server is hardware, may be implemented
At the distributed server cluster that multiple servers form, individual server also may be implemented into.It, can when server is software
To be implemented as multiple softwares or software module (such as providing Distributed Services), single software or software also may be implemented into
Module.It is not specifically limited herein.
It should be understood that the number of terminal device, network and server in Fig. 1 is only schematical.According to realization need
It wants, can have any number of terminal device, network and server.
With continued reference to Fig. 2, the process 200 of one embodiment of the data processing method according to the disclosure is shown.The number
According to processing method, comprising the following steps:
Step 201, target data information set is obtained.
In the present embodiment, the executing subject of data processing method can be from local or communication connection electronic equipment
Obtain target data information set.Wherein, data information includes source storage location information, target storage position information and current shape
State information.Target data information set can be arbitrary data information set.The determination of target data information set can be by
Technical staff is specified, can also be screened according to certain condition.As an example, target data information set can be daily
The set of the information of the pending data of generation.In practice, current state information be can be for characterize data current state
Information.According to the actual situation, current state may include multiple states, including but not limited to armed state, storage state,
State etc. is completed.It is appreciated that according to the actual situation, these states can have different statements.For example, armed state
It may also be referred to as upstream data ready state.For another example, state, which is completed, may also be referred to as data importing completion status.In practice,
Current state information can be the information of diversified forms, including but not limited to: number, letter, other characters etc..
In some optional implementations of the present embodiment, data information further include: Data Identification, the priority level information
Deng.
It step 202, is armed state for state indicated by target data information set, current state information
Data information, the source storage location indicated by the source storage location information in the data information reads data, by reading
Data are stored in target storage position indicated by target storage position information, and more by the current state information of the data information
The new information to characterize storage state.
In the present embodiment, for each data information in target data information set, it may be determined whether when
State indicated by preceding status information is the data information of armed state.Wherein, armed state is that can be upstream data
The state of data that is ready, being handled.If it does, being wait locate for state indicated by current state information
Each data information in each data information of reason state, above-mentioned executing subject can store position from the source in the data information
The indicated source storage location of confidence breath reads data, target indicated by the data deposit target storage position information by reading
Storage location.Further, it is also possible to which the current state information of the data information is updated to characterize the information of storage state.According to
Actual conditions, the information for characterizing storage state can be the information of diversified forms.For example, can be with different digital representations not
Same state.For example, can indicate storage state with digital " 2 ".
In some optional implementations of the present embodiment, in target data information set, current state is believed
The indicated state of breath is the data information of armed state, the source indicated by the source storage location information in the data information
Storage location reads data, target storage position indicated by the data deposit target storage position information by reading, and will
The current state information of the data information is updated to the information of characterization storage state, comprising: according to priority level by it is high on earth
Sequence, from target data information set choose current state information indicated by state be armed state data believe
Breath, executes following processing step: the source storage location indicated by the source storage location information in the data information reads data,
By target storage position indicated by the data of reading deposit target storage position information, and by the current shape of the data information
State information update is to characterize the information of storage state.
It step 203, is storage state for state indicated by target data information set, current state information
Data information, in response to determine the data information source storage location information and target storage position information indicated by position
The data stored are identical, and the current state information of the data information is updated to the information that state is completed in characterization.
In the present embodiment, for each data information in target data information set, above-mentioned executing subject can be true
It surely is the data information of storage state with the presence or absence of current state.That is, the data information handled by step 202.If deposited
It is being each data information of storage state for current state, can determining the source storage location information of the data information
Target storage position indicated by indicated source storage location and target storage position information, the number that the two positions are stored
According to whether identical.If they are the same, show that data accurately import target storage position by source storage location, so as to by the data
The current state information of information is updated to the information that state is completed in characterization.According to the actual situation, the letter of state is completed in characterization
Breath is also possible to various forms of information.For example, on the basis of indicating storage state with digital " 2 " digital " 3 " can be used
State is completed in expression.
With continued reference to the schematic diagram that Fig. 3, Fig. 3 are according to the application scenarios of the data processing method of the present embodiment.?
In the application scenarios of Fig. 3, data are stored in hive301.Wherein, hive is a Tool for Data Warehouse.Since hive is not propped up
The function of real-time query is held, thus ES (Elasticsearch) 302 is selected to be used as query engine.Thus needing will be in hive301
Data import ES302 in.Firstly, the executing subject of data processing method can obtain target data information set 303 first.
As shown, target data information set 303 under this application scene, can be the pending data of some period generation
The set of information.For purposes of illustration only, by each data information number 1-5.Each data information includes source storage location information,
It is exactly the information of storage location of the data in hive301.In addition, further including target storage position information and current state letter
Breath.Wherein, target storage position information can be the information that data intend the position stored in ES302.Current state information is adopted
With the different state of different digital representations.Digital " 1 " indicates armed state.Digital " 2 " indicate storage state.With number
" 3 " indicate that state is completed.
On this basis, above-mentioned executing subject can use some open sources middleware (such as spark, kafka etc.) it is right
It is handled in target data information set 303.Wherein, spark is the Universal-purpose quick for aiming at large-scale data processing and designing
Computing engines.Kafka is an open source stream process platform.For in target data information set 303, current state information
Indicated state is the data information of armed state, i.e. each data information in data information 1-3 is handled as follows:
The source storage location indicated by the source storage location information in the data information reads data, and the data of reading are stored in target
Target storage position indicated by storage location information, and the current state information of the data information is updated to characterization and has been deposited
The information of storage state.In practice, data information 1-3 can be handled simultaneously by multiple processes in above-mentioned executing subject.
In addition, being storage state for state indicated by target data information set 303, current state information
Data information, i.e. data information 4, above-mentioned executing subject can compare the source storage location information of the data information and target is deposited
The data that position indicated by storage location information stores respectively.In response to determining the source storage location information and mesh of the data information
The data that position indicated by mark storage location information is stored are identical, and the current state information of the data information is updated to table
The information of state is completed in sign.In addition to data information 4, after treatment due to data information 1-3, current state information
It may be updated to " 2 ".Accordingly it is also possible to be carried out by multiple processes for the data information that these current state informations are " 2 "
Reason.
The method that embodiment of the disclosure provides, can obtain target data information set first.Later, for number of targets
It is the data information of armed state according in information aggregate, state indicated by current state information, from the data information
Source storage location information indicated by source storage location read data, the data of reading are stored in target storage position information institute
The target storage position of instruction, and the current state information of the data information is updated to characterize the information of storage state.
It is the data information of storage state, response in target data information set, state indicated by current state information
In the data phase for determining that position indicated by the source storage location information and target storage position information of the data information is stored
Together, the current state information of the data information is updated to the information that state is completed in characterization.Thus, it is possible to which data are deposited from source
Storage space sets deposit target storage position.Meanwhile guaranteeing that the data of deposit target storage position are identical as source storage location data.
With further reference to Fig. 4, it illustrates the processes 400 of another embodiment of data processing method.The data processing
Method 400, comprising the following steps:
Step 401, target data information set is obtained.
It step 402, is armed state for state indicated by target data information set, current state information
Data information, the source storage location indicated by the source storage location information in the data information reads data, by reading
Data are stored in target storage position indicated by target storage position information, and more by the current state information of the data information
The new information to characterize storage state.
It step 403, is storage state for state indicated by target data information set, current state information
Data information, in response to determine the data information source storage location information and target storage position information indicated by position
The data stored are identical, and the current state information of the data information is updated to the information that state is completed in characterization.
In the present embodiment, the specific implementation of step 401-403 and its brought technical effect can be corresponding with reference to Fig. 2
Embodiment in step 201-203, details are not described herein.
Step 404, in response to determining that position indicated by source storage location information and target storage position information is stored
Data it is not identical, at preset time point, push is signified for characterizing source storage location information and target storage position information
The different prompt information of the data that the position shown is stored.
In the present embodiment, above-mentioned executing subject is in response to determining source storage location information and target storage position information institute
The data that the position of instruction is stored are not identical, can at preset time point, push for characterize source storage location information and
The different prompt information of the data that position indicated by target storage position information is stored.According to actual needs, prompt letter
Breath can be various forms of information.Wherein, since there are multiple data informations, according to actual needs, can set in response to
The data that position indicated by the source storage location information and target storage position information of all or part of data information is stored
It is not identical, push prompt information.
Optionally, in the present embodiment, above-mentioned executing subject can also update corresponding in addition to pushing above-mentioned prompt information
The status information of data information.For example, the current state information of the data information can be updated to characterize by above-mentioned executing subject
Store the information of error condition.In this way, can be by position institute indicated by source storage location information and target storage position information
The data of storage are identical and different data information distinguishes.On this basis, above-mentioned executing subject can be according to reality
Border need be to state indicated by current state information store error condition data information targetedly handled.
As an example, be the data information for storing error condition for state indicated by current state information, it is above-mentioned to hold
Row main body can store position to the corresponding target of the data information according to the instruction or preset processing logic that technical staff inputs
It sets stored data to be deleted, and the current state information of the data information is updated to the letter of characterization armed state
Breath.To which above-mentioned executing subject can execute step 402-404 to the data information again.
As an example, above-mentioned executing subject can also be generated according to the instruction or preset decision logic that technical staff inputs
Store error category information.Wherein, storage error category information is for characterizing storage error category.It, can be by technology in practice
Personnel are divided into different storage error categories for mistake is stored previously according to factors such as the reasons for causing storage mistake.In this base
On plinth, for different storage error categories, different processing operations can also be preset.For example, the first storage error category letter
The corresponding operation for deleting the data that the corresponding target storage position of data information is stored of the indicated storage error category of breath.The
The current state information of data information is updated to characterize by storage error category correspondence indicated by two storage error category information
The information of state is completed.I.e., it is possible to ignore the corresponding storage mistake of the storage error category.
Step 405, in response to determining that the current state information of each data information in target data information set is signified
The state shown is to be completed state, push for characterize each data information in target data information set it is processed at
Prompt information.
In the present embodiment, above-mentioned executing subject is in response to determining each data information in target data information set
State indicated by current state information is that state is completed, and is pushed for characterizing each number in target data information set
It is believed that breath it is processed at prompt information.Wherein, prompt information can be various forms of information.
Figure 4, it is seen that compared with the corresponding embodiment of Fig. 2, when increasing for data difference, push prompt letter
The step of breath.So as to be directed to different comparison results, targetedly handled.In addition, also adding when each data
The step of state indicated by the current state information of information is the state that is completed, and pushes related prompt message.So as to and
When by import complete data be shown or carry out subsequent processing.
With further reference to Fig. 5, as the realization to method shown in above-mentioned each figure, present disclose provides a kind of data processing dresses
It sets, the device is corresponding with embodiment of the method shown in Fig. 2, which specifically can be applied in various electronic equipments.
As shown in figure 5, the data processing equipment 500 of the present embodiment includes: acquiring unit 501, storage unit 502 and updates
Unit 503.Wherein, acquiring unit 501 is configured to obtain target data information set, and data information includes source storage location letter
Breath, target storage position information and current state information.Storage unit 502 is configured in target data information set
, the data information that state indicated by current state information is armed state, from the source storage location in the data information
Source storage location indicated by information reads data, and target indicated by the data deposit target storage position information by reading is deposited
Storage space is set, and the current state information of the data information is updated to characterize the information of storage state.503 quilt of updating unit
It is configured to believe the data that in target data information set, state indicated by current state information is storage state
Breath, in response to determining what position indicated by the source storage location information and target storage position information of the data information was stored
Data are identical, and the current state information of the data information is updated to the information that state is completed in characterization.
In the present embodiment, acquiring unit 501, storage unit 502 and the updating unit for including in data processing equipment 500
503 specific processing and brought technical effect can refer to the step 201-203 of the corresponding embodiment of Fig. 2, herein no longer
It repeats.
In some optional implementations of the present embodiment, data information further include: the priority level information.
In some optional implementations of the present embodiment, which can also include: the first push unit (figure
In be not shown).Wherein, the first push unit is configured in response to determine source storage location information and target storage position information
The data that indicated position is stored are not identical, at preset time point, push for characterizing source storage location information and mesh
The different prompt information of data that position indicated by mark storage location information is stored.
In some optional implementations of the present embodiment, which can also include: the second push unit (figure
In be not shown).Wherein, the second push unit is configured in response to determine each data information in target data information set
Current state information indicated by state be to be completed state, push for characterizing each number in target data information set
It is believed that breath it is processed at prompt information.
In some optional implementations of the present embodiment, storage unit 502 can be further configured to: according to excellent
For first rank by high sequence on earth, state indicated by current state information of choosing from target data information set is to be processed
The data information of state executes following processing step: the source indicated by the source storage location information in the data information stores
Data are read in position, by target storage position indicated by the data of reading deposit target storage position information, and by the number
It is believed that the current state information of breath is updated to characterize the information of storage state.
In the present embodiment, acquiring unit can obtain target data information set first.Later, storage unit is for mesh
Data information in data information set, that state indicated by current state information is armed state is marked, is believed from the data
Source storage location indicated by source storage location information in breath reads data, by the data deposit target storage position letter of reading
The indicated target storage position of breath, and the current state information of the data information is updated to characterize the letter of storage state
Breath.Updating unit is the number of storage state in target data information set, state indicated by current state information
It is believed that breath, in response to determining that position indicated by the source storage location information and target storage position information of the data information is deposited
The data of storage are identical, and the current state information of the data information is updated to the information that state is completed in characterization.Thus, it is possible to will
Data are stored in target storage position from source storage location.Meanwhile guaranteeing the data and source storage location of deposit target storage position
Data are identical.
Below with reference to Fig. 6, it illustrates the electronic equipment that is suitable for being used to realize embodiment of the disclosure, (example is as shown in figure 1
Server) 600 structural schematic diagram.Electronic equipment shown in Fig. 6 is only an example, should not be to embodiment of the disclosure
Function and use scope bring any restrictions.
As shown in fig. 6, electronic equipment 600 includes processing unit (such as central processing unit, graphics processor etc.) 601,
Random access storage device can be loaded into according to the program being stored in read-only memory (ROM) 602 or from storage device 608
(RAM) program in 603 and execute various movements appropriate and processing.In RAM 603, it is also stored with the behaviour of electronic equipment 600
Various programs and data needed for making.Processing unit 601, ROM 602 and RAM 603 are connected with each other by bus 604.It is defeated
Enter/export (I/O) interface 605 and is also connected to bus 604.
In general, following device can connect to I/O interface 605: including such as touch screen, touch tablet, keyboard, mouse, taking the photograph
As the input unit 606 of head, microphone, accelerometer, gyroscope etc.；Including such as liquid crystal display (LCD), loudspeaker, vibration
The output device 607 of dynamic device etc.；Storage device 608 including such as tape, hard disk etc.；And communication device 609.Communication device
609, which can permit electronic equipment 600, is wirelessly or non-wirelessly communicated with other equipment to exchange data.Although Fig. 6 shows tool
There is the electronic equipment 600 of various devices, it should be understood that being not required for implementing or having all devices shown.It can be with
Alternatively implement or have more or fewer devices.Each box shown in Fig. 6 can represent a device, can also root
According to needing to represent multiple devices.
Particularly, in accordance with an embodiment of the present disclosure, it may be implemented as computer above with reference to the process of flow chart description
Software program.For example, embodiment of the disclosure includes a kind of computer program product comprising be carried on computer-readable medium
On computer program, which includes the program code for method shown in execution flow chart.In such reality
It applies in example, which can be downloaded and installed from network by communication device 609, or from storage device 608
It is mounted, or is mounted from ROM 602.When the computer program is executed by middle processing unit 601, the reality of the disclosure is executed
Apply the above-mentioned function of limiting in the method for example.
It is situated between it should be noted that computer-readable medium described in embodiment of the disclosure can be computer-readable signal
Matter or computer readable storage medium either the two any combination.Computer readable storage medium for example can be with
System, device or the device of --- but being not limited to --- electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor, or it is any more than
Combination.The more specific example of computer readable storage medium can include but is not limited to: have one or more conducting wires
Electrical connection, portable computer diskette, hard disk, random access storage device (RAM), read-only memory (ROM), erasable type are programmable
Read-only memory (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), light storage device, magnetic are deposited
Memory device or above-mentioned any appropriate combination.In embodiment of the disclosure, computer readable storage medium, which can be, appoints
What include or the tangible medium of storage program that the program can be commanded execution system, device or device use or and its
It is used in combination.And in embodiment of the disclosure, computer-readable signal media may include in a base band or as carrier wave
The data-signal that a part is propagated, wherein carrying computer-readable program code.The data-signal of this propagation can be adopted
With diversified forms, including but not limited to electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal is situated between
Matter can also be that any computer-readable medium other than computer readable storage medium, the computer-readable medium can be sent out
It send, propagate or transmits for by the use of instruction execution system, device or device or program in connection.It calculates
The program code for including on machine readable medium can transmit with any suitable medium, including but not limited to: electric wire, optical cable, RF
Etc. or above-mentioned any appropriate combination.
Above-mentioned computer-readable medium can be included in above-mentioned electronic equipment；It is also possible to individualism, and not
It is fitted into the electronic equipment.Above-mentioned computer-readable medium carries one or more program, when said one or more
When a program is executed by the electronic equipment, so that the electronic equipment: obtaining target data information set, data information includes that source is deposited
Store up location information, target storage position information and current state information；For in target data information set, current state letter
The indicated state of breath is the data information of armed state, the source indicated by the source storage location information in the data information
Storage location reads data, target storage position indicated by the data deposit target storage position information by reading, and will
The current state information of the data information is updated to characterize the information of storage state；For it is in target data information set,
State indicated by current state information is the data information of storage state, in response to determining that the source of the data information stores position
Confidence breath is identical with the data that position indicated by target storage position information is stored, and the current state of the data information is believed
Breath is updated to the information that state is completed in characterization
The calculating of the operation for executing the disclosure can be write with one or more programming languages or combinations thereof
Machine program code, described program design language include object oriented program language-such as Java, Smalltalk, C+
+, it further include conventional procedural programming language-such as " C " language or similar programming language.Program code can
Fully to execute, partly execute on the user computer on the user computer, be executed as an independent software package,
Part executes on the remote computer or executes on a remote computer or server completely on the user computer for part.
In situations involving remote computers, remote computer can pass through the network of any kind --- including local area network (LAN)
Or wide area network (WAN)-is connected to subscriber computer, or, it may be connected to outer computer (such as utilize Internet service
Provider is connected by internet).
Flow chart and block diagram in attached drawing illustrate system, method and the computer of the various embodiments according to the disclosure
The architecture, function and operation in the cards of program product.In this regard, each box in flowchart or block diagram can be with
A part of a module, program segment or code is represented, a part of the module, program segment or code includes one or more
Executable instruction for implementing the specified logical function.It should also be noted that in some implementations as replacements, institute in box
The function of mark can also occur in a different order than that indicated in the drawings.For example, two boxes succeedingly indicated are practical
On can be basically executed in parallel, they can also be executed in the opposite order sometimes, and this depends on the function involved.Also it wants
It is noted that the combination of each box in block diagram and or flow chart and the box in block diagram and or flow chart, Ke Yiyong
The dedicated hardware based system of defined functions or operations is executed to realize, or can be referred to specialized hardware and computer
The combination of order is realized.
Being described in unit involved in embodiment of the disclosure can be realized by way of software, can also be passed through
The mode of hardware is realized.Described unit also can be set in the processor, for example, can be described as: a kind of processor
Including acquiring unit, storage unit and updating unit.Wherein, the title of these units is not constituted under certain conditions to the list
The restriction of member itself, for example, acquiring unit is also described as " obtaining the unit of target data information set ".
Above description is only the preferred embodiment of the disclosure and the explanation to institute's application technology principle.Those skilled in the art
Member is it should be appreciated that invention scope involved in the disclosure, however it is not limited to technology made of the specific combination of above-mentioned technical characteristic
Scheme, while should also cover in the case where not departing from foregoing invention design, it is carried out by above-mentioned technical characteristic or its equivalent feature
Any combination and the other technical solutions formed.Such as features described above has similar function with (but being not limited to) disclosed in the disclosure
Can technical characteristic replaced mutually and the technical solution that is formed.
Claims (12)
1. a kind of data processing method, comprising:
Target data information set is obtained, data information includes source storage location information, target storage position information and current shape
State information；
The data that in the target data information set, state indicated by current state information is armed state are believed
Breath, the source storage location indicated by the source storage location information in the data information read data, and the data of reading are stored in
Target storage position indicated by target storage position information, and the current state information of the data information is updated to characterize
The information of storage state；
For the data letter that in the target data information set, state indicated by current state information is storage state
Breath, in response to determining what position indicated by the source storage location information and target storage position information of the data information was stored
Data are identical, and the current state information of the data information is updated to the information that state is completed in characterization.
2. according to the method described in claim 1, wherein, data information further include: the priority level information.
3. according to the method described in claim 1, wherein, the method also includes:
The data stored in response to position indicated by determining source storage location information and target storage position information are not identical,
At preset time point, push is stored for characterizing position indicated by source storage location information and target storage position information
The different prompt information of data.
4. according to the method described in claim 1, wherein, the method also includes:
In response to determining that state indicated by the current state information of each data information in target data information set is
Completion status, push for characterize each data information in target data information set it is processed at prompt information.
5. described in the target data information set, current state according to the method described in claim 2, wherein
State indicated by information is the data information of armed state, indicated by the source storage location information in the data information
Source storage location reads data, target storage position indicated by the data deposit target storage position information by reading, and
The current state information of the data information is updated to characterize the information of storage state, comprising:
According to priority level by high sequence on earth, chosen indicated by current state information from the target data information set
State be armed state data information, execute following processing step: from the source storage location information in the data information
Indicated source storage location reads data, and target indicated by the data deposit target storage position information by reading stores position
It sets, and the current state information of the data information is updated to characterize the information of storage state.
6. a kind of data processing equipment, comprising:
Acquiring unit, is configured to obtain target data information set, and data information includes source storage location information, target storage
Location information and current state information；
Storage unit, be configured to be for state indicated by the target data information set, current state information
The data information of armed state, the source storage location indicated by the source storage location information in the data information read number
According to, target storage position indicated by the data deposit target storage position information by reading, and working as the data information
Preceding state information updating is the information for characterizing storage state；
Updating unit, be configured to be for state indicated by the target data information set, current state information
The data information of storage state, in response to determining source storage location information and the target storage position information institute of the data information
The data that the position of instruction is stored are identical, and the current state information of the data information is updated to the letter that state is completed in characterization
Breath.
7. device according to claim 6, wherein data information further include: the priority level information.
8. device according to claim 6, wherein described device further include:
First push unit is configured in response to determine position indicated by source storage location information and target storage position information
It is not identical to set stored data, at preset time point, pushes for characterizing source storage location information and target storage position
The different prompt information of the data that position indicated by information is stored.
9. device according to claim 6, wherein described device further include:
Second push unit is configured in response to determine the current state of each data information in target data information set
State indicated by information is the state that is completed, and pushes each data information for characterizing in target data information set and has located
Manage the prompt information completed.
10. device according to claim 7, wherein the storage unit is further configured to:
According to priority level by high sequence on earth, chosen indicated by current state information from the target data information set
State be armed state data information, execute following processing step: from the source storage location information in the data information
Indicated source storage location reads data, and target indicated by the data deposit target storage position information by reading stores position
It sets, and the current state information of the data information is updated to characterize the information of storage state.
11. a kind of server, comprising:
One or more processors；
Storage device is stored thereon with one or more programs,
When one or more of programs are executed by one or more of processors, so that one or more of processors are real
Now such as method as claimed in any one of claims 1 to 5.
12. a kind of computer-readable medium, is stored thereon with computer program, wherein real when described program is executed by processor
Now such as method as claimed in any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910053403.5A CN109766392A (en) | 2019-01-21 | 2019-01-21 | Data processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910053403.5A CN109766392A (en) | 2019-01-21 | 2019-01-21 | Data processing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109766392A true CN109766392A (en) | 2019-05-17 |
Family
ID=66454873
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910053403.5A Pending CN109766392A (en) | 2019-01-21 | 2019-01-21 | Data processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109766392A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111158881A (en) * | 2019-12-31 | 2020-05-15 | 北京字节跳动网络技术有限公司 | Data processing method and device, electronic equipment and computer readable storage medium |
| CN114676095A (en) * | 2022-05-27 | 2022-06-28 | 恒生电子股份有限公司 | File management method and system based on stream processing framework |
| CN117827980A (en) * | 2024-03-06 | 2024-04-05 | 大汉软件股份有限公司 | ES data cross-gate switching method based on distributed links |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1722107A (en) * | 2004-07-15 | 2006-01-18 | 株式会社日立制作所 | Computer system and method for migrating from one storage system to another |
| CN101399603A (en) * | 2007-09-30 | 2009-04-01 | 华为技术有限公司 | Resynchronization method, authentication method and device |
| CN101645930A (en) * | 2009-08-25 | 2010-02-10 | 中兴通讯股份有限公司 | Processing method and system of synchronization failure in dynamic content distributing (DCD) service |
| CN101887456A (en) * | 2010-06-30 | 2010-11-17 | 百视通网络电视技术发展有限责任公司 | System and method for migrating data |
| CN103067433A (en) * | 2011-10-24 | 2013-04-24 | 阿里巴巴集团控股有限公司 | Method, device and system of data migration of distributed type storage system |
| CN103514212A (en) * | 2012-06-27 | 2014-01-15 | 腾讯科技（深圳）有限公司 | Data writing method and system |
| CN105530273A (en) * | 2014-09-29 | 2016-04-27 | 国际商业机器公司 | Method and device for data migration |
| CN105657049A (en) * | 2016-02-26 | 2016-06-08 | 北京皮尔布莱尼软件有限公司 | Incremental data synchronization method and device and mobile terminal |
| CN105893471A (en) * | 2016-03-28 | 2016-08-24 | 联想(北京)有限公司 | Data processing method and electronic equipment |
| CN107943572A (en) * | 2017-11-16 | 2018-04-20 | 平安科技（深圳）有限公司 | Data migration method, device, computer equipment and storage medium |
| US20180267737A1 (en) * | 2017-03-17 | 2018-09-20 | Fujitsu Limited | Information processing apparatus and storage control method therefor |
-
2019
- 2019-01-21 CN CN201910053403.5A patent/CN109766392A/en active Pending
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1722107A (en) * | 2004-07-15 | 2006-01-18 | 株式会社日立制作所 | Computer system and method for migrating from one storage system to another |
| CN101399603A (en) * | 2007-09-30 | 2009-04-01 | 华为技术有限公司 | Resynchronization method, authentication method and device |
| CN101645930A (en) * | 2009-08-25 | 2010-02-10 | 中兴通讯股份有限公司 | Processing method and system of synchronization failure in dynamic content distributing (DCD) service |
| CN101887456A (en) * | 2010-06-30 | 2010-11-17 | 百视通网络电视技术发展有限责任公司 | System and method for migrating data |
| CN103067433A (en) * | 2011-10-24 | 2013-04-24 | 阿里巴巴集团控股有限公司 | Method, device and system of data migration of distributed type storage system |
| CN103514212A (en) * | 2012-06-27 | 2014-01-15 | 腾讯科技（深圳）有限公司 | Data writing method and system |
| CN105530273A (en) * | 2014-09-29 | 2016-04-27 | 国际商业机器公司 | Method and device for data migration |
| CN105657049A (en) * | 2016-02-26 | 2016-06-08 | 北京皮尔布莱尼软件有限公司 | Incremental data synchronization method and device and mobile terminal |
| CN105893471A (en) * | 2016-03-28 | 2016-08-24 | 联想(北京)有限公司 | Data processing method and electronic equipment |
| US20180267737A1 (en) * | 2017-03-17 | 2018-09-20 | Fujitsu Limited | Information processing apparatus and storage control method therefor |
| CN107943572A (en) * | 2017-11-16 | 2018-04-20 | 平安科技（深圳）有限公司 | Data migration method, device, computer equipment and storage medium |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111158881A (en) * | 2019-12-31 | 2020-05-15 | 北京字节跳动网络技术有限公司 | Data processing method and device, electronic equipment and computer readable storage medium |
| CN111158881B (en) * | 2019-12-31 | 2023-05-02 | 北京字节跳动网络技术有限公司 | Data processing method, device, electronic equipment and computer readable storage medium |
| CN114676095A (en) * | 2022-05-27 | 2022-06-28 | 恒生电子股份有限公司 | File management method and system based on stream processing framework |
| CN117827980A (en) * | 2024-03-06 | 2024-04-05 | 大汉软件股份有限公司 | ES data cross-gate switching method based on distributed links |
| CN117827980B (en) * | 2024-03-06 | 2024-05-10 | 大汉软件股份有限公司 | ES data cross-gate switching method based on distributed links |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109697075A (en) | File updating method, system and device | |
| CN108337314B (en) | Distributed system, information processing method and device for primary server | |
| CN109241141A (en) | The training data treating method and apparatus of deep learning | |
| CN109710859A (en) | Data query method and apparatus | |
| CN109814881A (en) | Method and apparatus for disposing data-base cluster | |
| CN109660397A (en) | For acquiring system, the method and apparatus of log | |
| CN109766392A (en) | Data processing method and device | |
| CN108958992A (en) | test method and device | |
| CN109447635A (en) | Information storage means and device for block chain | |
| CN109656923A (en) | A kind of data processing method, device, electronic equipment and storage medium | |
| CN110019080A (en) | Data access method and device | |
| CN108846753A (en) | Method and apparatus for handling data | |
| CN109657174A (en) | Method and apparatus for more new data | |
| CN110222775A (en) | Image processing method, device, electronic equipment and computer readable storage medium | |
| CN109871693A (en) | Method and apparatus for detecting loophole | |
| CN109388626A (en) | Method and apparatus for being numbered to traffic assignments | |
| CN110263565A (en) | Method and apparatus for calling service | |
| CN110321738A (en) | Information processing method and device | |
| CN108897874A (en) | Method and apparatus for handling data | |
| CN110391938A (en) | Method and apparatus for deployment services | |
| CN110287264A (en) | Batch data update method, device and the system of distributed data base | |
| CN109032620A (en) | The configuration file generation method and device of application program | |
| CN110365745A (en) | Data processing system, method, apparatus, server and computer-readable medium | |
| CN109828980A (en) | Data cache method, device, terminal and storage medium | |
| CN109885593A (en) | Method and apparatus for handling information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |