CN111078384A - Method and device for migrating core data, computer equipment and storage medium - Google Patents
Method and device for migrating core data, computer equipment and storage medium Download PDFInfo
- Publication number
- CN111078384A CN111078384A CN201911368125.9A CN201911368125A CN111078384A CN 111078384 A CN111078384 A CN 111078384A CN 201911368125 A CN201911368125 A CN 201911368125A CN 111078384 A CN111078384 A CN 111078384A
- Authority
- CN
- China
- Prior art keywords
- data
- task scheduling
- scheduling system
- core data
- general model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
- G06F9/5016—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals the resource being the memory
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
- G06F9/5088—Techniques for rebalancing the load in a distributed system involving task migration
Abstract
The invention discloses a method and a device for migrating core data, computer equipment and a storage medium, wherein the method comprises the following steps: reading core data needing to be migrated in an original task scheduling system; converting core data to be migrated into general model data; caching the general model data in a memory; reading the general model data from the memory, and converting the general model data into data with a format suitable for a target task scheduling system; and writing the converted data into a target task scheduling system. Compared with manual migration, the method has high migration efficiency and is convenient to operate.
Description
Technical Field
The present invention relates to data processing, and more particularly, to a method and apparatus for migrating core data in different task scheduling systems, a computer device, and a storage medium.
Background
In the existing task scheduling system usage scenario, when a task scheduling system a selected at the beginning is used for a period of time as a usage object, it is desired to migrate data of the task scheduling system a to a task scheduling system B with better performance, and data migration is difficult at present because of more data. At present, the common mode is manual migration or non-migration, the manual migration efficiency is low, and the non-migration is selected, so that only the task scheduling system with better service performance is abandoned.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides a method, a device, computer equipment and a storage medium for migrating core data.
In order to achieve the purpose, the invention adopts the following technical scheme: the method for migrating the core data is applied to a task scheduling system and comprises the following steps:
reading core data needing to be migrated in an original task scheduling system;
converting core data to be migrated into general model data;
caching the general model data in a memory;
reading the general model data from the memory, and converting the general model data into data with a format suitable for a target task scheduling system;
and writing the converted data into a target task scheduling system.
The further technical scheme is as follows: the core data includes projects, project tasks, task dependency information, and timed tasks.
The device for migrating the core data is applied to a task scheduling system and comprises a reading unit, a first conversion unit, a cache unit, a second conversion unit and a writing unit;
the reading unit is used for reading core data needing to be migrated in an original task scheduling system;
the first conversion unit is used for converting the core data to be migrated into the general model data;
the cache unit is used for caching the general model data in a memory;
the second conversion unit is used for reading the general model data from the memory and converting the general model data into data with a format suitable for the target task scheduling system;
and the writing unit is used for writing the converted data into the target task scheduling system.
The further technical scheme is as follows: the core data includes projects, project tasks, task dependency information, and timed tasks.
A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method steps of core data migration as described above when executing the computer program.
A storage medium storing a computer program comprising program instructions which, when executed by a processor, cause the processor to perform the method steps of core data migration as described above.
Compared with the prior art, the invention has the beneficial effects that: according to the method for migrating the core data, the core data needing to be migrated in the original task scheduling system is converted into the universal model data, the universal model data is converted into the data with the format suitable for the target task scheduling system, and the data is written into the target task scheduling system.
The foregoing description is only an overview of the technical solutions of the present invention, and in order to make the technical means of the present invention more clearly understood, the present invention may be implemented according to the content of the description, and in order to make the above and other objects, features, and advantages of the present invention more apparent, the following detailed description will be given of preferred embodiments.
Drawings
FIG. 1 is a flow chart of a method of core data migration in accordance with an embodiment of the present invention;
FIG. 2 is a block diagram illustrating the structure of an embodiment of a core data migration apparatus according to the present invention;
FIG. 3 is a schematic block diagram of one embodiment of a computer device of the present invention.
Detailed Description
In order to more fully understand the technical content of the present invention, the technical solution of the present invention will be further described and illustrated with reference to the following specific embodiments, but not limited thereto.
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the specification of the present invention and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be further understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
The present invention provides a method for migrating core data, which is applied to a task scheduling system, please refer to fig. 1, and the method includes the following steps:
and S10, reading the core data needing to be migrated in the original task scheduling system.
Specifically, in this embodiment, the core data includes a project, a project task, task dependency information, and a timing task. Of course, in other embodiments, the content included in the core data may depend on the characteristics of the service itself. The invention mainly considers the migration of the core data, because when the data of the task scheduling system A is considered to be migrated to the task scheduling system B with better performance, the main data to be migrated is the core data, and the non-core data does not need to be migrated. Of course, the present invention is not only applicable to migrating core data, but also applicable to migrating general data (e.g., non-core data).
In addition, the core data is read from the original task scheduling system by the following modes:
the first method comprises the following steps: directly reading a related data table by directly reading a database (Mysql) for storing data in the task scheduling system; and the second method comprises the following steps: the specified data can be read by the relevant Api of the task scheduling system. Preferably, the reading is performed in the second way, because the current task scheduling systems all provide corresponding APIs, the reading of the core data is convenient by using the API. Other possible ways of reading can of course be used.
And S20, converting the core data to be migrated into the general model data.
And S30, caching the general model data in a memory.
And S40, reading the general model data from the memory, and converting the general model data into data with a format suitable for the target task scheduling system.
Specifically, in the process of converting the general model data into the data with the format suitable for the target task scheduling system, the requirements of different target task scheduling systems on the format are different, so that the requirements of the target task scheduling system on the data format can be met according to the actual conditions by converting the general model data into the data with the format.
And S50, writing the converted data into the target task scheduling system.
Specifically, the method for writing the converted data into the target task scheduling system includes the following steps:
the first method comprises the following steps: by directly inserting data into the memory of the target task scheduling system; and the second method comprises the following steps: and writing through an API provided by the target task scheduling system. Of course, other possible writing schemes may be used.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.
Corresponding to the above method for migrating core data, the present invention further provides a device for migrating core data, which is applied to a task scheduling system, and please refer to fig. 2, where the device includes a reading unit 1, a first converting unit 2, a buffer unit 3, a second converting unit 4, and a writing unit 5;
the system comprises a reading unit 1, a processing unit and a processing unit, wherein the reading unit 1 is used for reading core data needing to be migrated in an original task scheduling system;
the first conversion unit 2 is used for converting the core data to be migrated into the general model data;
the cache unit 3 is used for caching the general model data in a memory;
the second conversion unit 4 is used for reading the general model data from the memory and converting the general model data into data with a format suitable for the target task scheduling system;
and the writing unit 5 is used for writing the converted data into the target task scheduling system.
As shown in fig. 3, the present embodiment further provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the steps of the data storage method of the SSD are implemented.
The computer device 700 may be a terminal or a server. The computer device 700 includes a processor 720, memory, and a network interface 750, which are connected by a system bus 710, where the memory may include non-volatile storage media 730 and internal memory 740.
The non-volatile storage medium 730 may store an operating system 731 and computer programs 732. The computer programs 732, when executed, enable the processor 720 to perform any of the data storage methods of the SSD.
The processor 720 is used to provide computing and control capabilities, supporting the operation of the overall computer device 700.
The internal memory 740 provides an environment for the operation of the computer program 732 in the non-volatile storage medium 730, and when the computer program 732 is executed by the processor 720, the processor 720 can execute any data storage method of the SSD.
The network interface 750 is used for network communication such as sending assigned tasks and the like. Those skilled in the art will appreciate that the configuration shown in fig. 3 is a block diagram of only a portion of the configuration relevant to the present teachings and is not intended to limit the computing device 700 to which the present teachings may be applied, and that a particular computing device 700 may include more or less components than those shown, or may combine certain components, or have a different arrangement of components. Wherein the processor 720 is configured to execute the program code stored in the memory to perform the following steps:
reading core data needing to be migrated in an original task scheduling system;
converting core data to be migrated into general model data;
caching the general model data in a memory;
reading the general model data from the memory, and converting the general model data into data with a format suitable for a target task scheduling system;
and writing the converted data into a target task scheduling system.
The further technical scheme is as follows: the core data includes projects, project tasks, task dependency information, and timed tasks.
It should be understood that, in the embodiment of the present Application, the Processor 720 may be a Central Processing Unit (CPU), and the Processor 720 may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, and the like. Wherein a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
Those skilled in the art will appreciate that the configuration of computer device 700 depicted in FIG. 3 is not intended to be limiting of computer device 700 and may include more or less components than those shown, or some components in combination, or a different arrangement of components.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present invention may be implemented in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, a network device, or the like) or a processor (processor) to execute all or part of the steps of the methods according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
It will be clear to those skilled in the art that, for convenience and simplicity of description, the foregoing division of the functional units is merely illustrated, and in practical applications, the above distribution of functions may be performed by different functional units according to needs, that is, the internal structure of the apparatus may be divided into different functional units to perform all or part of the functions described above. Each functional unit in the embodiments may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units are only used for distinguishing one functional unit from another, and are not used for limiting the protection scope of the application. For the specific working process of the units in the above-mentioned apparatus, reference may be made to the corresponding process in the foregoing method embodiment, which is not described herein again.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described device embodiments are merely illustrative, and for example, the division of the units is only one type of logical function division, and other division manners may be available in actual implementation, for example, a plurality of units or components may be combined or integrated into another device, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The technical contents of the present invention are further illustrated by the examples only for the convenience of the reader, but the embodiments of the present invention are not limited thereto, and any technical extension or re-creation based on the present invention is protected by the present invention. The protection scope of the invention is subject to the claims.
Claims (6)
1. The method for migrating the core data is applied to a task scheduling system and is characterized by comprising the following steps:
reading core data needing to be migrated in an original task scheduling system;
converting core data to be migrated into general model data;
caching the general model data in a memory;
reading the general model data from the memory, and converting the general model data into data with a format suitable for a target task scheduling system;
and writing the converted data into a target task scheduling system.
2. The method of core data migration according to claim 1, wherein said core data includes items, project tasks, task dependency information, and timed tasks.
3. The device for migrating the core data is applied to a task scheduling system and is characterized by comprising a reading unit, a first conversion unit, a cache unit, a second conversion unit and a writing unit;
the reading unit is used for reading core data needing to be migrated in an original task scheduling system;
the first conversion unit is used for converting the core data to be migrated into the general model data;
the cache unit is used for caching the general model data in a memory;
the second conversion unit is used for reading the general model data from the memory and converting the general model data into data with a format suitable for the target task scheduling system;
and the writing unit is used for writing the converted data into the target task scheduling system.
4. The apparatus for core data migration according to claim 3, wherein said core data comprises items, project tasks, task dependency information, and timed tasks.
5. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method steps of core data migration according to any one of claims 1-2 when executing the computer program.
6. A storage medium, characterized in that the storage medium stores a computer program comprising program instructions which, when executed by a processor, cause the processor to carry out the method steps of core data migration according to any one of claims 1-2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911368125.9A CN111078384A (en) | 2019-12-26 | 2019-12-26 | Method and device for migrating core data, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911368125.9A CN111078384A (en) | 2019-12-26 | 2019-12-26 | Method and device for migrating core data, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111078384A true CN111078384A (en) | 2020-04-28 |
Family
ID=70318381
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911368125.9A Pending CN111078384A (en) | 2019-12-26 | 2019-12-26 | Method and device for migrating core data, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111078384A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112130896A (en) * | 2020-08-17 | 2020-12-25 | 深圳云天励飞技术股份有限公司 | Neural network model migration method and device, electronic equipment and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102508929A (en) * | 2011-12-01 | 2012-06-20 | 浪潮(北京)电子信息产业有限公司 | System, device and method for realizing data migration of heterogeneous database |
| CN102999537A (en) * | 2011-09-19 | 2013-03-27 | 阿里巴巴集团控股有限公司 | System and method for data migration |
| CN110275914A (en) * | 2019-04-26 | 2019-09-24 | 武汉众邦银行股份有限公司 | Data migration method, device and the storage medium of integration across database |
-
2019
- 2019-12-26 CN CN201911368125.9A patent/CN111078384A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102999537A (en) * | 2011-09-19 | 2013-03-27 | 阿里巴巴集团控股有限公司 | System and method for data migration |
| CN102508929A (en) * | 2011-12-01 | 2012-06-20 | 浪潮(北京)电子信息产业有限公司 | System, device and method for realizing data migration of heterogeneous database |
| CN110275914A (en) * | 2019-04-26 | 2019-09-24 | 武汉众邦银行股份有限公司 | Data migration method, device and the storage medium of integration across database |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112130896A (en) * | 2020-08-17 | 2020-12-25 | 深圳云天励飞技术股份有限公司 | Neural network model migration method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110674053B (en) | SSD data storage node management method and device, computer equipment and storage medium | |
| EP2565786A1 (en) | Information processing device and task switching method | |
| CN111737221B (en) | Data read-write method, device and equipment of cluster file system and storage medium | |
| CN107277022B (en) | Process marking method and device | |
| CN103514140B (en) | For realizing the reconfigurable controller of configuration information multi-emitting in reconfigurable system | |
| CN109213450B (en) | Associated metadata deleting method, device and equipment based on flash memory array | |
| CN111078384A (en) | Method and device for migrating core data, computer equipment and storage medium | |
| US20130262812A1 (en) | Hardware Managed Allocation and Deallocation Evaluation Circuit | |
| US9471246B2 (en) | Data sharing using difference-on-write | |
| CN106598502B (en) | Data storage method and system | |
| CN111625180B (en) | Data writing method and device and storage medium | |
| WO2018022303A1 (en) | Capturing commands in a multi-engine graphics processing unit | |
| CN103761060A (en) | Data processing method and server | |
| CN109324838B (en) | Execution method and execution device of single chip microcomputer program and terminal | |
| CN109271538A (en) | A kind of picture storage method and relevant device | |
| CN115904240A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN111651124B (en) | Parallel reconstruction method, device, equipment and medium for SSD mapping table multi-core partition | |
| CN111858665B (en) | Method, system, terminal and storage medium for improving soft copy reading performance | |
| CN113342270A (en) | Volume unloading method and device and electronic equipment | |
| CN113220608A (en) | NVMe command processor and processing method thereof | |
| CN107645541B (en) | Data storage method and device and server | |
| CN116601616A (en) | Data processing device, method and related equipment | |
| CN111857604A (en) | Method, apparatus, device and medium for quickly reconstructing packet management mapping reverse lookup table | |
| CN111090633A (en) | Small file aggregation method, device and equipment of distributed file system | |
| CN112035656A (en) | Method, device, computer equipment and medium for quickly previewing document |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |