CN115168478B - Data type conversion method, electronic device and readable storage medium - Google Patents

Data type conversion method, electronic device and readable storage medium Download PDF

Info

Publication number
CN115168478B
CN115168478B CN202211081708.5A CN202211081708A CN115168478B CN 115168478 B CN115168478 B CN 115168478B CN 202211081708 A CN202211081708 A CN 202211081708A CN 115168478 B CN115168478 B CN 115168478B
Authority
CN
China
Prior art keywords
data
processed
data type
type
convertible
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202211081708.5A
Other languages
Chinese (zh)
Other versions
CN115168478A (en
Inventor
陈涛涛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Mingyuan Cloud Technology Co Ltd
Original Assignee
Shenzhen Mingyuan Cloud Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Mingyuan Cloud Technology Co Ltd filed Critical Shenzhen Mingyuan Cloud Technology Co Ltd
Priority to CN202211081708.5A priority Critical patent/CN115168478B/en
Publication of CN115168478A publication Critical patent/CN115168478A/en
Application granted granted Critical
Publication of CN115168478B publication Critical patent/CN115168478B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25—Integrating or interfacing systems involving database management systems
    • G06F16/258—Data format conversion from or to a database
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28—Databases characterised by their database models, e.g. relational or object models
    • G06F16/284—Relational databases
    • G06F16/285—Clustering or classification

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The application discloses a data type conversion method, electronic equipment and a readable storage medium, which are applied to the technical field of computers, wherein the data type conversion method comprises the following steps: acquiring each data to be processed, and predicting a predicted data type corresponding to each data to be processed; classifying each piece of data to be processed according to the predicted data type, the existing data type in the preset data type base and type conversion information corresponding to the existing data type to obtain a classification result, wherein the classification result comprises at least one of convertible data to be processed and non-convertible data to be processed; if the convertible to-be-processed data exists, adjusting the data type of each convertible to-be-processed data; and if the untransformable data to be processed exists, transferring the untransformable data to be processed to a pending queue, so that when the next batch of data to be processed is detected, the untransformable data to be processed is added to the next batch of data to be processed. The method and the device solve the technical problem of low data type conversion efficiency.

Description

Data type conversion method, electronic device and readable storage medium
Technical Field
The present application relates to the field of computer technologies, and in particular, to a data type conversion method, an electronic device, and a readable storage medium.
Background
With the rapid development of science and technology, computer technology is also developed more and more mature, at present, for data processing of multiple services, because the sources of data are different, the types of data are usually diversified, the data types of data need to be uniformly converted into the same data type to process the data, and the data types are usually analyzed manually, so that the data types are uniform.
Disclosure of Invention
The present application mainly aims to provide a data type conversion method, an electronic device, and a readable storage medium, and aims to solve the technical problem of low data type conversion efficiency in the prior art.
In order to achieve the above object, the present application provides a data type conversion method applied to a data type conversion device, where the data type conversion method includes:
acquiring each piece of data to be processed, and predicting a predicted data type corresponding to each piece of data to be processed, wherein at least two data types of data exist in each piece of data to be processed;
classifying the data to be processed according to the predicted data type, the existing data type in a preset data type base and type conversion information corresponding to the existing data type to obtain a classification result, wherein the classification result comprises at least one of convertible data to be processed and non-convertible data to be processed;
if the newly-added data to be processed exists, updating the preset data type base according to the data type corresponding to the newly-added data to be processed, and returning to the execution step: classifying the data to be processed according to the predicted data types, the existing data types in a preset data type library and the type conversion information corresponding to the existing data types to obtain a classification result;
if the convertible to-be-processed data exists, adjusting the data type of each convertible to-be-processed data;
if the non-convertible to-be-processed data exists, transferring the non-convertible to-be-processed data to a pending queue, and adding the non-convertible to-be-processed data to a next batch of to-be-processed data when the next batch of to-be-processed data is detected.
In order to achieve the above object, the present application further provides a data type conversion apparatus, which is applied to a data type conversion device, and the data type conversion apparatus includes:
the device comprises an acquisition module, a prediction module and a processing module, wherein the acquisition module is used for acquiring each piece of data to be processed and predicting a prediction data type corresponding to each piece of data to be processed, and at least two data types of data exist in each piece of data to be processed;
the classification module is used for classifying the data to be processed according to the predicted data type, the existing data type in a preset data type base and type conversion information corresponding to the existing data type to obtain a classification result, wherein the classification result comprises at least one of convertible data to be processed and non-convertible data to be processed;
the adjusting module is used for adjusting the data type of each convertible to-be-processed data if the convertible to-be-processed data exists;
and the transfer module is used for transferring the non-convertible to-be-processed data to a pending queue if the non-convertible to-be-processed data exists so as to add the non-convertible to-be-processed data to the next batch of to-be-processed data when the next batch of to-be-processed data is detected.
The present application further provides an electronic device, the electronic device including: a memory, a processor and a program of the data type conversion method stored on the memory and executable on the processor, which program, when executed by the processor, may implement the steps of the data type conversion method as described above.
The present application also provides a computer-readable storage medium having stored thereon a program for implementing the data type conversion method, which when executed by a processor implements the steps of the data type conversion method as described above.
The present application also provides a computer program product comprising a computer program which, when executed by a processor, performs the steps of the data type conversion method as described above.
Compared with a method for analyzing each data manually so as to unify the data type of each data, the data type conversion method comprises the steps of obtaining each data to be processed and predicting a predicted data type corresponding to each data to be processed, wherein at least two data types of data exist in each data to be processed; classifying the data to be processed according to the predicted data type, the existing data type in a preset data type base and type conversion information corresponding to the existing data type to obtain a classification result, wherein the classification result comprises at least one of convertible data to be processed and non-convertible data to be processed; if the convertible to-be-processed data exists, adjusting the data type of each convertible to-be-processed data; if the untransformable to-be-processed data exists, transferring the untransformable to-be-processed data to a shelving queue, adding the untransformable to-be-processed data to the next batch of to-be-processed data when the next batch of to-be-processed data is detected, classifying each to-be-processed data to obtain a classification result, and converting data types according to the classification result, so that an automatic flow of data type conversion is realized, the technical defect that each data type cannot be unified in a short time due to limited ability of people when the data amount is large or the data source is wide is avoided, and the data type conversion efficiency is improved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, and it is obvious for those skilled in the art to obtain other drawings without inventive exercise.
FIG. 1 is a schematic flow chart illustrating a first embodiment of a data type conversion method according to the present application;
FIG. 2 is a flowchart illustrating a data type conversion method according to a second embodiment of the present application;
FIG. 3 is a schematic view of a scenario of a data type conversion method according to the present application;
FIG. 4 is a schematic structural diagram of an apparatus involved in a data type conversion method in an embodiment of the present application;
fig. 5 is a schematic device structure diagram of a hardware operating environment related to a data type conversion method in an embodiment of the present application.
The objectives, features, and advantages of the present application will be further described with reference to the accompanying drawings.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present application more comprehensible, embodiments of the present application are described in detail below with reference to the accompanying drawings. It is to be understood that the embodiments described are only a few embodiments of the present application and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Example one
In a first embodiment of the data type conversion method of the present application, referring to fig. 1, the data type conversion method includes:
step S10, acquiring each data to be processed, and predicting a predicted data type corresponding to each data to be processed, wherein at least two data types exist in each data to be processed;
step S20, classifying the data to be processed according to the predicted data type, the existing data type in a preset data type base and type conversion information corresponding to the existing data type to obtain a classification result, wherein the classification result comprises at least one of convertible data to be processed and non-convertible data to be processed;
step S30, if the convertible data to be processed exists, adjusting the data type of each convertible data to be processed;
and S40, if the untransformable to-be-processed data exists, transferring the untransformable to-be-processed data to a pending queue, so that when a next batch of to-be-processed data is detected, the untransformable to-be-processed data is added to the next batch of to-be-processed data.
In this embodiment, it should be noted that the to-be-processed data includes a new batch of received data and data in the pending queue, the type conversion information is information integration of convertible data types of each data type, the convertible to-be-processed data is to-be-processed data of a corresponding predicted data type that can be converted into a target data type, the non-convertible to-be-processed data is to-be-processed data of a corresponding predicted data type that cannot be converted into a target data type, and the to-be-processed data may be business data of each department in an enterprise, and may also be other data used for data governance.
Exemplarily, steps S10 to S40 include: acquiring each piece of data to be processed, and predicting a predicted data type corresponding to each piece of data to be processed through a data type prediction model; classifying the data to be processed according to the predicted data types, the existing data types in a preset data type database and the type conversion information corresponding to the existing data types to obtain classification results; if the convertible to-be-processed data exists, acquiring a target data type corresponding to each convertible to-be-processed data, and adjusting the data type of each convertible to-be-processed data to the target data type; if the untransformable data exists, transferring the untransformable to-be-processed data to a pending queue, so that when a next batch of data to be processed is detected, the untransformable to-be-processed data is added to the next batch of data to be processed until the untransformable data to be processed can be converted into transformable data to be processed, and removing the untransformable data to be processed from the pending queue.
In step S10, the step of predicting the predicted data type corresponding to each piece of to-be-processed data includes:
step S21, acquiring a data type prediction model, wherein the data type prediction model comprises a covering network and a prediction network;
step S22, covering the data to be processed through the covering network to obtain covering data of a target quantity and uncovered data corresponding to the covering data;
and step S23, predicting through the prediction network according to uncovered data corresponding to the covered data to obtain a data type corresponding to each covered data, wherein the data type is used as a predicted data type corresponding to each to-be-processed data.
Exemplarily, steps S21 to S23 include: acquiring a trained data type prediction model; inputting the data to be processed into the data type prediction model, obtaining a covering proportion and a covering position corresponding to the data to be processed, and covering the data to be processed through the covering network according to the covering proportion and the covering position to obtain covering data of a target quantity and uncovered data corresponding to the covering data; and predicting through the prediction network according to the uncovered data corresponding to the covering data to obtain the data type corresponding to each covering data, so as to obtain the predicted data type corresponding to each data to be processed.
As an example, the step of obtaining the coverage ratio and the coverage position corresponding to the data to be processed includes:
and acquiring the covering proportion and the covering position corresponding to the data to be processed through a random array, wherein if the random number is (10, 5, 10, 15), for example, the covering proportion is determined to be 10%, and the number for covering the sequential positions of 5, 10, 15 in the data to be processed is determined. Or determining the covering proportion corresponding to the data to be processed through a covering network obtained through training, and acquiring the covering position corresponding to the data to be processed through a random array.
In step S20, before the step of classifying each piece of to-be-processed data according to the predicted data type, an existing data type in a preset data type library, and type conversion information corresponding to the existing data type to obtain a classification result, the method further includes:
step D10, judging whether newly-added data to be processed exists in each data to be processed according to the existing data type and the predicted data type corresponding to each data to be processed;
step D20, if yes, updating the preset data type base according to the data type corresponding to the newly added data to be processed, and returning to the execution step: judging whether newly added data to be processed exists in each data to be processed according to the existing data type and the predicted data type corresponding to each data to be processed;
step D30, if not, executing the following steps: and classifying the data to be processed according to the predicted data type, the existing data type in a preset data type library and the type conversion information corresponding to the existing data type to obtain a classification result.
In this embodiment, it should be noted that the newly added to-be-processed data is data whose corresponding predicted data type does not belong to an existing data type in the preset data type library.
Exemplarily, the steps D10 to D30 include: comparing the predicted data type corresponding to each piece of data to be processed with the existing data type, and judging whether newly-added data to be processed exists in each piece of data to be processed; if the newly added data to be processed exists in the data to be processed, updating the preset data type base according to the data type corresponding to the newly added data to be processed, and returning to the execution step: judging whether newly-added data to be processed exists in the data to be processed or not according to the existing data type and the predicted data type corresponding to the data to be processed; if the newly-added data to be processed does not exist in the data to be processed, executing the following steps: and classifying the data to be processed according to the predicted data type, the existing data type in a preset data type library and the type conversion information corresponding to the existing data type to obtain a classification result.
In step D10, the step of determining whether there is newly added to-be-processed data in each to-be-processed data according to the existing data type and the predicted data type corresponding to each to-be-processed data includes:
step D11, comparing each predicted data type with the existing data types in the preset data type base, and judging whether a newly added data type which does not belong to the existing data types exists in each predicted data type;
step D12, if yes, judging that newly-added data to be processed exists in the data to be processed;
and D13, if not, judging that the newly added data to be processed does not exist in the data to be processed.
Exemplarily, the steps D11 to D13 include: comparing each predicted data type with the existing data types in the preset data type base, judging whether a newly added data type which does not belong to the existing data types exists in each predicted data type, and if the newly added data type which does not belong to the existing data types exists in each predicted data type, judging that newly added data to be processed exists in each data to be processed; if the new data type which does not belong to the existing data type does not exist in each predicted data type, judging that the new data to be processed does not exist in each data to be processed, for example, when the existing data types in a preset data type library are a data type A, a data type B and a data type C, type conversion information only comprises that the data type A and the data type B can be converted with each other, and when the predicted data types in the data to be processed comprise the data type A, the data type B, the data type C and a data type D, taking the data type A and the data type B as convertible data to be processed, taking the data type C as non-convertible data to be processed, and taking the data type D as the new data to be processed.
In step S20, the classifying each piece of to-be-processed data according to the predicted data type, an existing data type in a preset data type library, and type conversion information corresponding to the existing data type to obtain a classification result, where the classification result includes at least one of convertible to-be-processed data and non-convertible to-be-processed data, and the step includes:
step S21, determining a target conversion type corresponding to each data to be processed according to the predicted data type and the type conversion information corresponding to the existing data type;
step S22, if the predicted data type corresponding to the first data to be processed exists in the data to be processed and can be converted into the target conversion type, taking the first data to be processed as the convertible data to be processed;
step S23, if the to-be-processed data has a predicted data type corresponding to a second to-be-processed data that is not convertible to the target conversion type, using the second to-be-processed data as the untransformable to-be-processed data.
Exemplarily, steps S21 to S23 include: determining prediction type conversion information corresponding to each prediction data type according to the prediction data type and type conversion information corresponding to the existing data type, and selecting a target conversion type corresponding to each to-be-processed data according to each prediction type conversion information; if the predicted data type corresponding to the first data to be processed exists in the data to be processed and can be converted into the target conversion type, taking the first data to be processed as the convertible data to be processed; and if the to-be-processed data has the predicted data type corresponding to the second to-be-processed data which can not be converted into the target conversion type, taking the second to-be-processed data as the untranslatable to-be-processed data.
In step S21, the step of determining a target conversion type corresponding to each piece of to-be-processed data according to the predicted data type and the type conversion information corresponding to the existing data type includes:
step A10, integrating convertible quantity of each predicted data type corresponding to each to-be-processed data according to type conversion information corresponding to the existing data type, wherein the convertible quantity is a data quantity of the existing to-be-processed data corresponding to each existing data type and convertible to the predicted data type;
step a20, selecting a target data type, of which the convertible number is greater than a preset number threshold, from the predicted data types.
In this embodiment, it should be noted that the preset number threshold is a convertible number threshold that is preset to determine that the predicted data type can be converted into the data to be processed corresponding to each existing data type and has a larger data number.
Exemplarily, the steps a10 to a20 include: accumulating the data quantity of the existing data to be processed corresponding to each prediction data type in the existing data to be processed which can be converted into the existing data to be processed corresponding to each existing data type according to the type conversion information corresponding to the existing data types, and integrating to obtain the convertible quantity of each prediction data type; and selecting the target data type of which the convertible number is greater than a preset number threshold value from the prediction data types.
Compared with a method for unifying data types of data by manually analyzing the data, the data type conversion method provided by the embodiment of the application acquires the data to be processed and predicts the predicted data type corresponding to the data to be processed, wherein the data to be processed at least has two data types; classifying the data to be processed according to the predicted data type, the existing data type in a preset data type base and type conversion information corresponding to the existing data type to obtain a classification result, wherein the classification result comprises at least one of convertible data to be processed and non-convertible data to be processed; if the convertible to-be-processed data exists, adjusting the data type of each convertible to-be-processed data; if the untransformable to-be-processed data exist, transferring the untransformable to-be-processed data to a shelving queue, adding the untransformable to-be-processed data to the next batch of to-be-processed data when the next batch of to-be-processed data is detected, classifying the to-be-processed data to obtain a classification result, and converting data types according to the classification result, so that an automatic flow of data type conversion is realized, the technical defect that the data types cannot be unified in a short time due to limited human capability when the data amount is large or the data source is wide is avoided, and the data type conversion efficiency is improved.
Example two
Further, referring to fig. 2, based on the first embodiment of the present application, in another embodiment of the present application, the same or similar contents to those of the first embodiment of the present application may be referred to the above description, and are not repeated herein. On this basis, in step S20, before the step of classifying each piece of to-be-processed data according to the predicted data type, an existing data type in a preset data type library, and type conversion information corresponding to the existing data type to obtain a classification result, where the classification result includes at least one of convertible to-be-processed data and non-convertible to-be-processed data, the method further includes:
step B10, obtaining each data type and the data type information which can be converted by each data type, and obtaining type conversion information which corresponds to each data type;
and step B20, constructing the preset data type library according to each data type and the corresponding type conversion information.
In this embodiment, it should be noted that the data type may be a data type derived from a relational database, such as MySQL, oracle, DB2, SQLServer, or a data type derived from a non-relational database, such as OceanBase, HBase, and MongoDB.
Exemplarily, the steps B10 to B20 include: the method comprises the steps that data types corresponding to users are pulled from a cloud server, wherein the users can be individuals or enterprises, transferable data type information of each data type is generated according to the attribute of each data type, and each data type and the corresponding data type information are integrated to obtain type conversion information corresponding to each data type; and constructing the preset data type library according to each data type and the corresponding type conversion information.
In step S11, before the step of obtaining a data type prediction model, where the data type prediction model includes a covering network and a prediction network, the method further includes:
step C10, acquiring a to-be-trained data type prediction model, training samples and real labels corresponding to the training samples, wherein the to-be-trained data type prediction model comprises a to-be-trained covering network and a to-be-trained prediction network;
step C20, covering the training samples through the covering network to be trained to obtain covering samples and uncovered samples with preset number;
step C30, predicting to obtain a training data type corresponding to each covered sample through the to-be-trained prediction network according to the uncovered samples;
and step C40, performing iterative optimization on the preset number and the to-be-trained data type prediction model according to the training data type and the real label to obtain a data type prediction model and a target number.
In this embodiment, it should be noted that the training samples are sample data used for training a data type prediction model corresponding to the data to be processed, and the real labels are real data types corresponding to the training samples.
Exemplarily, the steps C10 to C40 include: acquiring a data type prediction model to be trained, training samples and real labels corresponding to the training samples; covering the training samples through the covering network to be trained to obtain a preset number of covering samples and uncovered samples; according to the uncovered samples, predicting through the to-be-trained prediction network to obtain training data types corresponding to the covered samples; calculating the model loss of the to-be-trained data type prediction model according to the training data type and the real label, further judging whether the model loss is converged, if so, taking the to-be-trained data type prediction model as the preset classification model, taking the preset number as the target number, and if not, updating the to-be-trained preset classification model and correcting the preset number by a preset model updating method based on the gradient of model loss calculation, and returning to the execution step: the method comprises the steps of obtaining a data type prediction model to be trained, training samples and real labels corresponding to the training samples, wherein the preset model updating method comprises a gradient descent method, a gradient ascent method and the like.
In step D20, the step of updating the preset data type base according to the newly added data type corresponding to the newly added to-be-processed data includes:
determining a newly added data type corresponding to the newly added to-be-processed data, and determining a convertible data type corresponding to the newly added to-be-processed data; and updating the preset data type base according to the newly added data type and the corresponding convertible data type.
Illustratively, the method comprises the following steps: acquiring a new data type corresponding to the new data to be processed, and inquiring a cloud server according to the new data type to obtain a convertible data type corresponding to the new data to be processed; and updating the preset data type base according to the newly added data type and the corresponding convertible data type.
As an example, referring to fig. 3, fig. 3 includes data to be processed (illustrated as a, B, \8230; other sources), a predicted data type is obtained by predicting the data to be processed through a data type prediction model, the predicted data type is classified through a preset data type library to obtain convertible data to be processed, each convertible data to be processed is intelligently converted and synchronized to a target data type, and the data to be processed is processed.
Compared with a method for unifying data types of data by manually analyzing the data, the data type conversion method provided by the embodiment of the application acquires the data to be processed and predicts the predicted data type corresponding to the data to be processed, wherein the data to be processed at least has two data types; classifying the data to be processed according to the predicted data type, the existing data type in a preset data type base and type conversion information corresponding to the existing data type to obtain a classification result, wherein the classification result comprises at least one of convertible data to be processed and non-convertible data to be processed; if the convertible to-be-processed data exists, adjusting the data type of each convertible to-be-processed data; if the untransformable to-be-processed data exists, transferring the untransformable to-be-processed data to a shelving queue, adding the untransformable to-be-processed data to the next batch of to-be-processed data when the next batch of to-be-processed data is detected, classifying each to-be-processed data to obtain a classification result, and converting data types according to the classification result, so that an automatic flow of data type conversion is realized, the technical defect that each data type cannot be unified in a short time due to limited ability of people when the data amount is large or the data source is wide is avoided, and the data type conversion efficiency is improved.
EXAMPLE III
An embodiment of the present application further provides a data type conversion apparatus, where the data type conversion apparatus is applied to a data type conversion device, and referring to fig. 4, the data type conversion apparatus includes:
the device comprises an acquisition module, a prediction module and a processing module, wherein the acquisition module is used for acquiring each piece of data to be processed and predicting a prediction data type corresponding to each piece of data to be processed, and at least two data types of data exist in each piece of data to be processed;
the classification module is used for classifying the data to be processed according to the predicted data type, the existing data type in a preset data type database and type conversion information corresponding to the existing data type to obtain a classification result, wherein the classification result comprises at least one of convertible data to be processed and non-convertible data to be processed;
the adjusting module is used for adjusting the data type of each convertible to-be-processed data if the convertible to-be-processed data exists;
and the transfer module is used for transferring the non-convertible to-be-processed data to a pending queue if the non-convertible to-be-processed data exists so as to add the non-convertible to-be-processed data to the next batch of to-be-processed data when the next batch of to-be-processed data is detected.
Optionally, before the step of classifying each piece of to-be-processed data according to the predicted data type, an existing data type in a preset data type library, and type conversion information corresponding to the existing data type to obtain a classification result, where the classification result includes at least one of convertible to-be-processed data and non-convertible to-be-processed data, the data type conversion device is further configured to:
acquiring each data type and data type information which can be converted by each data type to obtain type conversion information corresponding to each data type;
and constructing the preset data type library according to each data type and the corresponding type conversion information.
Optionally, the obtaining module is further configured to:
acquiring a data type prediction model, wherein the data type prediction model comprises a covering network and a prediction network;
covering the data to be processed through the covering network to obtain covering data of a target quantity and uncovered data corresponding to the covering data;
and predicting through the prediction network according to the uncovered data corresponding to the covering data to obtain a data type corresponding to each covering data as a predicted data type corresponding to each to-be-processed data.
Optionally, before the obtaining the data type prediction model, wherein the data type prediction model includes a step of covering a network and predicting the network, the data type conversion apparatus is further configured to:
acquiring a to-be-trained data type prediction model, training samples and real labels corresponding to the training samples, wherein the to-be-trained data type prediction model comprises a to-be-trained covering network and a to-be-trained prediction network;
covering the training samples through the covering network to be trained to obtain a preset number of covering samples and uncovered samples;
according to the uncovered samples, predicting through the to-be-trained prediction network to obtain training data types corresponding to the covered samples;
and performing iterative optimization on the preset number and the to-be-trained data type prediction model according to the training data type and the real label to obtain a data type prediction model.
Optionally, before the step of classifying each piece of to-be-processed data according to the predicted data type, an existing data type in a preset data type library, and type conversion information corresponding to the existing data type to obtain a classification result, where the classification result includes at least one of convertible to-be-processed data and non-convertible to-be-processed data, the data type conversion device is further configured to:
judging whether newly-added data to be processed exists in the data to be processed or not according to the existing data type and the predicted data type corresponding to the data to be processed;
if the new data exists, updating the preset data type base according to the data type corresponding to the newly added data to be processed, and returning to the execution step: classifying the data to be processed according to the predicted data type, the existing data type in a preset data type base and type conversion information corresponding to the existing data type to obtain a classification result, wherein the classification result comprises at least one of convertible data to be processed and non-convertible data to be processed until the newly-added data to be processed is classified.
Optionally, the data type conversion apparatus is further configured to:
comparing each predicted data type with the existing data types in the preset data type base, and judging whether a newly added data type which does not belong to the existing data types exists in each predicted data type;
if yes, judging that newly-added data to be processed exists in the data to be processed;
if not, judging that the newly added data to be processed does not exist in the data to be processed.
Optionally, the classification module is further configured to:
determining a target conversion type corresponding to each data to be processed according to the predicted data type and the type conversion information corresponding to the existing data type;
if the predicted data type corresponding to the first data to be processed exists in the data to be processed and can be converted into the target conversion type, taking the first data to be processed as the convertible data to be processed;
and if the to-be-processed data has the predicted data type corresponding to the second to-be-processed data which can not be converted into the target conversion type, taking the second to-be-processed data as the untranslatable to-be-processed data.
Optionally, the classification module is further configured to:
integrating the convertible number of each predicted data type corresponding to each to-be-processed data according to type conversion information corresponding to the existing data type, wherein the convertible number is the data number of the existing to-be-processed data corresponding to each existing data type and convertible to the predicted data type;
and selecting the target data type of which the convertible number is greater than a preset number threshold value from all the predicted data types.
The data type conversion device provided by the application adopts the data type conversion method in the embodiment, and the technical problem of low data type conversion efficiency is solved. Compared with the prior art, the beneficial effects of the data type conversion device provided by the embodiment of the present application are the same as the beneficial effects of the data type conversion method provided by the above embodiment, and other technical features in the data type conversion device are the same as the features disclosed in the above embodiment method, which are not described herein again.
Example four
An embodiment of the present application provides an electronic device, which includes: at least one processor; and a memory communicatively coupled to the at least one processor; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the data type conversion method in the above embodiments.
Referring now to FIG. 5, shown is a schematic diagram of an electronic device suitable for use in implementing embodiments of the present disclosure. The electronic devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., car navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. The electronic device shown in fig. 5 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.
As shown in fig. 5, the electronic device may include a processing means (e.g., a central processing unit, a graphic processor, etc.) that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) or a program loaded from a storage means into a Random Access Memory (RAM). In the RAM, various programs and data necessary for the operation of the electronic apparatus are also stored. The processing device, the ROM, and the RAM are connected to each other through a bus. An input/output (I/O) interface is also connected to the bus.
Generally, the following systems may be connected to the I/O interface: input devices including, for example, touch screens, touch pads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, and the like; output devices including, for example, liquid Crystal Displays (LCDs), speakers, vibrators, and the like; storage devices including, for example, magnetic tape, hard disk, etc.; and a communication device. The communication means may allow the electronic device to communicate wirelessly or by wire with other devices to exchange data. While the figures illustrate an electronic device with various systems, it is to be understood that not all illustrated systems are required to be implemented or provided. More or fewer systems may alternatively be implemented or provided.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer-readable medium, the computer program comprising program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication means, or installed from a storage means, or installed from a ROM. The computer program, when executed by a processing device, performs the above-described functions defined in the methods of the embodiments of the present disclosure.
The electronic device provided by the application adopts the data type conversion method in the embodiment, so that the technical problem of low data type conversion efficiency is solved. Compared with the prior art, the beneficial effects of the electronic device provided by the embodiment of the present application are the same as the beneficial effects of the data type conversion method provided by the above embodiment, and other technical features of the electronic device are the same as those disclosed by the above embodiment method, which are not repeated herein.
It should be understood that portions of the present disclosure may be implemented in hardware, software, firmware, or a combination thereof. In the foregoing description of embodiments, the particular features, structures, materials, or characteristics may be combined in any suitable manner in any one or more embodiments or examples.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
EXAMPLE five
The present embodiment provides a computer-readable storage medium having computer-readable program instructions stored thereon for performing the method of the data type conversion method in the above-described embodiments.
The computer readable storage medium provided by the embodiments of the present application may be, for example, a usb disk, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, or device, or a combination of any of the above. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present embodiment, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, or device. Program code embodied on a computer readable storage medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
The computer-readable storage medium may be embodied in an electronic device; or may be present alone without being incorporated into the electronic device.
The computer readable storage medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: acquiring each piece of data to be processed, and predicting a predicted data type corresponding to each piece of data to be processed, wherein at least two data types of data exist in each piece of data to be processed; classifying the data to be processed according to the predicted data type, the existing data type in a preset data type base and type conversion information corresponding to the existing data type to obtain a classification result, wherein the classification result comprises at least one of convertible data to be processed and non-convertible data to be processed; if the convertible to-be-processed data exists, adjusting the data type of each convertible to-be-processed data; if the non-convertible to-be-processed data exists, transferring the non-convertible to-be-processed data to a pending queue, and adding the non-convertible to-be-processed data to a next batch of to-be-processed data when the next batch of to-be-processed data is detected.
Computer program code for carrying out operations for aspects of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in the embodiments of the present disclosure may be implemented by software or hardware. Wherein the names of the modules do not in some cases constitute a limitation of the unit itself.
The computer-readable storage medium provided by the application stores computer-readable program instructions for executing the data type conversion method, and solves the technical problem of low data type conversion efficiency. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the embodiment of the present application are the same as the beneficial effects of the data type conversion method provided by the foregoing implementation, and are not described herein again.
EXAMPLE six
The present application also provides a computer program product comprising a computer program which, when executed by a processor, performs the steps of the data type conversion method as described above.
The computer program product provided by the application solves the technical problem of low data type conversion efficiency. Compared with the prior art, the beneficial effects of the computer program product provided by the embodiment of the present application are the same as the beneficial effects of the data type conversion method provided by the above embodiment, and are not described herein again.
The above description is only a preferred embodiment of the present application, and not intended to limit the scope of the present application, and all equivalent structures or equivalent processes, which are directly or indirectly applied to other related technical fields, and which are not limited by the present application, are also included in the scope of the present application.

Claims (7)

1. A data type conversion method, characterized in that the data type conversion method comprises:
acquiring each piece of data to be processed, and predicting a predicted data type corresponding to each piece of data to be processed, wherein at least two data types of data exist in each piece of data to be processed;
classifying the data to be processed according to the predicted data type, the existing data type in a preset data type base and type conversion information corresponding to the existing data type to obtain a classification result, wherein the classification result comprises at least one of convertible data to be processed and non-convertible data to be processed;
if the convertible to-be-processed data exists, adjusting the data type of each convertible to-be-processed data;
if the untransformable to-be-processed data exists, transferring the untransformable to-be-processed data to a pending queue, so that when a next batch of to-be-processed data is detected, the untransformable to-be-processed data is added to the next batch of to-be-processed data;
the step of predicting the prediction data type corresponding to each piece of the data to be processed comprises the following steps:
acquiring a data type prediction model, wherein the data type prediction model comprises a covering network and a prediction network;
covering the data to be processed through the covering network to obtain covering data of a target quantity and uncovered data corresponding to the covering data;
according to uncovered data corresponding to the covering data, predicting through the prediction network to obtain a data type corresponding to each covering data, and using the data type as a predicted data type corresponding to each data to be processed;
classifying the data to be processed according to the predicted data type, the existing data type in a preset data type base and type conversion information corresponding to the existing data type to obtain a classification result, wherein the classification result comprises at least one of convertible data to be processed and non-convertible data to be processed, and the classification step comprises the following steps:
determining a target conversion type corresponding to each data to be processed according to the predicted data type and the type conversion information corresponding to the existing data type;
if the predicted data type corresponding to the first data to be processed exists in the data to be processed and can be converted into the target conversion type, taking the first data to be processed as the convertible data to be processed;
if the to-be-processed data has the predicted data type corresponding to the second to-be-processed data which can not be converted into the target conversion type, taking the second to-be-processed data as the non-convertible to-be-processed data;
the step of determining a target conversion type corresponding to each piece of data to be processed according to the predicted data type and the type conversion information corresponding to the existing data type includes:
integrating convertible quantity of each corresponding predicted data type in each piece of to-be-processed data according to type conversion information corresponding to the existing data type, wherein the convertible quantity is the data quantity of the existing to-be-processed data corresponding to each type of the existing data and convertible into the predicted data type;
and selecting the target data type of which the convertible number is greater than a preset number threshold value from the prediction data types.
2. The data type conversion method of claim 1, wherein before the step of obtaining a data type prediction model, wherein the data type prediction model comprises a masking network and a prediction network, further comprising:
acquiring a data type prediction model to be trained, training samples and real labels corresponding to the training samples, wherein the data type prediction model to be trained comprises a covering network to be trained and a prediction network to be trained;
covering the training samples through the covering network to be trained to obtain covering samples and uncovered samples with preset quantity;
according to the uncovered samples, predicting through the to-be-trained prediction network to obtain training data types corresponding to the covered samples;
and performing iterative optimization on the preset number and the to-be-trained data type prediction model according to the training data type and the real label to obtain a data type prediction model and a target number.
3. The method for converting data types according to claim 1, wherein before the step of classifying each of the to-be-processed data according to the predicted data type, an existing data type in a preset data type database, and type conversion information corresponding to the existing data type to obtain a classification result, the classification result includes at least one of convertible to-be-processed data and non-convertible to-be-processed data, the method further comprises:
acquiring each data type and data type information which can be converted by each data type to obtain type conversion information corresponding to each data type;
and constructing the preset data type library according to each data type and the corresponding type conversion information.
4. The data type conversion method according to claim 1, wherein before the step of classifying each of the data to be processed according to the predicted data type, an existing data type in a preset data type library and type conversion information corresponding to the existing data type to obtain a classification result, the classification result includes at least one of convertible data to be processed and non-convertible data to be processed, the method further comprises:
judging whether newly-added data to be processed exists in the data to be processed or not according to the existing data type and the predicted data type corresponding to the data to be processed;
if yes, updating the preset data type base according to the data type corresponding to the newly added data to be processed, and returning to the execution step: judging whether newly-added data to be processed exists in the data to be processed or not according to the existing data type and the predicted data type corresponding to the data to be processed;
if not, executing the following steps: and classifying the data to be processed according to the predicted data type, the existing data type in a preset data type library and the type conversion information corresponding to the existing data type to obtain a classification result.
5. The data type conversion method according to claim 4, wherein the step of determining whether there is newly added to-be-processed data in each to-be-processed data according to the existing data type and the predicted data type corresponding to each to-be-processed data comprises:
comparing each predicted data type with the existing data types in the preset data type base, and judging whether a newly added data type which does not belong to the existing data types exists in each predicted data type;
if yes, judging that newly-added data to be processed exists in the data to be processed;
if not, judging that the newly added data to be processed does not exist in the data to be processed.
6. An electronic device, characterized in that the electronic device comprises:
at least one processor; and (c) a second step of,
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the steps of the data type conversion method of any one of claims 1 to 5.
7. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon a program for implementing a data type conversion method, the program being executed by a processor to implement the steps of the data type conversion method according to any one of claims 1 to 5.
CN202211081708.5A 2022-09-06 2022-09-06 Data type conversion method, electronic device and readable storage medium Active CN115168478B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211081708.5A CN115168478B (en) 2022-09-06 2022-09-06 Data type conversion method, electronic device and readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211081708.5A CN115168478B (en) 2022-09-06 2022-09-06 Data type conversion method, electronic device and readable storage medium

Publications (2)

Publication Number Publication Date
CN115168478A CN115168478A (en) 2022-10-11
CN115168478B true CN115168478B (en) 2022-11-29

Family

ID=83481482

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211081708.5A Active CN115168478B (en) 2022-09-06 2022-09-06 Data type conversion method, electronic device and readable storage medium

Country Status (1)

Country Link
CN (1) CN115168478B (en)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109543154A (en) * 2018-10-11 2019-03-29 天津字节跳动科技有限公司 Method for converting types, device, storage medium and the electronic equipment of list data
CN110413786A (en) * 2019-07-26 2019-11-05 北京智游网安科技有限公司 Data processing method, intelligent terminal and storage medium based on web page text classification
CN111314329A (en) * 2020-02-03 2020-06-19 杭州迪普科技股份有限公司 Traffic intrusion detection system and method
CN111429968A (en) * 2020-03-11 2020-07-17 至本医疗科技（上海）有限公司 Method, electronic device, and computer storage medium for predicting tumor type
CN112862630A (en) * 2021-03-08 2021-05-28 海南省电力学校(海南省电力技工学校) Weather type index interval-based photovoltaic power prediction method, terminal and medium
CN112904370A (en) * 2019-11-15 2021-06-04 辉达公司 Multi-view deep neural network for lidar sensing
CN113435520A (en) * 2021-06-30 2021-09-24 深圳市商汤科技有限公司 Neural network training method, device, equipment and computer readable storage medium

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10678225B2 (en) * 2013-03-04 2020-06-09 Fisher-Rosemount Systems, Inc. Data analytic services for distributed industrial performance monitoring
US11263707B2 (en) * 2017-08-08 2022-03-01 Indigo Ag, Inc. Machine learning in agricultural planting, growing, and harvesting contexts
US20220061236A1 (en) * 2020-08-25 2022-03-03 The Board Of Trustees Of The University Of Illinois Accessing agriculture productivity and sustainability
CN114003721A (en) * 2021-11-02 2022-02-01 城云科技（中国）有限公司 Construction method, device and application of dispute event type classification model
CN114298232A (en) * 2021-12-30 2022-04-08 深圳前海微众银行股份有限公司 Method, device and storage medium for determining type information of user

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109543154A (en) * 2018-10-11 2019-03-29 天津字节跳动科技有限公司 Method for converting types, device, storage medium and the electronic equipment of list data
CN110413786A (en) * 2019-07-26 2019-11-05 北京智游网安科技有限公司 Data processing method, intelligent terminal and storage medium based on web page text classification
CN112904370A (en) * 2019-11-15 2021-06-04 辉达公司 Multi-view deep neural network for lidar sensing
CN111314329A (en) * 2020-02-03 2020-06-19 杭州迪普科技股份有限公司 Traffic intrusion detection system and method
CN111429968A (en) * 2020-03-11 2020-07-17 至本医疗科技（上海）有限公司 Method, electronic device, and computer storage medium for predicting tumor type
CN112862630A (en) * 2021-03-08 2021-05-28 海南省电力学校(海南省电力技工学校) Weather type index interval-based photovoltaic power prediction method, terminal and medium
CN113435520A (en) * 2021-06-30 2021-09-24 深圳市商汤科技有限公司 Neural network training method, device, equipment and computer readable storage medium

Also Published As

Publication number Publication date
CN115168478A (en) 2022-10-11

Similar Documents

Publication Publication Date Title
CN110865898B (en) Method, device, medium and equipment for converging crash call stack
CN111897827B (en) Data updating method and system for data warehouse and electronic equipment
CN113159453B (en) Resource data prediction method, device, equipment and storage medium
CN111857720B (en) User interface state information generation method and device, electronic equipment and medium
CN111428165A (en) Three-dimensional model display method and device and electronic equipment
CN113392018B (en) Traffic distribution method and device, storage medium and electronic equipment
CN114417169A (en) Information recommendation optimization method, device, medium, and program product
CN115994100B (en) System activity detection method and device, electronic equipment and readable storage medium
CN115168478B (en) Data type conversion method, electronic device and readable storage medium
CN116151961A (en) Credit risk prediction method, electronic device and readable storage medium
CN111143464B (en) Data acquisition method and device and electronic equipment
CN115578082A (en) Method and device for generating flow of automatic robot, electronic equipment and storage medium
CN111399902B (en) Client source file processing method and device, readable medium and electronic equipment
CN110458743B (en) Community management method, device, equipment and storage medium based on big data analysis
CN109284350B (en) Method and device for updating search content, storage medium and electronic equipment
CN111382556A (en) Data conversion method, device, equipment and storage medium
CN116910669B (en) Data classification method, device, electronic equipment and readable storage medium
CN117785977B (en) Metadata acquisition method, device and equipment
CN116483808B (en) Data migration method, device, electronic equipment and computer readable medium
CN112286609B (en) Method and device for managing shortcut setting items of intelligent terminal
CN116820354B (en) Data storage method, data storage device and data storage system
CN113672675B (en) Data detection method and device and electronic equipment
CN115374762B (en) Code template sharing method and device based on VS code, electronic equipment and medium
CN113778387B (en) Method and device for generating code
CN118860539A (en) Information processing method, device, electronic equipment and computer readable medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant