CN112348421A

CN112348421A - Data processing method and device

Info

Publication number: CN112348421A
Application number: CN201910730429.9A
Authority: CN
Inventors: 宋玉飞
Original assignee: Beijing Gridsum Technology Co Ltd
Current assignee: Beijing Gridsum Technology Co Ltd
Priority date: 2019-08-08
Filing date: 2019-08-08
Publication date: 2021-02-09

Abstract

The invention provides a data processing method and a data processing device, which are used for acquiring material information of a material to be analyzed and a data analysis model for analyzing the material information under a preset analysis dimension, inputting the material information into the data analysis model and obtaining an information analysis result under the corresponding preset analysis dimension, namely, the function of analyzing and managing the material main data of the material can be realized through the data processing method and the data processing device.

Description

Data processing method and device

Technical Field

The present invention relates to the field of data processing, and in particular, to a data processing method and apparatus.

Background

In the operation process of an enterprise, various main data, such as material main data, can be involved. The material generally refers to all materials used in the production, operation and operation processes of enterprises, and can be a real object (such as raw materials, semi-finished products, low-value consumable products and the like) or a service (such as maintenance service, operation and maintenance and the like). I.e. the material comprises raw materials, semi-finished products, services, etc., and the master data of the material comprises the corresponding description of the material purchased, produced and stored in the stock.

At present, the data volume accumulated by the material main data in the enterprise operation process is large, and the quality of the material main data of different materials is uneven, so that a method capable of performing unified analysis and management on the material main data is urgently needed to eliminate the defects.

Disclosure of Invention

In view of the above, the present invention provides a data processing method and apparatus that overcomes or at least partially solves the above problems.

A method of data processing, comprising:

acquiring material information of a material to be analyzed and a data analysis model for analyzing the material information under a preset analysis dimension; the data analysis models are constructed in advance, and different preset analysis dimensions correspond to different data analysis models;

and inputting the material information into the data analysis model to obtain an information analysis result corresponding to a preset analysis dimension.

Preferably, the data analysis model comprises a material similarity model; the material similarity model is used for determining the similarity between each material except the material to be analyzed and the material to be analyzed in a database in which the material information of the material to be analyzed is stored;

correspondingly, the material information is input into the data analysis model, and an information analysis result corresponding to a preset analysis dimension is obtained, including:

inputting the material information into the material similarity model to obtain the similarity between each material except the material to be analyzed and the material to be analyzed, and determining the material with the similarity larger than a preset similarity threshold as a target material;

acquiring material information corresponding to the target material;

and taking the similarity between the target material and the material to be analyzed and material information corresponding to the target material as an information analysis result under a corresponding preset analysis dimension.

Preferably, the process for constructing the material similarity model comprises the following steps:

acquiring first material sample information of a first material sample; the first material sample information comprises material main data of the material sample and business data of the material sample; the business data is data describing a material transaction process; the material samples comprise similar material samples and non-similar material samples;

determining a business scene of the material sample based on the business data of the material sample; the business scene is used for representing the transaction type of the transaction performed by the material;

and based on the material main data and the service scene of the material sample, obtaining the material similarity model by machine learning of the material main data and the service scene corresponding to the similar material sample and the dissimilar material sample respectively.

Preferably, the material information comprises material main data of the material to be analyzed and business data of the material to be analyzed; the business data is data describing a material transaction process;

after material information of a material to be analyzed and a data analysis model for analyzing the material information under a preset analysis dimension are obtained, the method further comprises the following steps:

determining a business scene of the material to be analyzed according to the business data of the material to be analyzed; the business scene is used for representing the transaction type of the transaction performed by the material;

correspondingly, inputting the material information into the material similarity model to obtain the similarity between each material except the material to be analyzed and the material to be analyzed, and determining the material with the similarity larger than a preset similarity threshold as a target material, including:

inputting the material information into the material similarity model, and obtaining similarity corresponding to material main data and service scenes between each material except the material to be analyzed and the material to be analyzed through semantic analysis;

and screening out the materials with the similarity of the main material data and the business scene respectively larger than a set specified threshold value as the target materials.

Preferably, the data analysis model comprises a data integrity evaluation model; the data integrity evaluation model stores the corresponding relation between various materials and the data required for describing the materials and the corresponding relation between the data required for describing the various materials and the data format required by each data;

inputting the material information into the data integrity evaluation model to determine whether the data in the material information is complete or not based on the corresponding relationship between the various materials and the data required for describing the materials, determine missing data when the data is incomplete, determine whether the data format of each data in the material information is correct or not based on the corresponding relationship between the data required for describing the various materials and the data format required by each data, and determine the data with the incorrect data format when the data format is incorrect;

and taking the analysis result of whether the data in the material information is complete, the analysis result of whether the data format of each data in the material information is correct, and the determined missing data and the data with the incorrect data format when the data is incomplete or the data format is incorrect as the information analysis result under the corresponding preset analysis dimensionality.

Preferably, the generating process of the data integrity evaluation model includes:

acquiring second material sample information of a second material sample; the second material sample information comprises sample data required by different materials under different service scenes and the format of each sample data; the business scene is used for representing the transaction type of the transaction performed by the material;

determining the corresponding relation between various materials and data required for describing the materials according to the sample data required by the different materials under different service scenes;

and determining the corresponding relation between the data required for describing various materials and the data format required by each data according to the sample data required by the different materials in different service scenes and the format of each sample data.

Preferably, the data analysis model comprises a material information validity model; the material information validity model is used for calculating the time difference between the last inquired time of the material information and the current time, and determining whether the material information is valid or not based on the size relation between the time difference and a preset time threshold;

inputting the material information into the data analysis model to obtain an information analysis result corresponding to a preset analysis dimension, wherein the information analysis result comprises the following steps:

inputting the material information into the data analysis model to obtain the relationship between the time difference between the last inquired time and the current time of the material information and the preset time threshold;

judging whether the material is effective or not based on the size relation;

and taking the result of whether the material is effective as an information analysis result under the corresponding preset analysis dimensionality.

Preferably, the data analysis model comprises a material information timeliness model; the material information timeliness model is used for determining the current business progress of the material to be analyzed according to the business data of the material to be analyzed, and determining whether the current business progress is in uncleaned business or not so as to determine whether the business corresponding to the material to be analyzed is uncleaned business or not;

inputting the material business data into the material information timeliness model to obtain the current business progress of the material to be analyzed and the analysis result of whether the business corresponding to the material to be analyzed is the unclean business;

and taking the current business progress of the material to be analyzed and an analysis result of whether the business corresponding to the material to be analyzed is unsettled business or not as an information analysis result under a corresponding preset analysis dimension.

A data processing apparatus comprising:

the system comprises an information acquisition module, a data analysis module and a data analysis module, wherein the information acquisition module is used for acquiring material information of a material to be analyzed and a data analysis model for analyzing the material information under a preset analysis dimension; the data analysis models are constructed in advance, and different preset analysis dimensions correspond to different data analysis models;

and the information analysis module is used for inputting the material information into the data analysis model to obtain an information analysis result corresponding to a preset analysis dimension.

A processor for executing a program, wherein the program executes the data processing method described above.

By means of the technical scheme, the invention provides a data processing method and a data processing device, material information of a material to be analyzed and a data analysis model for analyzing the material information in a preset analysis dimension are obtained, the material information is input into the data analysis model, and an information analysis result corresponding to the preset analysis dimension is obtained, namely, the function of analyzing and managing the material main data of the material can be realized through the data processing method and the data processing device.

The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.

Drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:

fig. 1 is a flowchart illustrating a method of a data processing method according to an embodiment of the present invention;

FIG. 2 is a flow chart of a method of another data processing method according to an embodiment of the present invention;

FIG. 3 is a flow chart of a method of another data processing method according to an embodiment of the present invention;

fig. 4 is a schematic structural diagram illustrating a data processing apparatus according to an embodiment of the present invention.

Detailed Description

Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

An embodiment of the present invention provides a data processing method, and with reference to fig. 1, the data processing method may include:

s11, acquiring material information of the material to be analyzed and a data analysis model for analyzing the material information under a preset analysis dimension.

The data analysis models are constructed in advance, and different preset analysis dimensions correspond to different data analysis models. In practical application, the data analysis model is various, and may include a material similarity model, a data integrity evaluation model, a material information validity model, a material information timeliness model, and the like. Each data analysis model is used for analyzing one characteristic of material information of the material to be analyzed, such as data integrity, effectiveness, timeliness, whether similar materials exist or not and the like.

The material information of the material to be analyzed can comprise material main data of the material to be analyzed and business data of the material to be analyzed, wherein the material main data comprises data such as material codes, types, units, prices, cost and the like. The business data may include application departments, volume of deals, and the like. Taking the material to be analyzed as iron as an example, the main data of the material comprises: material numbering: 001. type (2): raw materials and units: ton, price: 3500 Yuan/ton. The service data comprises: application department: sales department, volume of deals: 100 tons.

The source of the material information of the material to be analyzed can be ERP (Enterprise Resource Planning), MES (Manufacturing Execution System), bang permanent EBS engineering Enterprise management software, and the like. After the material information is obtained, the material information may be analyzed, cleaned, and converted using an Extract-Transform-Load (ETL) technology.

And S12, inputting the material information into the data analysis model to obtain an information analysis result corresponding to the preset analysis dimension.

After the data analysis model is obtained, the material information is directly input into the model, and a corresponding information analysis result can be obtained.

It should be noted that the information analysis results corresponding to different data analysis models are different, and if the data analysis model is the data integrity evaluation model, the information analysis result is whether the data is complete, and a missing part, and if the data analysis model is the material information validity model, the information analysis result is whether the material information is valid, and the like.

In this embodiment, the material information of the material to be analyzed and the data analysis model for analyzing the material information in the preset analysis dimension are acquired, and the material information is input into the data analysis model to obtain the information analysis result in the corresponding preset analysis dimension, that is, the function of analyzing and managing the material main data of the material can be realized by the invention.

The data analysis model is introduced, and when the data analysis model is different, the executed analysis process is also different, and now, the following specific descriptions are respectively given:

1. the data analysis model includes a material similarity model.

The material similarity model is used for determining the similarity between each material except the material to be analyzed and the material to be analyzed in the database in which the material information of the material to be analyzed is stored.

The similarity between each material except the material to be analyzed and the material to be analyzed is determined in order to determine whether the condition of one material with multiple codes exists or not, and ideally, one material should correspond to one code, but in the encoding process, due to reasons such as manual errors and the like, one material with multiple codes can appear, and at the moment, the multiple codes all correspond to one material, so that the management of the material is not facilitated, and the condition of one material with multiple codes needs to be screened out and the codes are adjusted.

The material similarity model is generated in advance based on machine learning, and a construction process of the material similarity model is described, referring to fig. 2, the construction process of the material similarity model may include:

and S31, obtaining first material sample information of the first material sample.

The first material sample information comprises material main data of the material sample and business data of the material sample; the business data is data describing a material transaction process; the material samples include similar material samples and non-similar material samples.

Specifically, first material sample information of a large number of first material samples is obtained to be used as input data for building a material similarity model.

The material main data and the business data of the material sample are described above, please refer to the corresponding embodiments above.

In order to construct a neural network sample, the embodiment of the invention needs a positive sample and a negative sample, wherein the positive sample is a similar material sample, and the negative sample is a non-similar material sample.

And S22, determining the business scene of the material sample based on the business data of the material sample.

The business scenario is used for representing the transaction type of the transaction performed by the material, and can include purchasing, selling, producing, paying and the like, and the business scenario of the material sample can be determined through voice analysis according to the content of the business data of the material sample. Specifically, if the business data includes purchase units, purchase data, and the like, this may be determined as purchase, and if the business data includes price, sales amount, and the like, this may be determined as sales.

And S23, learning the material main data and the service scene respectively corresponding to the similar material samples and the dissimilar material samples by a machine based on the material main data and the service scene of the material samples to obtain a material similarity model.

In the embodiment, a conventional machine learning algorithm is adopted for machine learning, so that a material similarity model can be obtained.

On the basis of this embodiment, step S12 "inputting the material information into the data analysis model to obtain the information analysis result corresponding to the preset analysis dimension" may include:

1) inputting material information into a material similarity model to obtain the similarity between each material except the material to be analyzed and the material to be analyzed, and determining the material with the similarity larger than a preset similarity threshold as a target material;

2) acquiring material information corresponding to a target material;

3) and taking the similarity between the target material and the material to be analyzed and material information corresponding to the target material as an information analysis result under the corresponding preset analysis dimensionality.

Further, the material information comprises material main data of the material to be analyzed and business data of the material to be analyzed; the business data is data describing a material transaction process;

after the material information of the material to be analyzed and the data analysis model for analyzing the material information under the preset analysis dimensionality are obtained, the method further comprises the following steps:

correspondingly, inputting the material information into the material similarity model to obtain the similarity between each material except the material to be analyzed and the material to be analyzed, and determining the material with the similarity larger than a preset similarity threshold as a target material, wherein the method comprises the following steps:

inputting the material information into a material similarity model, and obtaining similarity corresponding to material main data and business scenes between each material except the material to be analyzed and the material to be analyzed through semantic analysis;

and screening out the materials with the similarity of the main material data and the business scene respectively greater than a set specified threshold value as target materials.

The material similarity model can directly obtain the similarity between each material except the material to be analyzed and the material to be analyzed, and a similarity threshold value, namely a designated threshold value such as 90, is set here. The similarity is more than 90, namely the similar materials are considered as target materials of the materials to be analyzed.

And then extracting the material information of the target material, taking the similarity between the target material and the material to be analyzed and the material information corresponding to the target material as information analysis results under similar material dimensions (namely preset material dimensions), and outputting the information analysis results.

It should be noted that, in practical application, the material similarity model needs to be periodically run, a material similarity report list is generated, and the similarity between materials is marked.

Now, for example, similar materials are illustrated, the target material obtained by screening may have the same business data (the same type of operator, business document, and business with similar business volume) under the same business scenario as the material to be analyzed, the same output (the same input, the same output, the same process route, and the same type of work order) and the same storage (the same type of stock, the same type of shelf, and the same location), the same sales (the same group or the same group of salespersons, the same type of customers, the same business of sales channel and distribution group), and the like, that is, the information of the target material obtained by screening is the same as or similar to the material description of the material to be analyzed, the material applicant, the material unit, the specification, the model, the size, and the like.

If the same department applies for the materials of the same material type in the same business scene, the system needs to remind the user whether the materials are the repeated applications.

The different departments apply for the materials, but if the application departments are similar to the material types, the user is also reminded whether to apply for the materials repeatedly.

The existing business data is checked, different materials are used under the same business scene, and the materials are similar in description, so that a user needs to be reminded whether the materials are repeated or not.

It should be noted that when similar materials are obtained by using the material similarity model, it is necessary to check the invalid factors (such as blank space, special symbol, case, and the like) in the material description, and if the invalid factors are met, it is necessary to analyze whether different material codes belong to the same material in combination with specific contents in the material description, and alarm. If one material is described as a blue-steel plate and the other material is described as a blue-steel plate, the two materials belong to the same material, and if one material is described as a blue-steel plate and the other material is described as a black-steel plate, the two materials do not belong to the same material.

In this embodiment, a method for determining similar materials is provided, and then materials similar to the target material can be determined according to the method in this embodiment.

2. The data analysis model includes a data integrity evaluation model.

The data integrity evaluation model stores the corresponding relation between various materials and the data required for describing the materials, and the corresponding relation between the data required for describing the various materials and the data format required by each data. Namely, the data integrity evaluation model specifies the data required for describing each material and the data format adopted by each data.

Optionally, on the basis of this embodiment, the material information is input into the data analysis model, and an information analysis result corresponding to the preset analysis dimension is obtained, including:

inputting material information into a data integrity evaluation model to determine whether data in the material information is complete or not based on the corresponding relationship between various materials and data required for describing the materials, determining missing data when the data is incomplete, determining whether the data format of each data in the material information is correct or not based on the corresponding relationship between the data required for describing various materials and the data format required by each data, and determining the data with the incorrect data format when the data format is incorrect;

and taking the analysis result of whether the data in the material information is complete, the analysis result of whether the data format of each data in the material information is correct, and the determined missing data and the data with the incorrect data format as the information analysis result under the corresponding preset analysis dimensionality when the data is incomplete or the data format is incorrect.

In the embodiment, the integrity of the material information is evaluated, a data integrity evaluation rule of the material information in the data integrity evaluation model needs to be preset, and the data integrity evaluation rule is used for indicating that the material data required in a material application scene is specified, for example, which material data are required by the specification, each material data is in a character form, and the like.

For example, the data required for the material may include:

material codes (definable unrepeatable codes), basic attributes that can carry the material (e.g., material type, units, length, width, height, weight, etc.), purchase attributes (e.g., purchase units, purchase organization), value attributes (cost price), sales attributes (sales organization), production attributes (BOM information), etc.

Taking the material as iron as an example, the data required by the material can include material code, type, unit, price, cost and the like. The format of the material code may be a number and the format of the price may be a reserved integer.

Data required by different materials can be associated through material codes, and a material information integrity logic model diagram is established so as to be convenient for reference in the use process of an enterprise. And the material information integrity logic model graph represents the incidence relation among different materials. For example, the boat is constructed of steel plates, bearings, screws, and the like.

Comparing a preset data integrity evaluation rule with material information obtained in an actually acquired material service scene to determine whether the actually obtained material information is consistent with the data integrity evaluation rule, wherein the consistency comprises the following steps: the types of the material data and the display forms (characters) of the material data are the same, and if the types of the material data and the display forms (characters) of the material data are the same, the material data are complete.

This embodiment gives a formula for calculating the integrity of the master data (the amount of data satisfying the integrity definition/the number of master data satisfying the condition) × 100%.

If the main data of the material information is incomplete, comparing the existing material information with the necessary material information to determine missing data, then determining whether the data format of each data in the material information is consistent with the corresponding data format, and if not, screening out the data with the incorrect data format.

In the embodiment, the accuracy of the material information of one material can be inquired by a user through the established data integrity determination rule, the accuracy of the material main data maintenance field is checked, the result can be displayed graphically, and alarm information can be output when the result is inaccurate.

In the above description, the generation process of the data integrity evaluation model is now described, and specifically, referring to fig. 3, the generation process of the data integrity evaluation model may include:

and S31, acquiring second material sample information of the second material sample.

The second material sample information comprises sample data required by different materials under different service scenes and the format of each sample data; the business scenario is used to characterize the transaction type of the transaction conducted on the material.

The business scenario may include a purchase scenario, a sales scenario, a production scenario, etc., and the data required by different business scenarios is different. The second material sample information comprises sample data required by different materials under different service scenes and the format of each sample data.

And S32, determining the corresponding relation between various materials and the data required for describing the materials according to the sample data required by the different materials in different service scenes.

And S33, determining the corresponding relation between the data required by describing various materials and the data format required by each data according to the sample data required by different materials in different service scenes and the format of each sample data.

Specifically, based on sample data required by different materials in different service scenes and the format of each sample data, machine learning is performed to obtain the corresponding relationship between each type of material and the data required for describing the material, and the corresponding relationship between the data required for describing each type of material and the data format required for each data.

It should be noted that after the second material sample information is obtained, part of the sample information may be used for machine learning data, and the other part of the sample information is used for verifying whether the generated data integrity evaluation model is accurate, and if not, the data integrity evaluation model is continuously adjusted.

When the data integrity evaluation model is accurate, the material information of the target material can be input into the data integrity evaluation model, and whether the material information is complete or not can be obtained.

3. The data analysis model comprises a material information effectiveness model.

The material information validity model is used for calculating the time difference between the last inquired time of the material information and the current time, and determining whether the material information is valid or not based on the size relation between the time difference and a preset time threshold.

Specifically, the validity rules are formulated, monitored and displayed according to different stages (the stages can be defined by themselves) of 1-3-5 years. If the unused material information is found to be invalid for more than 5 years (which can be realized by setting a threshold value), an alarm is given.

Correspondingly, inputting the material information into the data analysis model to obtain an information analysis result corresponding to the preset analysis dimension, which may include:

1) inputting the material information into a data analysis model to obtain the relationship between the time difference between the last inquired time and the current time of the material information and a preset time threshold;

2) judging whether the materials are effective or not based on the size relationship;

3) and taking the result of whether the material is effective as an information analysis result under the corresponding preset analysis dimensionality.

Specifically, the unused material information with the material information validity rule of more than 5 years is set as invalid, the last inquired time and the current time of the material information of the target material are obtained, the time difference is calculated, and if the time difference is more than 5 years, the material information is considered invalid.

The effectiveness of the materials can be set for each material, the effectiveness time of the materials can be manually modified, the materials are screened according to the effectiveness of the materials, effective materials and ineffective materials can be obtained, and graphical display is carried out respectively.

In addition, the material effectiveness can be respectively statistically displayed according to the use state of each organization, material grouping, material types and the like. The organization may be different units of the same group, different departments of the same unit, etc. The material groups can be raw materials, finished products, spare parts and the like, or computer accessories, office supplies, printed matters and the like. The material types can be raw materials, semi-finished products, services and the like.

The main data effective value (the effective material information amount satisfying the condition/the total material information amount satisfying the condition) is 100%.

The materials can also be sorted according to different rules, for example, the materials can also be sorted according to the use frequency.

In this embodiment, validity analysis can be performed on the material information, and then whether the material information is valid or not can be determined.

4. The data analysis model comprises a material information timeliness model.

The material information timeliness model is used for determining the current business progress of the material to be analyzed according to the business data of the material to be analyzed, and determining whether the current business progress is in the uncleaned business or not so as to determine whether the business corresponding to the material to be analyzed is the uncleaned business or not.

And the material information timeliness rule in the material information timeliness model is used for determining whether the business corresponding to the material to be analyzed is unclean business or not according to the business data.

Wherein, the service comprises: purchase application, purchase order, existing inventory; planning independent demands, planning orders and producing orders; sales orders, invoices, and the like.

And (4) judging the uncleaned service:

a. a purchase voucher has been created but not put in storage;

b. the invoice is not checked after being put in storage;

c. checking if the ticket is issued but not paid;

d. sales vouchers have appeared but not shipped;

e. shipped but not invoiced;

f. billed but not billed.

Correspondingly, the material information is input into the data analysis model, and an information analysis result corresponding to the preset analysis dimension is obtained, wherein the information analysis result comprises the following steps:

inputting material business data into a material information timeliness model to obtain the current business progress of a material to be analyzed and an analysis result of whether a business corresponding to the material to be analyzed is an unclean business;

and taking the current business progress of the material to be analyzed and the analysis result of whether the business corresponding to the material to be analyzed is the uncleaned business or not as the information analysis result under the corresponding preset analysis dimension.

Specifically, material business data are input into a material information timeliness model, the current business progress of the material to be analyzed is obtained, and whether the current business progress meets the a-f is determined according to the current business progress. And if so, outputting the current business progress of the material to be analyzed and an analysis result of whether the material is an uncleaned business. When the data is output, the data can be displayed in a chart form, and the authority is set for different users to inquire.

In the embodiment, a material information timeliness rule is given, and whether the material has uncleaned business can be determined.

The above introduces the analysis of different attributes of the material to be analyzed when the data analysis models are different, such as five parts of effectiveness, completeness, accuracy, uniqueness and timeliness of the material are analyzed, and then the quality condition of the material information can be monitored in real time.

Optionally, on the basis of the embodiment of the data processing method, another embodiment of the present invention provides a data processing apparatus, and with reference to fig. 4, the data processing apparatus may include:

the information acquisition module 101 is used for acquiring material information of a material to be analyzed and a data analysis model for analyzing the material information under a preset analysis dimension; the data analysis models are constructed in advance, and different preset analysis dimensions correspond to different data analysis models;

the information analysis module 102 is configured to input the material information into the data analysis model, and obtain an information analysis result corresponding to a preset analysis dimension.

It should be noted that, for the working process of each module in this embodiment, please refer to the corresponding description in the above embodiments, which is not described herein again.

Optionally, on the basis of the above embodiment of the data processing apparatus, the data analysis model includes a material similarity model; the material similarity model is used for determining the similarity between each material except the material to be analyzed and the material to be analyzed in the database in which the material information of the material to be analyzed is stored;

correspondingly, the information analysis module 102 is configured to input the material information into the data analysis model, and when an information analysis result corresponding to a preset analysis dimension is obtained, specifically configured to:

inputting material information into a material similarity model to obtain the similarity between each material except the material to be analyzed and the material to be analyzed, and determining the material with the similarity larger than a preset similarity threshold as a target material;

acquiring material information corresponding to a target material;

and taking the similarity between the target material and the material to be analyzed and material information corresponding to the target material as an information analysis result under the corresponding preset analysis dimensionality.

Further, the system further comprises a similarity model building module, and the similarity model building module specifically may include:

the first information acquisition submodule is used for acquiring first material sample information of the first material sample; the first material sample information comprises material main data of the material sample and business data of the material sample; the business data is data describing a material transaction process; the material samples comprise similar material samples and non-similar material samples;

the scene determining submodule is used for determining the business scene of the material sample based on the business data of the material sample; the business scene is used for representing the transaction type of the transaction performed by the material;

and the data training submodule is used for learning the material main data and the service scene respectively corresponding to the similar material sample and the dissimilar material sample through a machine based on the material main data and the service scene of the material sample to obtain a material similarity model.

further, the data processing apparatus further includes:

the scene determining module is used for determining the business scene of the material to be analyzed according to the business data of the material to be analyzed; the business scene is used for representing the transaction type of the transaction performed by the material;

correspondingly, the information analysis module 102 is configured to input the material information into the material similarity model, obtain the similarity between each material except for the material to be analyzed and the material to be analyzed, and when determining the material with the similarity greater than the preset similarity threshold as the target material, specifically:

It should be noted that, for the working processes of each module and sub-module in this embodiment, please refer to the corresponding description in the above embodiments, which is not described herein again.

Optionally, on the basis of the above embodiment of the data processing apparatus, the data analysis model includes a data integrity evaluation model; the data integrity evaluation model stores the corresponding relation between various materials and the data required for describing the materials and the corresponding relation between the data required for describing the various materials and the data format required by each data;

Further, the system further comprises an integrity evaluation building module, and specifically, the integrity evaluation model building module may include:

the second information acquisition submodule is used for acquiring second material sample information of the second material sample; the second material sample information comprises sample data required by different materials under different service scenes and the format of each sample data; the business scene is used for representing the transaction type of the transaction performed by the material;

the first relation determining submodule is used for determining the corresponding relation between various materials and data required for describing the materials according to sample data required by different materials under different service scenes;

and the second relation determining submodule is used for determining the corresponding relation between the data required by describing various materials and the data format required by each data according to the sample data required by different materials under different service scenes and the format of each sample data.

In this embodiment, the material information of the target material may be input into the data integrity evaluation model, so as to determine whether the material information is complete.

Optionally, on the basis of the embodiment of the data processing apparatus, the data analysis model includes a material information validity model; the material information validity model is used for calculating the time difference between the last inquired time of the material information and the current time, and determining whether the material information is valid or not based on the size relation between the time difference and a preset time threshold;

further, the information analysis module 102 is configured to input the material information into the data analysis model, and when an information analysis result corresponding to a preset analysis dimension is obtained, specifically configured to:

inputting the material information into a data analysis model to obtain the relationship between the time difference between the last inquired time and the current time of the material information and a preset time threshold;

judging whether the materials are effective or not based on the size relationship;

Optionally, on the basis of the embodiment of the data processing apparatus, the data analysis model includes a material information timeliness model; the material information timeliness model is used for determining the current business progress of the material to be analyzed according to the business data of the material to be analyzed, and determining whether the current business progress is in the uncleaned business or not so as to determine whether the business corresponding to the material to be analyzed is the uncleaned business or not;

further, the information analysis module is used for inputting the material information into the data analysis model, and when an information analysis result corresponding to a preset analysis dimension is obtained, the information analysis module is specifically used for:

The data processing device comprises a processor and a memory, wherein the information acquisition module, the information analysis module and the like are stored in the memory as program units, and the processor executes the program units stored in the memory to realize corresponding functions.

The processor comprises a kernel, and the kernel calls the corresponding program unit from the memory. The kernel can be set to be one or more than one, and the function of analyzing and managing the material main data of the material is realized by adjusting the kernel parameters.

The memory may include volatile memory in a computer readable medium, Random Access Memory (RAM) and/or nonvolatile memory such as Read Only Memory (ROM) or flash memory (flash RAM), and the memory includes at least one memory chip.

An embodiment of the present invention provides a storage medium on which a program is stored, the program implementing a data processing method when executed by a processor.

The embodiment of the invention provides a processor, which is used for running a program, wherein the program runs to execute a data processing method.

The embodiment of the invention provides equipment, which comprises a processor, a memory and a program which is stored on the memory and can run on the processor, wherein the processor executes the program and realizes the following steps:

a method of data processing, comprising:

and inputting the material information into a data analysis model to obtain an information analysis result corresponding to a preset analysis dimension.

Further, the data analysis model comprises a material similarity model; the material similarity model is used for determining the similarity between each material except the material to be analyzed and the material to be analyzed in the database in which the material information of the material to be analyzed is stored;

acquiring material information corresponding to a target material;

Further, the construction process of the material similarity model comprises the following steps:

determining a service scene of the material sample based on the service data of the material sample; the business scene is used for representing the transaction type of the transaction performed by the material;

and based on the material main data and the service scene of the material sample, respectively learning the material main data and the service scene corresponding to the similar material sample and the dissimilar material sample by a machine to obtain a material similarity model.

Further, the data analysis model comprises a data integrity evaluation model; the data integrity evaluation model stores the corresponding relation between various materials and the data required for describing the materials and the corresponding relation between the data required for describing the various materials and the data format required by each data;

Further, the generation process of the data integrity evaluation model comprises the following steps:

determining the corresponding relation between various materials and data required for describing the materials according to sample data required by different materials in different service scenes;

and determining the corresponding relation between the data required by describing various materials and the data format required by each data according to the sample data required by different materials in different service scenes and the format of each sample data.

Further, the data analysis model comprises a material information validity model; the material information validity model is used for calculating the time difference between the last inquired time of the material information and the current time, and determining whether the material information is valid or not based on the size relation between the time difference and a preset time threshold;

inputting the material information into a data analysis model to obtain an information analysis result under a corresponding preset analysis dimension, wherein the information analysis result comprises the following steps:

Further, the data analysis model comprises a material information timeliness model; the material information timeliness model is used for determining the current business progress of the material to be analyzed according to the business data of the material to be analyzed, and determining whether the current business progress is in the uncleaned business or not so as to determine whether the business corresponding to the material to be analyzed is the uncleaned business or not;

The device herein may be a server, a PC, a PAD, a mobile phone, etc.

The present application further provides a computer program product adapted to perform a program for initializing the following method steps when executed on a data processing device:

a method of data processing, comprising:

acquiring material information corresponding to a target material;

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.

Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in the process, method, article, or apparatus that comprises the element.

The above are merely examples of the present application and are not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A data processing method, comprising:

2. The data processing method of claim 1, wherein the data analysis model comprises a material similarity model; the material similarity model is used for determining the similarity between each material except the material to be analyzed and the material to be analyzed in a database in which the material information of the material to be analyzed is stored;

acquiring material information corresponding to the target material;

3. The data processing method of claim 2, wherein the building process of the material similarity model comprises:

4. The data processing method according to claim 2, wherein the material information includes material main data of the material to be analyzed and business data of the material to be analyzed; the business data is data describing a material transaction process;

5. The data processing method of claim 1, wherein the data analysis model comprises a data integrity evaluation model; the data integrity evaluation model stores the corresponding relation between various materials and the data required for describing the materials and the corresponding relation between the data required for describing the various materials and the data format required by each data;

6. The data processing method of claim 5, wherein the generating of the data integrity evaluation model comprises:

7. The data processing method of claim 1, wherein the data analysis model comprises a material information validity model; the material information validity model is used for calculating the time difference between the last inquired time of the material information and the current time, and determining whether the material information is valid or not based on the size relation between the time difference and a preset time threshold;

judging whether the material is effective or not based on the size relation;

8. The data processing method of claim 3, wherein the data analysis model comprises a material information timeliness model; the material information timeliness model is used for determining the current business progress of the material to be analyzed according to the business data of the material to be analyzed, and determining whether the current business progress is in uncleaned business or not so as to determine whether the business corresponding to the material to be analyzed is uncleaned business or not;

9. A data processing apparatus, comprising:

10. A processor, characterized in that the processor is configured to run a program, wherein the program is configured to execute the data processing method according to any one of claims 1 to 8 when running.