CN117785918A - Data processing method, device, storage medium, program product and electronic equipment - Google Patents

Data processing method, device, storage medium, program product and electronic equipment Download PDF

Info

Publication number
CN117785918A
CN117785918A CN202311689717.7A CN202311689717A CN117785918A CN 117785918 A CN117785918 A CN 117785918A CN 202311689717 A CN202311689717 A CN 202311689717A CN 117785918 A CN117785918 A CN 117785918A
Authority
CN
China
Prior art keywords
data
calculation
analysis
processing
preset
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311689717.7A
Other languages
Chinese (zh)
Inventor
马春亮
郝聪
武晓达
靳晓敏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Great Wall Motor Co Ltd
Original Assignee
Great Wall Motor Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Great Wall Motor Co Ltd filed Critical Great Wall Motor Co Ltd
Priority to CN202311689717.7A priority Critical patent/CN117785918A/en
Publication of CN117785918A publication Critical patent/CN117785918A/en
Pending legal-status Critical Current

Links

Landscapes

  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The embodiment of the application discloses a data processing method, a device, a storage medium, a program product and electronic equipment, which are applied to the technical field of computers, and the method comprises the following steps: acquiring initial data respectively corresponding to a plurality of data systems; carrying out structuring treatment on each initial data according to a preset label system to obtain at least one structuring data corresponding to each data system; and packaging the preset at least one data calculation rule, the at least one problem analysis rule and the plurality of structured data to obtain packaging data for providing data calculation service and problem analysis service. The method can solve the problem of data islanding among a plurality of data systems, and provide data computing services and problem analysis services for users according to data from the plurality of data systems.

Description

Data processing method, device, storage medium, program product and electronic equipment
Technical Field
The present application relates to the field of computers, and more particularly, to a data processing method, apparatus, storage medium, program product, and electronic device in the field of computers.
Background
Data is a representation of facts, concepts, or instructions that may be processed by manual or automated means. After the data is interpreted and given a certain meaning, the data becomes information. Data processing (data processing) is the collection, storage, retrieval, processing, transformation, and transmission of data. The basic purpose of data processing is to extract and derive data that is valuable and meaningful to some particular person from a large, unorganized, unintelligible amount of data. Data processing is a fundamental link of system engineering and automatic control. Data processing extends throughout various areas of social production and social life.
Taking data related to automobiles as an example, the amount of test data accumulated in the whole automobile test process is huge, and the test data are scattered on a personal computer or a cloud disk in the forms of documents, pictures and the like. The unstructured storage mode has the problems of non-uniform data standard, non-pull-through of test data of different brands of vehicle types, poor sharing property, low massive data query or retrieval efficiency, poor safety, incapability of effectively exerting data value and the like. How to realize structuring and standardization of test data generated by huge manpower, material resources and financial resources and fully exert the value of the test data becomes a problem to be solved urgently.
Disclosure of Invention
The embodiment of the application provides a data processing method, a device, a storage medium, a program product and electronic equipment, wherein the method can solve the problem of data islanding among a plurality of data systems and provide data computing service and problem analysis service for users according to data from the plurality of data systems. The technical scheme is as follows:
in a first aspect, an embodiment of the present application provides a data processing method, including:
acquiring initial data respectively corresponding to a plurality of data systems;
carrying out structuring treatment on each initial data according to a preset label system to obtain at least one structuring data corresponding to each data system;
and carrying out packaging processing on at least one preset data calculation rule, at least one problem analysis rule and a plurality of structured data to obtain packaging data for providing data calculation service and problem analysis service.
Through the technical scheme, the initial data respectively corresponding to the plurality of data systems are structured through the preset tag system, so that clean, accurate and complete data assets are accumulated, and the efficiency of inquiring and searching in mass data by a user is improved; further, in the specification, the plurality of structured data obtained after the structuring treatment, at least one preset data calculation rule and at least one problem analysis rule are packaged, and data calculation service and problem analysis service are provided for a user according to the packaged data, so that the problem of data island among a plurality of data systems is eliminated, the whole business flow from data to calculation to analysis is pulled through, and the data value is fully mined through multi-dimensional problem analysis layer by layer.
With reference to the first aspect, in some possible implementations, after the structuring processing is performed on each initial data according to a preset tag system to obtain at least one structured data corresponding to each data system, the method includes:
classifying the plurality of structured data according to a plurality of preset topics to obtain at least one structured data corresponding to each topic.
Through the technical scheme, the structured data can be better arranged, and a data basis is provided for subsequent data calculation and problem analysis.
With reference to the first aspect, in some possible implementations, the structuring processing is performed on each initial data according to a preset tag system to obtain at least one structured data corresponding to each data system, where the structuring processing includes:
according to the attribute information of a target tag in a plurality of tags included in a preset tag system, determining structural data matched with the attribute information of the target tag in the initial data, and obtaining at least one piece of structural data included in the initial data.
Through the technical scheme, each initial data can be subjected to structuring treatment according to the preset label system, and the efficiency and accuracy of structuring treatment are improved.
With reference to the first aspect, in some possible implementations, the method further includes:
and carrying out association processing on each structured data and the target label corresponding to the structured data to obtain association data for query service.
Through the technical scheme, the user can inquire the corresponding structured data through the target label according to the association relation between the structured data and the target label, so that the efficiency of inquiring the user data is improved.
With reference to the first aspect, in some possible implementations, the method further includes:
acquiring a label setting instruction;
in response to the tag setting instruction, a tag system including a plurality of tags is set.
Through the technical scheme, the tag system comprising a plurality of tags can be set according to the tag setting instruction, the tag system is used for carrying out structuring processing on a plurality of initial data, the setting mode of the tag system is flexible, and the user-defined requirement of a user on the tags is met.
With reference to the first aspect, in some possible implementations, the method further includes:
receiving a calculation instruction and/or an analysis instruction sent by electronic equipment;
calling the packaging data to perform data calculation corresponding to the first calculation instruction in response to the calculation instruction, and obtaining the calculation result, and/or calling the packaging data to perform problem analysis corresponding to the analysis instruction in response to the analysis instruction, and obtaining the analysis result;
And sending the calculation result and/or the analysis result to the electronic equipment.
Through the technical scheme, the user can send the calculation instruction and/or the analysis instruction according to the actual calculation or analysis requirement, and check the calculation result and/or the analysis result, so that the complex personalized requirement of the user is met.
In a second aspect, embodiments of the present application provide a data processing apparatus, the apparatus including:
the initial acquisition module is used for acquiring initial data respectively corresponding to the plurality of data systems;
the structure processing module is used for carrying out structuring processing on the initial data according to a preset label system to obtain at least one structuring data corresponding to each data system;
and the packaging processing module is used for packaging at least one preset data calculation rule, at least one problem analysis rule and a plurality of structured data to obtain packaging data for providing data calculation service and problem analysis service.
In a third aspect, embodiments of the present application provide a computer storage medium storing a plurality of instructions adapted to be loaded by a processor and to perform the above-described method steps.
In a fourth aspect, embodiments of the present application provide a computer program product storing a plurality of instructions adapted to be loaded by a processor and to perform the above-described method steps.
In a fifth aspect, embodiments of the present application provide an electronic device, which may include: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to perform the above-mentioned method steps.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of a data processing method according to an embodiment of the present disclosure;
FIG. 2 is a schematic flow chart of a data processing method according to an embodiment of the present application;
FIG. 3 is a schematic flow chart of a data processing method according to an embodiment of the present application;
FIG. 4 is a schematic flow chart of a data processing method according to an embodiment of the present application;
FIG. 5 is a schematic flow chart of a data processing method according to an embodiment of the present application;
FIG. 6 is a schematic diagram of a data processing apparatus according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
In the description of the present application, it should be understood that the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. In the description of the present application, it is to be understood that the terms "comprise" and "have," and any variations thereof, are intended to cover non-exclusive inclusions, unless otherwise specifically defined and defined. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those listed steps or elements but may include other steps or elements not listed or inherent to such process, method, article, or apparatus. The specific meaning of the terms in this application will be understood by those of ordinary skill in the art in a specific context. Furthermore, in the description of the present application, unless otherwise indicated, "a plurality" means two or more. "and/or", describes an association relationship of an association object, and indicates that there may be three relationships, for example, a and/or B, and may indicate: a exists alone, A and B exist together, and B exists alone. The character "/" generally indicates that the context-dependent object is an "or" relationship.
The present application is described in detail with reference to specific examples.
It should be noted that, information (including but not limited to user equipment information, user personal information, etc.), data (including but not limited to data for analysis, stored data, presented data, etc.), and signals according to the embodiments of the present disclosure are all authorized by the user or are fully authorized by the parties, and the collection, use, and processing of relevant data is required to comply with relevant laws and regulations and standards of relevant countries and regions. For example, data, instructions, and the like referred to in this specification are all acquired with sufficient authorization.
Data is a representation of facts, concepts, or instructions that may be processed by manual or automated means. After the data is interpreted and given a certain meaning, the data becomes information. Data processing (data processing) is the collection, storage, retrieval, processing, transformation, and transmission of data. The basic purpose of data processing is to extract and derive data that is valuable and meaningful to some particular person from a large, unorganized, unintelligible amount of data. Data processing is a fundamental link of system engineering and automatic control. Data processing extends throughout various areas of social production and social life.
Referring to fig. 1, fig. 1 is a schematic diagram of a data processing method according to an embodiment of the present application. In the present embodiment, the data processing server 101, a plurality of data systems for storing initial data, and a plurality of electronic devices for transmitting information indicating that the data processing server 101 provides a problem analysis service and a data calculation service are included. Wherein the plurality of electronic devices includes at least electronic device 1021, electronic device 1022, and electronic device 1023, and the plurality of data systems includes at least data system 1031, data system 1032, and data system 1033. It will be appreciated that the number of data processing servers 101, the plurality of electronic devices, and the plurality of data systems shown in fig. 1 is merely illustrative, and the embodiments of the present application are not limited in this regard.
The data processing server 101 may be understood as a server or a cluster of multiple servers, where the data processing server 101 receives requests or information through multiple interfaces configured to provide corresponding data or services based on the requested content of the requests. In this embodiment of the present application, the data processing server 101 is configured to obtain initial data stored in a plurality of data systems, where the initial data stored in each data system is not identical. The data processing server 101 is further configured to receive analysis instructions and/or calculation instructions sent from a plurality of electronic devices, send a problem analysis result corresponding to the analysis instructions to an electronic device corresponding to the analysis instructions, and send a data calculation result corresponding to the calculation instructions to an electronic device corresponding to the calculation instructions.
In this embodiment, the data processing server includes an OLTP (On-Line Transaction Processing, online transaction) application and an OLAP (On-Line Analytical Processing, online analysis processing) application capable of supporting different traffic scenarios, and supporting calculation of ultra-large-scale data volume to satisfy different demands of different scenarios.
It will be appreciated that the data processing server 101 also has other service capabilities and functions to accomplish the tasks of the embodiments described below. For example, the data processing server 101 also provides portal services, resource management services, CI/CD services, and the like.
The data processing server 101 may be a plurality of physical servers, and the plurality of physical servers are independent in hardware; or multiple servers are in multiple virtual servers, the multiple virtual servers are deployed in the same hardware resource pool, and the deployment modes of the virtual servers include but are not limited to: VMware, virtual Box, and Virtual PC.
The data processing server 101 may communicate with a plurality of electronic devices, a plurality of data systems via communication links established based on a communication protocol, for example: gRPC Protocol, gRPC is a high-performance, general open-source remote server call (Remote Procedure Call, RPC) framework, which is mainly developed for mobile applications and designed based on HTTP/2 Protocol standards, is developed based on Protocol Buffers (PB) serialization protocols, and supports numerous development languages. In addition, the communication link may also be a wireless communication link or a wired communication link, such as: the wired communication link may include an optical fiber, twisted pair or coaxial cable, and the WIreless communication link may include a Bluetooth communication link, a WIreless-FIdelity (Wi-Fi) communication link, a microwave communication link, or the like.
The user communicates with the data processing server 101 via the electronic device, i.e. sends calculation instructions or analysis instructions to the data processing server 101 via the electronic device. For example, the user transmits a calculation instruction to the data processing server 101 through the electronic device 1021, transmits an analysis instruction to the data processing server 101 through the electronic device 1022, or transmits a calculation instruction and an analysis instruction to the data processing server 101 through the electronic device 1023. The content of the analysis instructions or calculation instructions may be any content in any format that is set by the user via the electronic device. Electronic devices including, but not limited to, physical or virtual servers, mobile Stations (MSs), mobile Terminal devices (Mobile terminals), mobile phones (Mobile telephones), handsets (handsets), and portable devices (portable equipment), bluetooth headsets, smart watches, etc. may communicate with one or more core networks via a radio access network (Radio Access Network, RAN). It is understood that the embodiments of the present application are not limited to the types of electronic devices described above.
In the embodiment of the present specification, the electronic device may further be provided with a display device, and the display device may be various devices capable of realizing a display function, for example: the display device may be a cathode ray tube display (Cathode raytubedisplay, CR), a Light-emitting diode display (Light-emitting diodedisplay, LED), an electronic ink screen, a liquid crystal display (Liquid crystal display, LCD), a plasma display panel (Plasma displaypanel, PDP), or the like. The user can view the result of the analysis of the problem or the result of the calculation of the data transmitted from the data processing server 101 by using the display means on the electronic device. And the user may send instructions to the electronic device through the display device, for example, by performing long press or click, double click operations on the display device of the electronic device, including sending calculation instructions and/or analysis instructions to the data processing server 101 through the electronic device, and the like.
A large amount of initial data is stored on a plurality of data systems, and the data systems have the problem of data islands among each other. Taking data related to automobiles as an example, the amount of test data accumulated in the whole automobile test process is huge, and the test data are scattered on a personal computer or a cloud disk in the forms of documents, pictures and the like. The unstructured storage mode has the problems of non-uniform data standard, non-pull-through of test data of different brands of vehicle types, poor sharing property, low massive data query or retrieval efficiency, poor safety, incapability of effectively exerting data value and the like. How to realize structuring and standardization of test data generated by huge manpower, material resources and financial resources and fully exert the value of the test data becomes a problem to be solved urgently.
Therefore, the embodiment of the application proposes a data processing method to solve the above-mentioned problems. In one embodiment, shown in FIG. 2, a schematic diagram of a data processing method according to an embodiment of the present application, which may be implemented in dependence on a computer program, may be run on a data processing apparatus based on the von Neumann system. The computer program may be integrated in the application or may run as a stand-alone tool class application.
Specifically, the data processing method includes:
s101, initial data respectively corresponding to a plurality of data systems is acquired.
Initial data is stored on each data system, and the initial data stored on different data systems may be the same or different or not the same. The data form of the initial data may be any form of image, text, video, audio, etc. The plurality of data systems may include some or all of the following data systems: applications and product systems (Systemsapplications and products in data processing, SAP), attendance systems, enterprise management systems (Enterprise Application Suites, ESA), manufacturing enterprise process execution systems (ManufacturingExecutionSystem, MES), manufacturing view systems (General Bill of Material, G-BOM), comprehensive maintenance systems (Tivoli Storage Manager, TSM), test data analysis systems, and the like.
In the following description, a data system and data related to a vehicle are taken as examples, and it is understood that the embodiments of the present disclosure further include initial data in different fields of different services in other scenarios, and a data system corresponding to the initial data. The initial data included in the test data analysis system includes at least vehicle-related attribute data, which is data reflecting vehicle attribute characteristics and user attribute characteristics of a user to whom the vehicle belongs. The vehicle-associated attribute data may include vehicle attribute data and user attribute data of a user to whom the vehicle belongs. Alternatively, the vehicle attribute data may include a sales area, a usage type, a brand, a train, a model, a color, a style, a configuration, an intelligent network type, an engine type, a option id, an option name, a seat number, a belonging platform, and a purchase-dealer name of the vehicle. Optionally, the user attribute data may include the gender and age of the user, the type of customer to which the user corresponds, the region in which the user is located, and the brand, train and model of the vehicle in which the user is driving.
In one embodiment, the data processing server includes a data lake and a data center tool having a data acquisition, i.e., data acquisition, function capable of acquiring initial data for each data system. The data processing server acquires initial data corresponding to a plurality of asynchronous data systems in a T+1 data acquisition mode through the data center tool. Wherein 'T' represents the data acquisition day, 'T+1' represents the data acquisition day plus one day, and the acquisition of asynchronous data can be realized by a 'T+1' data acquisition mode. In addition, one of the Sqoop, kettle and flume can be used for replacing a data center tool, namely, service data corresponding to a plurality of target service systems can be acquired in a T+1 data acquisition mode through one of the Sqoop, kettle and flume. The asynchronously collected data systems may be G-BOM systems, TSM systems, and test data analysis systems.
In another embodiment, the initial data in each data system may also be acquired by means of real-time acquisition. Data systems suitable for real-time acquisition may be Technical Review (TR) and SystemWeave systems. In the application, different data acquisition modes are adopted aiming at different data systems, so that the acquisition efficiency of initial data can be improved.
After initial data corresponding to each data system are acquired, initial data corresponding to each data system are obtained, the initial data corresponding to each data system are sent to a data lake, and lake entering operation is carried out on the received initial data corresponding to each data system through the data lake, so that distributed storage of the initial data is realized.
In one embodiment, the initial data is preprocessed after each initial data is obtained, so as to obtain preprocessed initial data. The preprocessing process can comprise methods of data normalization, data outlier rejection, data completion, heterogeneous data fusion and the like. The initial data can also be preprocessed by various methods such as natural language processing, multi-source data fusion, spatial data processing and the like. By preprocessing the initial data, the data redundancy of a plurality of initial data can be reduced, and the efficiency of the subsequent structuring processing of the plurality of initial data can be improved.
S102, carrying out structuring processing on each initial data according to a preset label system to obtain at least one structuring data corresponding to each data system.
Data tags are a technical means of describing objects with concisely abbreviated data content. Essentially, a method for analyzing, counting and abstracting data is added on the basis of extracting long text into refined and effective information, and the method has the characteristics of short text, semanteme, repeatable marking and the like.
Data tags are typically a set of artificially defined, highly condensed features items that are used to characterize information hidden behind data by one or more tag applications, unlike attributes, which are business conclusions drawn based on several attributes of an entity, based on artificially given business rules, or business rules trained through machine learning techniques. Data tagging is a simple and effective method of improving the efficiency of incremental data clustering by assigning each newly added data point to the cluster most similar thereto.
The preset tag system includes a plurality of tags. The plurality of labels can have an association relationship or a hierarchical relationship, the association relationship characterizes the association among the plurality of labels, and the hierarchical relationship is used for representing that the plurality of labels have a superior-inferior structure.
For example, the label system in the test data analysis system comprises a plurality of labels with association relation, and specifically comprises labels such as test basic data, test result data, test plan data and the like.
For another example, a tag hierarchy in a data system includes a plurality of tags having a hierarchical relationship, specifically including a vehicle attribute and a user attribute that are first level tags, and a diode tag having a hierarchical relationship with the tags of the vehicle attribute: brand, train, model, color, style, and diode labels having a hierarchical relationship with the labels of the user attributes: the gender and age of the user, the type of customer, the region in which the user is located.
In one embodiment, before performing the structuring processing on each initial data according to the preset tag system to obtain at least one structured data corresponding to each data system, the method further includes: acquiring a label setting instruction; in response to the tag setting instruction, a tag system including a plurality of tags is set.
Specifically, the tag system including a plurality of tags is set by receiving a tag setting instruction of a user through an electronic device having a communication connection with the data processing server or through an input device of the data processing server, and by tag information carried in the tag setting instruction. The tag information is used for representing attribute information of each tag and hierarchical relations and association relations among a plurality of tags, the attribute information of each tag is used for indicating data which the tag should be associated with, and the tag information also comprises information such as names of the tags.
In this embodiment, a tag system including a plurality of tags may be set according to a tag setting instruction, where the tag system is configured to perform structural processing on a plurality of initial data, and a setting manner of the tag system is flexible, so as to satisfy a user-defined requirement of a user on the tags.
In one embodiment, the specific step of structuring the plurality of initialization data includes: according to the attribute information of a target tag in a plurality of tags included in a preset tag system, determining structural data matched with the attribute information of the target tag in the initial data, and obtaining at least one piece of structural data included in the initial data. In other words, each initial data comprises at least one structured data, or a plurality of structured data constitutes an initial data.
Specifically, firstly, according to attribute information corresponding to a plurality of labels in a preset label system, different initial data are structured through a certain algorithm design or algorithm model, at least one piece of characteristic data in the initial data is extracted to serve as structured data, and the structured data has characteristic information for quantifying object characteristics of the structured data, namely basic description of the object characteristics.
Further, based on the characteristic information of the plurality of structured data and the plurality of labels in the label system, the structured data corresponding to the target label in the plurality of labels is determined. For example, the plurality of labels comprise colors, a train and gender, and the plurality of structured data can be extracted by carrying out structuring treatment on one piece of initial data, and the structured data comprises yellow structured data, T-series structured data and female structured data; and matching the plurality of structured data with the plurality of labels based on the attribute information of each label, wherein the structured data corresponding to the target label with the color is yellow, the structured data corresponding to the target label with the train is T series, and the structured data corresponding to the target label with the gender of the user is female.
It should be noted that, in this embodiment, the structuring process of the initial data needs to be performed in different processing manners according to different object types and different data types.
Specifically, for initial data mainly including text, image, etc., text mining, machine learning, natural language processing, social network analysis, etc., may be used to perform keyword recognition, topic recognition, emotion feature recognition, etc. of the object, which may represent object properties or behavior features.
For the initial data of digital data, statistical data and other data types, various statistical analysis methods such as index system design, multiple linear regression, analytic hierarchy process and the like can be comprehensively adopted to carry out data statistical analysis, so that data depiction is carried out on object objects through the methods such as data clustering, classification and the like, and structured data with characteristic information is obtained.
In this embodiment, each initial data may be structured according to a preset tag system, so as to improve efficiency and accuracy of the structuring process.
In one embodiment, after performing the structuring process on each initial data according to the preset tag system to obtain at least one structured data corresponding to each data system, the method further includes: and carrying out association processing on each structured data and the target label corresponding to the structured data to obtain association data for inquiring the service. In other words, the association data includes a pair of relationships consisting of the target tag and at least one structured data corresponding to the target tag.
In this embodiment, the user may query the structured data corresponding to the target tag according to the association relationship between the structured data and the target tag, thereby improving the efficiency of user data query.
For example, a user sends a query instruction to a data processing server by clicking display controls corresponding to a plurality of target labels on a display device of an electronic device, for example, the name of the display control is color, and then the target label corresponding to the display control is color; the data processing server responds to the plurality of target labels corresponding to the query instruction, and obtains structured data corresponding to each target label through the association relation corresponding to each target label, for example, obtains structured data with yellow content corresponding to the target label, and sends the plurality of structured data to the electronic equipment, so that a user can check according to the display device of the electronic equipment.
S103, packaging at least one preset data calculation rule, at least one problem analysis rule and a plurality of structured data to obtain packaging data for providing data calculation service and problem analysis service.
The data calculation rule is used for performing data calculation on the plurality of structured data. In this embodiment, the data calculation rule includes at least one of the following calculation rules: injury value algorithm rules, CFC filter algorithm rules, frequency multiplication algorithm rules, and trend analysis rules. The data calculation rules may include, in addition to the general algorithm rules described above, calculation rules or calculation models that are set by a user in a user-defined manner, which is not limited in this application.
The problem analysis rule is used for analyzing the problems of the plurality of structured data or the calculation results of the plurality of structured data. In this embodiment, the at least one problem analysis rule includes at least one of the following calculation rules: test scheme analysis rules, problem diagnosis rules, test report rules, data comparison rules, and problem positioning rules. The test plan analysis rule is used for analyzing the test plan, the problem diagnosis rule is used for diagnosing problems of structural data such as the test plan, the test report, the test plan and the like, the data comparison rule is used for comparing at least two specified structural data, the test report rule is used for generating a test report based on a plurality of structural data, and the test report is used for representing a conclusion of a test process of a target object such as a vehicle.
And packaging the preset at least one data calculation rule, the at least one problem analysis rule and the plurality of structured data, wherein the obtained packaged data has the capability of providing data calculation service and problem analysis service. In other words, the data calculation result and the problem analysis result can be obtained by calling the package data through a preset interface. The manner of invoking the encapsulated data may be implemented in accordance with instructions or requests output by the user.
For example, by calling the package data in which at least one data calculation rule, at least one problem analysis rule and a plurality of structured data are packaged, functions such as data configuration of experimental form, accurate problem positioning, AE automatic inspection, test scheme management and control, test structured report, problem auxiliary diagnosis and the like can be realized, that is, the data processing server provides the services to the user through the package data and sends results corresponding to the services to the electronic devices.
In the method, initial data respectively corresponding to a plurality of data systems are structured through a preset tag system, so that clean, accurate and complete data assets are accumulated, and the efficiency of inquiring and searching in mass data by a user is improved; further, in the specification, the plurality of structured data obtained after the structuring treatment, at least one preset data calculation rule and at least one problem analysis rule are packaged, and data calculation service and problem analysis service are provided for a user according to the packaged data, so that the problem of data island among a plurality of data systems is eliminated, the whole business flow from data to calculation to analysis is pulled through, and the data value is fully mined through multi-dimensional problem analysis layer by layer.
In one embodiment, shown in FIG. 3, a schematic diagram of a data processing method according to an embodiment of the present application, which may be implemented in dependence on a computer program, may be run on a data processing apparatus based on the von Neumann system. The computer program may be integrated in the application or may run as a stand-alone tool class application.
Specifically, the data processing method includes:
s201, initial data respectively corresponding to a plurality of data systems is acquired.
See S101 above, and will not be described here again.
S202, carrying out structuring processing on each initial data according to a preset label system to obtain at least one structuring data corresponding to each data system.
See S102 above, and will not be described here again.
S203, classifying the plurality of structured data according to a plurality of preset topics to obtain at least one structured data corresponding to each topic.
A topic may be understood as a feature that characterizes a clustering or classifying of a plurality of structured data, each topic corresponding to at least one tag. In one embodiment, the preset plurality of topics includes at least one of the following topics: test requirements topic, test plan topic, test results topic, and test report topic.
The structured data corresponding to the test requirement theme is related to the requirement or target of the target object in the test process or the test process, for example, the maximum mileage required to be reached by the vehicle and the maximum endurance required to be reached by the vehicle. The structural data corresponding to the test plan theme is related to the overall scheme for testing the target object, the structural data corresponding to the test plan theme is related to the specific step scheme for testing the target object, the structural data corresponding to the test result theme is related to the test data of the target object after the test or the test is completed, and the structural data corresponding to the test report theme is related to the report of the target object after the test or the test is completed.
In this embodiment, the multiple structured data are classified according to preset multiple topics, so that multiple initial data can be better sorted and deposited, and a good data basis is provided for subsequent data calculation and problem analysis. For example, the labels are brands, train, models, colors, styles and structured data corresponding to each label can be classified into test report topics, and the labels are maximum load, highest speed of time and structured data corresponding to each label can be classified into test demand topics.
The method for obtaining the preset multiple topics may be to receive a topic setting instruction sent by an electronic device having a communication link with the data processing server, or receive a topic setting instruction input by a user through an input device of the data processing server, respond to the topic setting instruction, and set multiple topics according to attribute information corresponding to multiple topics carried by the topic setting instruction. The present application also includes other methods of obtaining multiple topics, which are not intended to be limiting in any way.
S204, packaging at least one preset data calculation rule, at least one problem analysis rule and a plurality of structured data to obtain packaging data for providing data calculation service and problem analysis service.
And packaging the preset at least one data calculation rule, the at least one problem analysis rule and the plurality of structured data, wherein the obtained packaged data has the capability of providing data calculation service and problem analysis service. In other words, the data calculation result and the problem analysis result can be obtained by calling the package data through a preset interface. The manner of invoking the encapsulated data may be implemented in accordance with instructions or requests output by the user.
Fig. 4 is a schematic flow chart of a data processing method according to an embodiment of the present application, as shown in fig. 4. The plurality of data systems included in the present embodiment at least includes: G-BOM system 2012, TSM system 201, and test data analysis system 2011, TR system 2014, and SystemWeave system 2015. Each data system includes corresponding initial data, each initial data is structured according to a preset tag system to obtain at least one structured data corresponding to each data system, specifically, the G-BOM system 2012 includes test owner data, the TSM system 2013 includes test order data, the test data analysis system 2011 includes test base data, test result data and test plan data, the TR system 2014 includes test target value comparison data, and the SystemWeave system 2015 includes test requirement data and test case data.
And carrying out structuring treatment on each initial data according to a preset label system, and carrying out classification treatment on a plurality of initial data according to a preset theme to obtain at least one structuring data corresponding to each data system. In this embodiment, at least a test requirement topic 2021, a test plan topic 2022, a test plan topic 2023, a test result topic 2024, and a test report topic 2025 are included. It is to be understood that the illustrated subject matter is merely an example, and that the present application also includes subject matter of other amounts of content.
The preset at least one data computing rule 2032, at least one problem analysis rule 2031, and a plurality of subject data sets are packaged to obtain packaged data 204 for providing a data computing service and a problem analysis service. In this embodiment, the data calculation rule includes at least one of the following calculation rules: the damage value algorithm rule, the CFC filtering algorithm rule, the frequency multiplication algorithm rule and the trend analysis rule, and the at least one problem analysis rule at least comprises one of the following calculation rules: test scheme analysis rules, problem diagnosis rules, test report rules, data comparison rules, and problem positioning rules.
In one embodiment, the plurality of data systems includes a review system, and the initial data corresponding to the plurality of data systems includes test requirement data and test case data included in the review system. As shown in fig. 4, the review system includes a TR system 2014 and a SystemWeave system 2015. And the evaluation system is used for comparing the actual test data according to the test requirement data and the test case data, specifically comparing the test target value and obtaining the test target value comparison data.
In this embodiment, after performing a packaging process on at least one preset data calculation rule, at least one problem analysis rule, and a plurality of structured data to obtain packaging data for providing a data calculation service and a problem analysis service, the method further includes: receiving a calculation instruction and/or an analysis instruction sent by a review system; the package data is called in response to the calculation instruction to calculate data corresponding to the calculation instruction, a calculation result is obtained, and/or the package data is called in response to the analysis instruction to analyze problems corresponding to the analysis instruction, and an analysis result is obtained; and sending the calculation result and/or the analysis result to the review system so that the review system can realize test plan management and control according to the calculation result and/or the analysis result, and the test requirement data and the test case data included in the review system.
The calculation instruction is used for indicating the data processing server to call the package data to perform data calculation corresponding to the calculation instruction and obtain a calculation result, and the analysis instruction is used for indicating the data processing server to call the package data to perform problem analysis corresponding to the analysis instruction and obtain an analysis result. For example, the data processing server calls the package data to perform trend analysis calculation based on the calculation instruction, a trend analysis result is obtained as a calculation result, the data processing server calls the package data to perform process data comparison based on the analysis instruction, and a comparison result is obtained as an analysis result.
As shown in fig. 4, initial data corresponding to a review system including the TR system 2014 and the SystemWeave system 2015 also enters a data lake, is packaged into packaged data, and simultaneously invokes the packaged data to use a data calculation service or a problem analysis service of the packaged data, or receives a data calculation result 2051 or a problem analysis result 2052.
In this embodiment, data calculation and problem analysis may be performed according to the test requirement data and the test case data of the review system, and the data calculation result and the problem analysis result may be sent to the review system, so that the review result may adjust the test scheme, and obtain the test requirement data and the test case data for testing the vehicle based on the adjusted test scheme, thereby improving the test effect of the test scheme and accelerating the process of adjusting the test scheme.
In the method, initial data respectively corresponding to a plurality of data systems are structured through a preset tag system, so that clean, accurate and complete data assets are accumulated, and the efficiency of inquiring and searching in mass data by a user is improved; further, in the specification, the plurality of structured data obtained after the structuring treatment, at least one preset data calculation rule and at least one problem analysis rule are packaged, and data calculation service and problem analysis service are provided for a user according to the packaged data, so that the problem of data island among a plurality of data systems is eliminated, the whole business flow from data to calculation to analysis is pulled through, and the data value is fully mined through multi-dimensional problem analysis layer by layer.
In one embodiment, shown in FIG. 5, a schematic diagram of a data processing method according to an embodiment of the present application, which may be implemented in dependence on a computer program, may be run on a data processing apparatus based on the von Neumann system. The computer program may be integrated in the application or may run as a stand-alone tool class application.
Specifically, the data processing method includes:
s301, initial data respectively corresponding to a plurality of data systems is acquired.
See S101 above, and will not be described here again.
S302, carrying out structuring processing on each initial data according to a preset label system to obtain at least one structuring data corresponding to each data system.
See S102 above, and will not be described here again.
S303, packaging at least one preset data calculation rule, at least one problem analysis rule and a plurality of structured data to obtain packaging data for providing data calculation service and problem analysis service.
See S203 above, and will not be described here again.
S304, receiving a calculation instruction and/or an analysis instruction sent by the electronic equipment.
The data processing server receives the calculation instructions and/or analysis instructions sent by the electronic equipment through a communication link with the electronic equipment. The calculation instructions and/or analysis instructions may be instructions in any form or format. The electronic device may be any of the electronic devices shown in fig. 1.
S305, calling the package data to perform data calculation corresponding to the calculation instruction in response to the calculation instruction, obtaining a calculation structure, and/or calling the package data to perform problem analysis corresponding to the analysis instruction in response to the analysis instruction, and obtaining an analysis result.
The calculation instruction is used for indicating the data processing server to call the package data to perform data calculation corresponding to the calculation instruction and obtain a calculation result, and the analysis instruction is used for indicating the data processing server to call the package data to perform problem analysis corresponding to the analysis instruction and obtain an analysis result.
For example, the calculation instructions sent by the electronic device instruct trend analysis calculations to be performed based on the vehicle speed related data in the test report data. The data processing server receives the calculation instruction, calls the package data to perform trend analysis calculation in response to the calculation instruction, obtains a trend analysis result as a calculation result, and further sends the trend analysis result to the electronic equipment, so that a user can view the trend analysis result through a display device of the electronic equipment, and the speed related data of a certain vehicle can be further known.
For another example, the analysis instruction sent by the electronic device indicates to compare process data between the battery power consumption and the temperature according to the test requirement data, the data processing server receives the analysis instruction and calls the package data to compare the process data in response to the analysis instruction, a comparison result is obtained as an analysis result, and the comparison result is further sent to the electronic device, so that a user can see the analysis result through a display device of the electronic device to know that the battery power consumption and the temperature are inversely related.
Through the technical scheme, the user can send the calculation instruction and/or the analysis instruction according to the actual calculation or analysis requirement, and check the calculation result and/or the analysis result, so that the complex personalized requirement of the user is met.
S306, sending the calculation result and/or the analysis result to the electronic equipment.
As shown in fig. 5, the calculation result may be any one or more of the data calculation results 2051 shown in fig. 5, and the analysis result may be any one or more of the problem analysis results 2052 shown in fig. 5. The data calculation results 2051 and the problem analysis results 2052 include tag data configuration results, accurate problem positioning results, test data query results, data analysis results, test solution analysis results, process data comparison results, and automatically generated test report results.
In the method, initial data respectively corresponding to a plurality of data systems are structured through a preset tag system, so that clean, accurate and complete data assets are accumulated, and the efficiency of inquiring and searching in mass data by a user is improved; further, in the specification, the plurality of structured data obtained after the structuring treatment, at least one preset data calculation rule and at least one problem analysis rule are packaged, and data calculation service and problem analysis service are provided for a user according to the packaged data, so that the problem of data island among a plurality of data systems is eliminated, the whole business flow from data to calculation to analysis is pulled through, and the data value is fully mined through multi-dimensional problem analysis layer by layer.
The following are device embodiments of the present application, which may be used to perform method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.
Referring to fig. 6, a schematic diagram of a data processing apparatus according to an exemplary embodiment of the present application is shown. The data processing apparatus may be implemented as all or part of an apparatus by software, hardware or a combination of both. The data processing apparatus includes: an initial acquisition module 401, a structure processing module 402, and a package processing module 403.
An initial acquisition module 401, configured to acquire initial data from a plurality of data systems respectively corresponding to the data systems;
the structure processing module 402 is configured to perform a structuring process on each initial data according to a preset tag system, so as to obtain at least one structured data corresponding to each data system;
the packaging processing module 403 is configured to perform packaging processing on at least one preset data calculation rule, at least one problem analysis rule, and a plurality of structured data, so as to obtain packaging data for providing a data calculation service and a problem analysis service.
In the method, initial data respectively corresponding to a plurality of data systems are structured through a preset tag system, so that clean, accurate and complete data assets are accumulated, and the efficiency of inquiring and searching in mass data by a user is improved; further, in the specification, the plurality of structured data obtained after the structuring treatment, at least one preset data calculation rule and at least one problem analysis rule are packaged, and data calculation service and problem analysis service are provided for a user according to the packaged data, so that the problem of data island among a plurality of data systems is eliminated, the whole business flow from data to calculation to analysis is pulled through, and the data value is fully mined through multi-dimensional problem analysis layer by layer.
It should be noted that, in the data processing apparatus provided in the foregoing embodiment, when the data processing method is executed, only the division of the foregoing functional modules is used as an example, in practical application, the foregoing functional allocation may be performed by different functional modules, that is, the internal structure of the device is divided into different functional modules, so as to complete all or part of the functions described above. In addition, the data processing apparatus and the data processing method embodiment provided in the foregoing embodiments belong to the same concept, which embody the detailed implementation process in the method embodiment, and are not described herein again.
The foregoing embodiment numbers of the present application are merely for describing, and do not represent advantages or disadvantages of the embodiments.
The embodiment of the present application further provides a computer storage medium, where the computer storage medium may store a plurality of instructions, where the instructions are adapted to be loaded by a processor and execute the data processing method according to the embodiment shown in fig. 1 to 5, and a specific execution process may refer to a specific description of the embodiment shown in fig. 1 to 5, which is not repeated herein.
The present application further provides a computer program product, where at least one instruction is stored, where the at least one instruction is loaded by the processor and executed by the processor to perform the data processing method according to the embodiment shown in fig. 1 to fig. 5, and the specific execution process may refer to the specific description of the embodiment shown in fig. 1 to fig. 5, which is not repeated herein.
Referring to fig. 7, a schematic structural diagram of an electronic device is provided in an embodiment of the present application. As shown in fig. 7, the electronic device 500 may include: at least one processor 501, at least one network interface 504, a user interface 503, a memory 505, at least one communication bus 502.
Wherein a communication bus 502 is used to enable connected communications between these components.
The user interface 503 may include a Display screen (Display) and a Camera (Camera), and the optional user interface 503 may further include a standard wired interface and a standard wireless interface.
The network interface 504 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface), among others.
Wherein the processor 501 may include one or more processing cores. The processor 501 connects various portions of the overall server 500 using various interfaces and lines to perform various functions of the server 500 and process data by executing or executing instructions, programs, code sets, or instruction sets stored in the memory 505, and invoking data stored in the memory 505. Alternatively, the processor 501 may be implemented in hardware in at least one of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), programmable logic array (Programmable Logic Array, PLA). The processor 501 may integrate one or a combination of several of a central processing unit (Central Processing Unit, CPU), an image processor (Graphics Processing Unit, GPU), and a modem, etc. The CPU mainly processes an operating system, a user interface, an application program and the like; the GPU is used for rendering and drawing the content required to be displayed by the display screen; the modem is used to handle wireless communications. It will be appreciated that the modem may not be integrated into the processor 501 and may be implemented by a single chip.
The Memory 505 may include a random access Memory (Random Access Memory, RAM) or a Read-Only Memory (Read-Only Memory). Optionally, the memory 505 comprises a non-transitory computer readable medium (non-transitory computer-readable storage medium). Memory 505 may be used to store instructions, programs, code sets, or instruction sets. The memory 505 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the above-described various method embodiments, etc.; the storage data area may store data or the like involved in the above respective method embodiments. The memory 505 may also optionally be at least one storage device located remotely from the processor 501. As shown in fig. 7, an operating system, a network communication module, a user interface module, and a data processing application may be included in the memory 505, which is a type of computer storage medium.
In the electronic device 500 shown in fig. 7, the user interface 503 is mainly used for providing an input interface for a user, and acquiring data input by the user; and the processor 501 may be used to invoke data processing applications stored in the memory 505 and to specifically execute data processing methods.
It will be appreciated by those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional modules is illustrated, and in practical application, the above-described functional allocation may be performed by different functional modules according to needs, i.e. the internal structure of the apparatus is divided into different functional modules to perform all or part of the functions described above.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of modules or units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another apparatus, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other forms.
The foregoing is merely specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily think about changes or substitutions within the technical scope of the present application, and the changes and substitutions are intended to be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (10)

1. A method of data processing, the method comprising:
acquiring initial data respectively corresponding to a plurality of data systems;
carrying out structuring treatment on each initial data according to a preset label system to obtain at least one structuring data corresponding to each data system;
and carrying out packaging processing on at least one preset data calculation rule, at least one problem analysis rule and a plurality of structured data to obtain packaging data for providing data calculation service and problem analysis service.
2. The method for processing data according to claim 1, wherein after the structuring processing is performed on each initial data according to a preset tag system to obtain at least one structured data corresponding to each data system, the method further comprises:
classifying the plurality of structured data according to a plurality of preset topics to obtain at least one structured data corresponding to each topic.
3. The method for processing data according to claim 1, wherein the structuring processing is performed on each initial data according to a preset tag system to obtain at least one structured data corresponding to each data system, and the method comprises:
According to the attribute information of a target tag in a plurality of tags included in a preset tag system, determining structural data matched with the attribute information of the target tag in the initial data, and obtaining at least one piece of structural data included in the initial data.
4. A data processing method according to claim 3, further comprising:
and carrying out association processing on each structured data and the target label corresponding to the structured data to obtain association data for query service.
5. The data processing method of claim, further comprising:
acquiring a label setting instruction;
in response to the tag setting instruction, a tag system including a plurality of tags is set.
6. The data processing method according to claim 1, characterized by further comprising:
receiving a calculation instruction and/or an analysis instruction sent by electronic equipment;
calling the packaging data to perform data calculation corresponding to the calculation instruction in response to the calculation instruction, and obtaining the calculation result, and/or calling the packaging data to perform problem analysis corresponding to the analysis instruction in response to the analysis instruction, and obtaining the analysis result;
And sending the calculation result and/or the analysis result to the electronic equipment.
7. A data processing apparatus, the apparatus comprising:
the initial acquisition module is used for acquiring initial data respectively corresponding to the plurality of data systems;
the structure processing module is used for carrying out structuring processing on the initial data according to a preset label system to obtain at least one structuring data corresponding to each data system;
and the packaging processing module is used for packaging at least one preset data calculation rule, at least one problem analysis rule and a plurality of structured data to obtain packaging data for providing data calculation service and problem analysis service.
8. A computer storage medium storing a plurality of instructions adapted to be loaded by a processor and to perform the method steps of any one of claims 1 to 6.
9. A computer program product, characterized in that it stores a plurality of instructions adapted to be loaded by a processor and to perform the method steps of any of claims 1-6.
10. An electronic device, wherein the vehicle comprises:
A memory for storing executable program code;
a processor for calling and running the executable program code from the memory to cause the vehicle to perform the method of any one of claims 1-6.
CN202311689717.7A 2023-12-08 2023-12-08 Data processing method, device, storage medium, program product and electronic equipment Pending CN117785918A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311689717.7A CN117785918A (en) 2023-12-08 2023-12-08 Data processing method, device, storage medium, program product and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311689717.7A CN117785918A (en) 2023-12-08 2023-12-08 Data processing method, device, storage medium, program product and electronic equipment

Publications (1)

Publication Number Publication Date
CN117785918A true CN117785918A (en) 2024-03-29

Family

ID=90380716

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311689717.7A Pending CN117785918A (en) 2023-12-08 2023-12-08 Data processing method, device, storage medium, program product and electronic equipment

Country Status (1)

Country Link
CN (1) CN117785918A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118096091A (en) * 2024-04-29 2024-05-28 山东逐浪云华数字科技有限公司 Mechanical engineering project demand analysis method, system, equipment and medium

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118096091A (en) * 2024-04-29 2024-05-28 山东逐浪云华数字科技有限公司 Mechanical engineering project demand analysis method, system, equipment and medium

Similar Documents

Publication Publication Date Title
CN111352921A (en) ELK-based slow query monitoring method and device, computer equipment and storage medium
WO2022252363A1 (en) Data processing method, computer device and readable storage medium
US20190147539A1 (en) Method and apparatus for outputting information
CN107862425B (en) Wind control data acquisition method, device and system and readable storage medium
CN113157947A (en) Knowledge graph construction method, tool, device and server
CN117785918A (en) Data processing method, device, storage medium, program product and electronic equipment
CN110866040B (en) User portrait generation method, device and system
US20130246463A1 (en) Prediction and isolation of patterns across datasets
CN109471783A (en) The method and apparatus for predicting task run parameter
CN104657437A (en) Monitoring method and monitoring device for promotion status data
US10762089B2 (en) Open ended question identification for investigations
CN112015971A (en) Recommendation method and device for cloud product, electronic equipment and computer readable medium
CN113626624B (en) Resource identification method and related device
CN110399459B (en) Online document searching method, device, terminal, server and storage medium
CN110059172B (en) Method and device for recommending answers based on natural language understanding
CN110019813A (en) Life insurance case retrieving method, retrieval device, server and readable storage medium storing program for executing
CN110674123B (en) Data preprocessing method, device, equipment and medium
CN111488386B (en) Data query method and device
CN115203194A (en) Metadata information generation method, related device, equipment and storage medium
CN116467607B (en) Information matching method and storage medium
CN116450723A (en) Data extraction method, device, computer equipment and storage medium
CN112363716A (en) Method, system and device for dynamically assembling evaluation model
CN111414553A (en) Data processing system, method and storage medium
CN111008078A (en) Batch processing method, device and equipment of data and computer storage medium
CN117592452A (en) Document template generation method and device, storage medium and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination