CN112671845A

CN112671845A - Data processing method and device, electronic equipment, storage medium and cloud system

Info

Publication number: CN112671845A
Application number: CN202011449101.9A
Authority: CN
Inventors: 乌尼日其其格; 杜孝平; 王羿凯; 褚文博; 张虹; 邓亚辉
Original assignee: Guoqi Beijing Intelligent Network Association Automotive Research Institute Co ltd
Current assignee: Guoqi Beijing Intelligent Network Association Automotive Research Institute Co ltd
Priority date: 2020-12-11
Filing date: 2020-12-11
Publication date: 2021-04-16
Anticipated expiration: 2040-12-11
Also published as: CN112671845B

Abstract

The disclosure discloses a data processing method, a data processing device, electronic equipment, a storage medium and a cloud system, wherein the data processing method comprises the steps of obtaining target data; classifying the target data according to the service request of the target data to obtain classified data; denoising the classified data to obtain denoised data; encrypting the de-noised data by using a public key to obtain a ciphertext; setting a header for the ciphertext according to the information type of the target data to obtain preprocessed data; and storing the preprocessed data into a data warehouse corresponding to the service request. The data is classified and stored according to the service request, so that the required data can be quickly found according to the service request when the service request is received, the service request processing time is greatly reduced, and the response speed of the service request is improved.

Description

Data processing method and device, electronic equipment, storage medium and cloud system

Technical Field

The disclosure relates to the technical field of intelligent networked automobiles, in particular to a data processing method and device, electronic equipment, a storage medium and a cloud system.

Background

In recent years, cloud platforms and cloud computing have made rapid progress, and more services are deployed on the cloud platforms to provide services to the outside. Especially, the communication capacity improvement brought by the 5G network provides powerful support for the development of intelligent networking automobiles.

The intelligent networked automobile cloud platform is a cloud platform which takes three-level cloud of edge cloud, regional cloud and central cloud as a framework. In the prior art, a unified storage management mode is adopted for the intelligent networked automobile cloud platform. The intelligent networked automobile cloud platform is large in data processing magnitude, information streams can be called among different middleware and platforms, and high requirements are placed on information transmission efficiency, information security guarantee and user privacy management. When the data stored in this way is called by a service request, it is determined for many times whether the data is data required by the service, and a large amount of determinations result in a great reduction in data processing efficiency.

Disclosure of Invention

The embodiment of the disclosure provides a data processing method and device, an electronic device, a storage medium and a cloud system, so as to at least solve the problem of low data processing efficiency of an existing cloud platform.

The technical scheme of the disclosure is as follows:

according to a first aspect of the embodiments of the present disclosure, there is provided a data processing method, including:

acquiring target data;

classifying the target data according to the service request of the target data to obtain classified data;

denoising the classified data to obtain denoised data;

encrypting the de-noised data by using a public key to obtain a ciphertext;

setting a header for the ciphertext according to the information type of the target data to obtain preprocessed data;

and storing the preprocessed data into a data warehouse corresponding to the service request.

Further, the target data includes at least one of:

vehicle end data;

path end data;

cloud data;

third party data.

Further, the denoising processing of the classified data includes at least one of:

removing redundant information in the classified data;

and removing the information with smaller value fluctuation range in the classified data.

Further, the service request corresponds to a service instruction, and the service instruction includes the following information:

the service corresponding priority, the service data, the calculation unit or middleware required by the service, the calculation required by the service and the data processing flow in the platform.

According to a second aspect of embodiments of the present disclosure, there is provided a data processing method, which may include:

receiving a service request;

inquiring a service instruction corresponding to the service request;

and extracting target data in a data warehouse corresponding to the business instruction, and adding a message with the business instruction to the target data to obtain message instruction data.

Further, the method further comprises:

calculating the message instruction data according to the service instruction to obtain processing data;

and transmitting the processing data.

Further, the calculating and processing the message instruction data according to the service instruction to obtain the processing data includes:

based on the service instruction, decrypting the target data by using a private key to obtain decrypted data;

and processing and calculating the decrypted data by using a calculating unit or a middleware to obtain the processing data.

Further, the method further comprises:

and destroying the service instruction based on the completion of sending the processing data.

According to a third aspect of the embodiments of the present disclosure, there is provided a data processing apparatus, which may include:

the acquisition module is used for acquiring target data;

the classification module is used for classifying the target data according to the service request of the target data to obtain classified data;

the de-noising module is used for de-noising the classified data to obtain de-noised data;

the encryption module is used for encrypting the de-noised data by using the public key to obtain a ciphertext;

the header setting module is used for setting a header for the ciphertext according to the information type of the target data to obtain preprocessed data;

and the storage module is used for storing the preprocessed data into the data warehouse corresponding to the service request.

According to a fourth aspect of embodiments of the present disclosure, there is provided a data processing apparatus, which may include:

the receiving module is used for receiving the service request;

the instruction query module is used for querying a service instruction corresponding to the service request;

and the extraction module is used for extracting the target data in the data warehouse corresponding to the service instruction.

According to a fifth aspect of the embodiments of the present disclosure, there is provided a cloud system, which may include:

the storage module comprises a plurality of data bins, and each data bin in the plurality of data bins is used for storing the data of the data bin corresponding to the service request;

the storage module stores a service instruction, and the service instruction is used for indicating a flow required by executing a service request, a middleware required to be deployed and a data bin where required data is located.

According to a sixth aspect of embodiments of the present disclosure, there is provided an electronic apparatus, which may include:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to execute the instructions to implement the data processing method as shown in any embodiment of the first aspect.

According to a seventh aspect of embodiments of the present disclosure, there is provided a storage medium, in which instructions are executed by a processor of an information processing apparatus or a server to cause the information processing apparatus or the server to implement the data processing method as shown in any one of the embodiments of the first aspect.

The technical scheme provided by the embodiment of the disclosure at least brings the following beneficial effects:

the embodiment of the disclosure obtains target data; classifying the target data according to the service request of the target data to obtain classified data; denoising the classified data to obtain denoised data; encrypting the de-noised data by using a public key to obtain a ciphertext; setting a header for the ciphertext according to the information type of the target data to obtain preprocessed data; and storing the preprocessed data into a data warehouse corresponding to the service request. The data is classified and stored according to the service request, so that the required data can be quickly found according to the service request when the service request is received, the service request processing time is greatly reduced, and the response speed of the service request is improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure and are not to be construed as limiting the disclosure.

FIG. 1 is a schematic flow diagram illustrating a data processing method according to an exemplary embodiment;

FIG. 2 is a schematic diagram of a data warehouse architecture, according to an exemplary embodiment;

FIG. 3 is an illustration of a business instruction structure according to an exemplary embodiment;

FIG. 4 is a cloud data sharing flow diagram, shown in accordance with an exemplary embodiment;

FIG. 5 is a schematic illustration of a different level business instruction processing flow shown in accordance with an exemplary embodiment;

FIG. 6 is a schematic diagram illustrating a data sharing process between cloud platforms in accordance with an illustrative embodiment;

FIG. 7 is a schematic diagram of an electronic device shown in accordance with an exemplary embodiment;

FIG. 8 is a diagram illustrating an electronic device hardware architecture in accordance with an exemplary embodiment.

Detailed Description

In order to make the technical solutions of the present disclosure better understood by those of ordinary skill in the art, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the disclosure described herein are capable of operation in sequences other than those illustrated or otherwise described herein. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

As shown in fig. 1, in a first aspect of the embodiments of the present disclosure, there is provided a data processing method, including:

step 100: acquiring target data;

step 200: classifying the target data according to the service request of the target data to obtain classified data;

step 300: denoising the classified data to obtain denoised data;

step 400: encrypting the de-noised data by using a public key to obtain a ciphertext;

step 500: setting a header for the ciphertext according to the information type of the target data to obtain preprocessed data;

step 600: and storing the preprocessed data into a data warehouse corresponding to the service request.

The invention provides a method for managing the cloud basic platform of the intelligent networked automobile, which takes the service request as the guide, carries out hierarchical processing on the service, allocates data resources under the calling of different service grades, and correspondingly, a large amount of basic data can be encrypted and recorded with information headers through a preprocessing stage.

The basic information can be divided into: the vehicle end data, the road end data, the cloud segment data and the third party data can further refine the information to meet the requirement of quick service information query: traffic participant personnel related data, traffic participant vehicle related data, road data, map data, edge cloud and MEC related data, cloud control base platform data, and the like, as shown in fig. 2.

The embodiment realizes the classified storage of the data according to the service request, so that the required data can be quickly found according to the service request when the service request is received, the service request processing time is greatly reduced, and the response speed of the service request is improved.

In some embodiments of the present disclosure, the target data includes at least one of:

vehicle end data;

path end data;

cloud data;

third party data.

In some embodiments of the present disclosure, denoising the classified data comprises at least one of:

removing redundant information in the classified data;

In some embodiments of the present disclosure, the service request corresponds to a service instruction, and the service instruction includes the following information:

When the information of different types is converged into the cloud platform, redundant information or information with a small numerical value fluctuation range is removed through information screening to relieve the calculation pressure. And then the screened information is respectively stored in corresponding data warehouses, so that the related information can be rapidly extracted when a service request exists. After the information is classified, the information enters a data warehouse to be encrypted through a public key to obtain a ciphertext, and an information header is correspondingly generated to correspond to the information category. The information preprocessing process is beneficial to more clearly planning information flow after screening, classifying and encrypting.

The invention provides a method for establishing a business warehouse in different cloud platforms (edge cloud, regional cloud and central cloud), and internally storing a series of business requirements, wherein each business requirement is provided with a series of information such as a flow, required data and required deployment middleware required for executing the business. In particular, some businesses may need to process data through a plurality of middleware for a second time to become a final result, and the data flow under the business requirement plays a role in standardizing the data trend and leading the data to travel under a correct path.

After the service is initiated, the service warehouse receives the instruction and searches a corresponding instruction set downwards; the instruction set goes to a data warehouse under the path to respectively extract the required data; after data are extracted, the data are sent to different middleware for calculation or secondary processing under the push of an instruction set; after the final calculation result is obtained, determining the next destination (sending to the next cloud platform or vehicle-mounted terminal) according to the standard flow and sending to the corresponding gateway; after the instruction set operation is completed, the log-off is performed, the memory is released, and the information contained in the service instruction is shown in fig. 3, and the specific flow is shown in fig. 4.

When data enters a corresponding computing unit or middleware, the data carries message information of the carried instruction, and the computing unit or middleware executes corresponding computing operation according to the instruction information.

receiving a service request;

inquiring a service instruction corresponding to the service request;

In some embodiments of the disclosure, the method further comprises:

and transmitting the processing data.

In some embodiments of the present disclosure, the calculating the message instruction data according to the service instruction to obtain the processing data includes:

The sharing and using process of the data among different middleware and different platforms is realized through the processes. The effective utilization of data is guaranteed by taking the service instruction as a guide, thereby not only avoiding the confusion of service calculation caused by simultaneously converging a large amount of data, but also avoiding the overstaffed caused by performing excessive grading modes on the data.

Optionally, the service instructions are classified according to a safety warning class, a traffic efficiency class and an information service class, and the priority is sequentially reduced. Aiming at the safety early warning service, the safety early warning service belongs to a high-level high-priority service, and in an information channel, an instruction with high priority can be preferentially calculated and sent so as to ensure that the vehicle-mounted terminal can preferentially receive safety early warning information. In the instruction set of the service warehouse, all information headers corresponding to the service priority are provided, and each computing unit in the platform can preferentially identify the header priority information to perform targeted computing. The business process is shown in figure 5.

In some embodiments of the disclosure, the method further comprises:

Optionally, when a certain item of data is called by the computing platform or the middleware at the same time to perform multiple service requests, the computing platform or the middleware preferably performs high-priority computation according to the instruction information packet carried in the data information.

Optionally, the data may be encrypted by a public key in the preprocessing stage, and the data carrying the service instruction may be decrypted by a private key after passing through the computing unit or the middleware, so as to be used.

When a certain cloud platform needs to acquire data from a data warehouse of another specified cloud platform, the cloud platform generates data request instruction information, and the instruction information is sent to the specified cloud platform through a corresponding gateway interface. And after the cloud platform receives the instruction information, the specified data is taken out of the data warehouse and is sent to the data request end through the gateway, and after the operation is finished, the instruction information is destroyed to release the computing memory. The business process is shown in fig. 6.

Specifically, taking a cloud platform as an example, according to the method in the embodiment, when data is imported into the cloud platform, the data is firstly uniformly entered into a data warehouse, and the data warehouse is subjected to detailed screening, cleaning, classification and encryption. Data can be divided by source into: the vehicle end data, the road end data, the cloud segment data and the third party data can further refine the information to meet the requirement of quick service information query: traffic participant personnel related data, traffic participant vehicle related data, road data, map data, edge cloud and MEC related data, cloud control base platform data, and the like. And further, an optimization engine can be established according to the data use frequency, and optimization processing is carried out on the data warehouse called frequently, so that the data warehouse can be called to each computing unit or middleware more smoothly and quickly. Each service is stored in a service warehouse inside the cloud platform, and accordingly, the instruction set corresponding to each service is in the service warehouse. It comprises the following components: the corresponding service priority, the data required by the service, the calculation unit or the middleware required by the execution of the service, the calculation to be executed by the data in the calculation unit or the middleware and the data processing flow. After the service instruction is executed, the instruction is automatically destroyed, and the memory space is released. The service instruction can carry the corresponding data to a computing platform or a middleware, and the computing platform or the middleware executes corresponding computation according to the corresponding service instruction identification when the data enters. After the calculation result is obtained, the service instruction can be sent to the corresponding gateway to be sent to the corresponding platform or terminal. The computing unit and the middleware in the three-level cloud architecture of the intelligent networked automobile can identify the priority level of a service instruction corresponding to the imported data, perform high-priority computing on safety early warning service requests, preferentially allocate memory computing resources and guarantee the safety protection of the intelligent networked automobile.

the acquisition module is used for acquiring target data;

the receiving module is used for receiving the service request;

a processor;

a memory for storing processor-executable instructions;

Optionally, as shown in fig. 7, an electronic device 700 is further provided in this embodiment of the present application, and includes a processor 701, a memory 702, and a program or an instruction stored in the memory 702 and executable on the processor 701, where the program or the instruction is executed by the processor 701 to implement each process of the data processing method embodiment, and can achieve the same technical effect, and no further description is provided here to avoid repetition.

It should be noted that the electronic device in the embodiment of the present application includes the mobile electronic device and the non-mobile electronic device described above.

Fig. 8 is a schematic diagram of a hardware structure of an electronic device implementing an embodiment of the present application.

The electronic device 800 includes, but is not limited to: a radio frequency unit 801, a network module 802, an audio output unit 803, an input unit 804, a sensor 805, a display unit 806, a user input unit 807, an interface unit 808, a memory 809, and a processor 810.

Those skilled in the art will appreciate that the electronic device 800 may further comprise a power source (e.g., a battery) for supplying power to the various components, and the power source may be logically connected to the processor 810 via a power management system, so as to manage charging, discharging, and power consumption management functions via the power management system. The electronic device structure shown in fig. 8 does not constitute a limitation of the electronic device, and the electronic device may include more or less components than those shown, or combine some components, or arrange different components, and thus, the description is omitted here.

It should be understood that in the embodiment of the present application, the input Unit 804 may include a Graphics Processing Unit (GPU) 8041 and a microphone 8042, and the Graphics Processing Unit 8041 processes image data of a still picture or a video obtained by an image capturing device (such as a camera) in a video capturing mode or an image capturing mode. The display unit 806 may include a display panel 8061, and the display panel 8061 may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like. The user input unit 807 includes a touch panel 8071 and other input devices 8072. A touch panel 8071, also referred to as a touch screen. The touch panel 8071 may include two portions of a touch detection device and a touch controller. Other input devices 8072 may include, but are not limited to, a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which are not described in detail herein. The memory 809 may be used to store software programs as well as various data including, but not limited to, application programs and operating systems. The processor 810 may integrate an application processor, which mainly handles operating systems, user interfaces, application programs, etc., and a modem processor, which mainly handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into processor 810.

The embodiment of the present application further provides a readable storage medium, where a program or an instruction is stored on the readable storage medium, and when the program or the instruction is executed by a processor, the program or the instruction implements each process of the data processing method embodiment, and can achieve the same technical effect, and in order to avoid repetition, details are not repeated here.

The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and so on.

The embodiment of the present application further provides a chip, where the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is configured to execute a program or an instruction to implement each process of the data processing method embodiment, and can achieve the same technical effect, and the details are not repeated here to avoid repetition.

It should be understood that the chips mentioned in the embodiments of the present application may also be referred to as system-on-chip, system-on-chip or system-on-chip, etc.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element. Further, it should be noted that the scope of the methods and apparatus of the embodiments of the present application is not limited to performing the functions in the order illustrated or discussed, but may include performing the functions in a substantially simultaneous manner or in a reverse order based on the functions involved, e.g., the methods described may be performed in an order different than that described, and various steps may be added, omitted, or combined. In addition, features described with reference to certain examples may be combined in other examples.

Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present application.

While the present embodiments have been described with reference to the accompanying drawings, it is to be understood that the invention is not limited to the precise embodiments described above, which are meant to be illustrative and not restrictive, and that various changes may be made therein by those skilled in the art without departing from the spirit and scope of the invention as defined by the appended claims.

Claims

1. A data processing method, comprising:

acquiring target data;

denoising the classified data to obtain denoised data;

encrypting the de-noising data by using a public key to obtain a ciphertext;

2. The method of claim 1, wherein the target data comprises at least one of:

vehicle end data;

path end data;

cloud data;

third party data.

3. The method of claim 1, wherein de-noising the classified data comprises at least one of:

removing redundant information in the classified data;

4. The method according to any of claims 1-3, wherein the service request corresponds to a service instruction, and the service instruction comprises the following information:

the service corresponding priority, the data of the service, the calculation unit or the middleware required by the service, the calculation required by the service and the data processing flow in the platform.

5. A data processing method, comprising:

receiving a service request;

inquiring a service instruction corresponding to the service request;

6. The method of claim 5, further comprising:

and sending the processing data.

7. The method according to claim 6, wherein the calculating the message instruction data according to the service instruction to obtain the processing data comprises:

and processing and calculating the decrypted data by using a calculating unit or a middleware to obtain processing data.

8. The method of claim 6, further comprising:

9. A data processing apparatus, comprising:

the acquisition module is used for acquiring target data;

the encryption module is used for encrypting the de-noising data by using a public key to obtain a ciphertext;

and the storage module is used for storing the preprocessed data into a data warehouse corresponding to the service request.

10. A data processing apparatus, comprising:

the receiving module is used for receiving the service request;

11. An electronic device, comprising:

a processor;

a memory for storing the processor-executable instructions;

wherein the processor is configured to execute the instructions to implement the data processing method of any one of claims 1-8.

12. A storage medium characterized in that instructions in the storage medium, when executed by a processor of an information processing apparatus or a server, cause the information processing apparatus or the server to realize the data processing method according to any one of claims 1 to 8.

13. A cloud system, comprising: