CN110750206A - Data processing method, device and system - Google Patents

Data processing method, device and system Download PDF

Info

Publication number
CN110750206A
CN110750206A CN201810821368.2A CN201810821368A CN110750206A CN 110750206 A CN110750206 A CN 110750206A CN 201810821368 A CN201810821368 A CN 201810821368A CN 110750206 A CN110750206 A CN 110750206A
Authority
CN
China
Prior art keywords
data
server
processed
storage space
management server
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201810821368.2A
Other languages
Chinese (zh)
Other versions
CN110750206B (en
Inventor
罗庆超
许呙兢
徐伟杰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alibaba Group Holding Ltd
Original Assignee
Alibaba Group Holding Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba Group Holding Ltd filed Critical Alibaba Group Holding Ltd
Priority to CN201810821368.2A priority Critical patent/CN110750206B/en
Publication of CN110750206A publication Critical patent/CN110750206A/en
Application granted granted Critical
Publication of CN110750206B publication Critical patent/CN110750206B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0602Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0608Saving storage space on storage systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638Organizing or formatting or addressing of data
    • G06F3/0644Management of space entities, e.g. partitions, extents, pools
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0668Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

The application discloses a data processing method, device and system. Wherein, the method comprises the following steps: receiving a data processing request from terminal equipment, wherein the data processing request is used for requesting to process data to be processed; responding to the data processing request, sending a space allocation request to a data server so that the data server allocates a storage space for the to-be-processed data based on the space allocation request; receiving a storage space address returned by the data server; and sending the storage space address to the terminal equipment so that the terminal equipment can send the data to be processed to the data server and store the data to the storage space. The method and the device solve the technical problem that direct data interaction processing cannot be achieved between the terminal device and the data server in the prior art.

Description

Data processing method, device and system
Technical Field
The present application relates to the field of computer technologies, and in particular, to a data processing method, apparatus, and system.
Background
With the continuous update and development of artificial intelligence technology, the artificial intelligence technology has become the leading technology in the field of science and technology. In a conventional cooperation mode between a terminal device and a cloud server, since the computing power of the terminal device (e.g., a camera device or a smart phone) is limited, all data is only uploaded to the cloud server, and then the AI capability on the cloud server is provided by using the powerful computing power on the cloud server.
The existing cooperation mode has the following defects: a large amount of network bandwidth and data storage area is required, and artificial intelligence AI analysis costs are high. Therefore, a new cooperation mode needs to be provided, that is, the terminal device integrates the artificial intelligence function to obtain the intelligent device, so that face identification, article identification and the like can be performed on the intelligent device, and all data do not need to be transmitted to the cloud server.
However, in the above prior art, there are problems how to manage a plurality of terminal devices and implement interactive processing of data between the terminal devices and the data server, and no effective solution has been proposed at present.
Disclosure of Invention
The embodiment of the application provides a data processing method, a data processing device and a data processing system, which are used for at least solving the technical problem that direct data interaction processing cannot be realized between terminal equipment and a data server in the prior art.
According to an aspect of an embodiment of the present application, there is provided a data processing method, including: receiving a data processing request from terminal equipment, wherein the data processing request is used for requesting to process data to be processed; responding to the data processing request, sending a space allocation request to a data server so that the data server allocates a storage space for the to-be-processed data based on the space allocation request; receiving a storage space address returned by the data server; and sending the storage space address to the terminal equipment so that the terminal equipment can send the data to be processed to the data server and store the data to the storage space.
According to an aspect of the embodiments of the present application, there is also provided a data processing method, including: sending a data processing request to a management server so that the management server requests to allocate a storage space to the data server, wherein the data processing request is used for requesting to process data to be processed, and the storage space is used for storing the data to be processed; receiving the storage space address returned by the data server and forwarded by the management server; and sending the data to be processed to the data server and storing the data to the storage space.
According to an aspect of the embodiments of the present application, there is also provided a data processing method, including: receiving a space allocation request from a management server; responding to the space allocation request, and allocating a storage space for the data to be processed of the terminal equipment; returning a storage space address to the management server so that the management server can send the storage space address to the terminal equipment; and receiving the data to be processed sent by the terminal equipment, and storing the data to be processed into the storage space.
According to another aspect of the embodiments of the present application, there is also provided a data processing apparatus, including: the first receiving module is used for receiving a data processing request from the terminal equipment, wherein the data processing request is used for requesting to process data to be processed; a first response module, configured to send a space allocation request to a data server in response to the data processing request, so that the data server allocates a storage space for the to-be-processed data based on the space allocation request; the second receiving module is used for receiving the storage space address returned by the data server; and the first sending module is used for sending the storage space address to the terminal equipment so that the terminal equipment can send the data to be processed to the data server and store the data to the storage space.
According to another aspect of the embodiments of the present application, there is also provided a data processing apparatus, including: a second sending module, configured to send a data processing request to a management server, so that the management server requests to allocate a storage space to the data server, where the data processing request is used to request to process data to be processed, and the storage space is used to store the data to be processed; a third receiving module, configured to receive the address of the storage space returned by the data server and forwarded by the management server; and the third sending module is used for sending the data to be processed to the data server and storing the data to the storage space.
According to another aspect of the embodiments of the present application, there is also provided a data processing apparatus, including: the fourth receiving module is used for receiving the space allocation request from the management server; the second response module is used for responding to the space allocation request and allocating storage space for the data to be processed of the terminal equipment; a return module, configured to return a storage space address to the management server, so that the management server sends the storage space address to the terminal device; and a fifth receiving module, configured to receive the to-be-processed data sent by the terminal device, and store the to-be-processed data in the storage space.
According to another aspect of the embodiments of the present application, there is also provided a data processing system, including: the management server is connected with the terminal equipment and used for responding to a data processing request initiated by the terminal equipment, sending a space allocation request to the data server, receiving a storage space address returned by the data server, and sending the storage space address to the terminal equipment so that the terminal equipment can send the data to be processed to the data server and store the data to be processed in a storage space; and the data server is connected with the management server and used for allocating the storage space for the data to be processed based on the space allocation request and returning the storage space address to the management server.
According to another aspect of the embodiments of the present application, there is also provided a storage medium, where the storage medium includes a stored program, and when the program runs, the apparatus on which the storage medium is located is controlled to perform the following functions: receiving a data processing request from terminal equipment, wherein the data processing request is used for requesting to process data to be processed; responding to the data processing request, sending a space allocation request to a data server so that the data server allocates a storage space for the to-be-processed data based on the space allocation request; receiving a storage space address returned by the data server; and sending the storage space address to the terminal equipment so that the terminal equipment can send the data to be processed to the data server and store the data to the storage space.
According to another aspect of the embodiments of the present application, there is also provided a processor, configured to execute a program, where the program executes to perform the following functions: receiving a data processing request from terminal equipment, wherein the data processing request is used for requesting to process data to be processed; responding to the data processing request, sending a space allocation request to a data server so that the data server allocates a storage space for the to-be-processed data based on the space allocation request; receiving a storage space address returned by the data server; and sending the storage space address to the terminal equipment so that the terminal equipment can send the data to be processed to the data server and store the data to the storage space.
In the embodiment of the application, a data processing request from a terminal device is received, wherein the data processing request is used for requesting to process data to be processed; responding to the data processing request, sending a space allocation request to a data server so that the data server allocates a storage space for the to-be-processed data based on the space allocation request; receiving a storage space address returned by the data server; and sending the storage space address to the terminal equipment so that the terminal equipment can send the data to be processed to the data server and store the data to the storage space.
Therefore, the purpose of directly carrying out data interaction processing between the terminal equipment and the data server is achieved, the technical effects of effectively reducing network bandwidth and data storage areas and reducing artificial intelligence AI analysis cost are achieved, and the technical problem that direct data interaction processing cannot be carried out between the terminal equipment and the data server in the prior art is solved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
FIG. 1 is a block diagram of a data processing system according to an embodiment of the present application;
FIG. 2 is a block diagram of an alternative data processing system according to an embodiment of the present application;
fig. 3 is a block diagram of a hardware structure of a computer terminal (or mobile device) for implementing a data processing method according to an embodiment of the present application;
FIG. 4 is a flow chart of a method of data processing according to an embodiment of the present application;
FIG. 5 is a flow chart of an alternative data processing method according to an embodiment of the present application;
FIG. 6 is a flow chart of an alternative data processing method according to an embodiment of the present application;
FIG. 7 is an interaction flow diagram of an alternative data processing method according to an embodiment of the application;
FIG. 8 is an interaction flow diagram of an alternative data processing method according to an embodiment of the application;
FIG. 9 is a flow chart of another data processing method according to an embodiment of the present application;
FIG. 10 is a flow chart of yet another method of data processing according to an embodiment of the present application;
FIG. 11 is a block diagram of a data processing apparatus according to an embodiment of the present application;
FIG. 12 is a schematic diagram of another data processing apparatus according to an embodiment of the present application; and
fig. 13 is a schematic structural diagram of another data processing apparatus according to an embodiment of the present application.
Detailed Description
In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only partial embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
First, some terms or terms appearing in the description of the embodiments of the present application are applicable to the following explanations:
artificial Intelligence (AI) is a new technical science to study and develop theories, methods, techniques and application systems for simulating, extending and expanding human Intelligence. As a branch of computer science, artificial intelligence refers to intelligence represented by artificially manufactured machines, i.e., human-like intelligence technology implemented by means of ordinary computer programs.
Metadata, which may also be referred to as intermediate data, relay data, refers to information about the organization, data domain, and relationship of data, and in short, data describing data, mainly information describing data attributes, is used to support functions such as indicating storage location, history data, resource lookup, file record, and the like.
Example 1
Before describing further details of embodiments of the present application, one suitable data processing system that may be used to implement the principles of the present application will be described with reference to FIG. 1.
FIG. 1 is a block diagram of a data processing system in accordance with an embodiment of the present application, the depicted architecture is only one example of a suitable environment for purposes of description and is not intended to suggest any limitation as to the scope of use or functionality of the application. Neither should the data processing system be interpreted as having any dependency or requirement relating to any one or combination of components illustrated in FIG. 1.
The system embodiment provided by embodiment 1 of the present application can be widely applied to the internet, for example, in the technical field of artificial intelligence. Due to the following disadvantages of the existing collaboration mode: a large amount of network bandwidth and data storage area is required, and artificial intelligence AI analysis costs are high. Moreover, through the hammer of the AI technology on the cloud server, the AI is gradually miniaturized and is in a chip form, so that the artificial intelligence function can be integrated on the terminal device to obtain the intelligent device, face recognition, character recognition, voice recognition, NLP (Natural Language Processing) and the like can be performed on the intelligent device, all data do not need to be transmitted to the cloud server, the network bandwidth and the data storage area can be effectively reduced by adopting a new cooperation mode, and the artificial intelligence AI analysis cost can be effectively reduced.
However, in the above prior art, there are problems how to manage a plurality of terminal devices and implement interactive processing of data between the terminal devices and the data server, and no effective solution has been proposed at present.
In order to solve the above problems, according to the embodiments of the present application, an embodiment of a data processing system is provided, which can achieve the purpose of directly performing data interaction processing between a terminal device and a data server, thereby achieving the technical effects of effectively reducing network bandwidth and data storage area, and reducing artificial intelligence AI analysis cost, and further solving the technical problem that data interaction processing cannot be directly performed between a terminal device and a data server in the prior art.
It should be noted that the data processing system provided in the embodiment of the present application may be applied to scenes such as new retail, video monitoring, and the like, but not limited to the scenes, and the purpose of greatly reducing network cost may be achieved.
As shown in fig. 1, the data processing system includes: terminal equipment 101, management server 103, data server 105, wherein:
a management server 103, connected to the terminal device 101, configured to send a space allocation request to a data server in response to a data processing request initiated by the terminal device, receive a storage space address returned by the data server, and send the storage space address to the terminal device, so that the terminal device sends the data to be processed to the data server and stores the data in a storage space; the data server 105 is connected to the management server 103, and is configured to allocate the storage space for the to-be-processed data based on the space allocation request, and return the storage space address to the management server.
As an optional embodiment, the terminal device 101 is configured to send a data processing request, where the data processing request is used to request processing of data to be processed; the terminal device 101 may be any artificial intelligence AI device, such as an AI camera device, an AI smartphone, an AI wearable device, and the like.
The AI device has an artificial intelligence AI processing capability, for example, an artificial intelligence AI chip may be integrated into the AI device, so that functions such as face recognition, character recognition, voice recognition, NLP, etc. may be implemented, and all data (e.g., audio data, image data, video data, etc.) may not be uploaded to a data server for processing, but may be processed locally at the AI device.
In an alternative embodiment, the management server 103 may be a management server for managing data cooperation between the terminal device and the data server, where the management server has, but is not limited to, a terminal device registration function and a storage space allocation function of the data server, and may also manage a plurality of terminal devices and implement data interaction between the terminal device and the data server.
In another optional embodiment, the data server refers to a cloud server that can directly perform data interaction with the terminal device, and may be, for example, any cloud data server, where the data server may be, but is not limited to, capable of understanding AI processing capability information and metadata format information of the terminal device.
It should be noted that, the management server and the data server are implemented by different software modules, and may be deployed by using different servers, so that respective performance and isolation may be better ensured, and the management server and the data server may also be deployed on the same server.
In an alternative embodiment, the terminal device may register on the management server in advance, and after the registration is completed, the terminal device may perform data collaboration with the data server.
As an optional embodiment, if data coordination between the terminal device and the data server needs to be implemented, a connection between the terminal device and the data server needs to be established, and the terminal device initiates a data processing request to the management server, where the data processing request carries a device ID of the terminal device, data characteristic information (e.g., AI processing capability information and metadata format information), and the like. The management server responds to the data processing request and sends a space allocation request to the data server, the data server allocates a storage space for the data to be stored, the address of the storage space is returned to the management server, and the management server returns the address of the storage space to the terminal equipment; and the terminal equipment uploads the data to be processed to the data server according to the received address of the storage space and stores the data to the corresponding storage space.
In an optional embodiment, the data server allocates a block of storage space to the terminal device in the data coordination in response to the space allocation request sent by the management server, and when allocating the storage space, one of the existing storage spaces may be selected, or a block of storage space may be newly created.
In the optional embodiment, the space allocation request carries a cloud account number for accessing the data server by the management server, the management server stores an association relationship between the device ID of the terminal device and the cloud account number, and the management server determines the cloud account number according to the device ID in the data processing request and requests the data server to allocate a storage space by using the cloud account number.
It should be noted that the terminal device 101, the management server 103, and the data server 105 may perform communication connection by accessing a Wireless Local Area Network (WLAN), the internet, bluetooth, RFID, Zigbee, NFC, and the like, so as to perform data interaction and data processing.
It should be noted that, in the control plane, the management server may, but is not limited to, implement registration of the terminal device, and may manage millions of devices; the ability to allocate data access channels to the terminal device on the data server is achieved through the management server. In a data layer, the management server can also realize a terminal device direct transmission data server, wherein the metadata and the data share a transmission channel, so that the efficiency of data interaction between the terminal device and the data server can be effectively improved.
As another optional embodiment, while applying for a storage space to be accessed by the terminal device to a data server by using the cloud account, the management server may send data characteristic information (e.g., AI processing capability information and metadata format information) of the terminal device to the data server, so that after the terminal device sends the to-be-processed data to the data server, the data server verifies the to-be-processed data according to the data characteristic information.
In an optional embodiment, after returning the address information of the storage space to the terminal device, the management server may further obtain cloud service access statistical information from the data server, where the cloud service access statistical information is used to record an address information interaction process between the terminal device and the storage space.
In an optional embodiment, before the terminal device sends the data processing request to the management server, the terminal device may further send a registration request to the management server, where the registration request includes: a terminal device identifier; after the management server receives the registration request, the management server allocates a data server to the terminal device, and stores the association relationship between the terminal device identifier and the data server identifier, so that the data server corresponding to the terminal device can be determined according to the stored association relationship before the management server subsequently sends a space allocation request to the data server.
In another optional embodiment, before the terminal device sends the data processing request to the management server, the terminal device may further send a registration request to the management server, where the registration request includes: a terminal device identifier and a data server identifier; after the management server receives the registration request, the management server does not need to allocate a data server to the terminal device, but directly stores the association relationship between the terminal device identifier and the data server identifier, and can determine the data server corresponding to the terminal device according to the stored association relationship before the management server subsequently sends a space allocation request to the data server.
In another optional embodiment, after the terminal device sends the data to be processed to the data server and stores the data to the storage space, the management server is further configured to receive feedback information sent by the data server, where the feedback information includes: whether the process information was successfully stored and stored.
Fig. 2 is a schematic structural diagram of an alternative data processing system according to an embodiment of the present application, and as shown in fig. 2, the alternative data processing system includes: a terminal device 101, a management server 103, a data server 105, a policy control module 1051, a first storage area 1053, and a second storage area 1055.
In an alternative embodiment, the data to be processed includes user data and metadata associated with the user data, and the data server 105 includes: a policy control module 1051, a first storage area 1053, a second storage area 1055, wherein:
the policy control module 1051 is configured to store data feature information, where the data feature information may be used to verify validity of user data and metadata of the user data (i.e., metadata of data to be processed); a first storage area 1053 for storing the user data; the second storage area 1055 is used to store the above-described metadata.
It should be noted that the policy refers to format definitions of user data and metadata, and the policy control refers to that the data server needs to detect whether the user data and the metadata uploaded by the terminal device conform to corresponding format definitions, and if unmatched fields occur, errors are indicated, so that whether errors occur in data uploaded by the terminal device can be detected, and whether attacks by malicious software are detected.
As an alternative embodiment, the data server receives a data processing request from the terminal device, where the data processing request includes: storing a space address, AI processing capacity, user data to be written and metadata; the data server verifies whether the format information of the storage space address, the AI processing capability and the metadata in the data processing request is consistent with the corresponding storage space address, the AI processing capability and the format information of the metadata when the storage space is allocated, if so, the data server stores the user data, otherwise, the data server refuses to store the user data, and after the data server successfully stores the user data, the data server respectively sends feedback messages to the terminal equipment and the management server.
In the above optional embodiment, the data server extracts user data and the metadata from the data processing request, verifies the validity of the user data by using the data feature information, and stores the user data in the first storage area if the user data passes the verification; and verifying the validity of the metadata by using the data characteristic information, and storing the metadata in the second storage area when the metadata passes the verification.
In an optional embodiment, taking the terminal device as an AI camera device as an example, the AI camera device may capture a picture of a face existing in a video, where the picture is user data, and may analyze the picture through an intelligent camera of the AI camera device to obtain face information in the picture, for example, position information of the face in the picture, age information and gender information corresponding to the face, where the face information is metadata. And then the AI camera device can directly transmit the user data (face picture) and the metadata (face information in the picture) to the data server.
According to an alternative embodiment, metadata format information of the AI camera may be defined in advance, and may be, for example, information defined in JSON format or XML format for indicating an AI processing capability of the AI camera and a data field format processed under the AI processing capability. For example, metadata format information of the AI image capturing apparatus can be defined as follows:
in the above metadata format information, the Data type of the Data field is Object, and is used for storing picture content description (such as a classmate gathering photo); the data type of the AIType field is String, which is used to indicate the AI processing capability (e.g., face recognition) provided by the AI device; the AIMeta field defines a format of a data field processed under the AI processing capability, wherein a data type of the Age field is Int for indicating an Age (e.g., 22 years) corresponding to a recognized face, and a data type of the Confidence field is Float for indicating a Confidence level (e.g., 98.8%) of the Age recognition result.
The metadata format information defined above may be used as the data characteristic information of the AI image capturing apparatus. When the AI image pickup apparatus transmits a data processing request to the management server, the data processing request carries the data feature information together. The management server sends the data characteristic information to a data server, so that when the AI camera device uploads user data and metadata to the data server, the data server verifies whether the data uploaded by the AI camera device is legal or not according to the data characteristic information, and if the format of the metadata uploaded by the AI device is consistent with the data characteristic information, the uploaded user data and metadata are considered to be legal; otherwise, the uploaded user data and metadata are considered to be illegal.
Optionally, the first storage area and the second storage area are operation support system OSS cloud storage spaces for which the data server applies to the upper cloud. In another optional embodiment, the data server stores the user data in a first storage area, and the data server stores the metadata in a second storage area, that is, the data server stores the user data and the metadata in different or the same OSS cloud storage spaces.
In the embodiment of the application, the management server can manage a plurality of terminal devices according to the AI processing capability and the metadata of the terminal devices, and can also realize resource docking with the data server by using a registration mechanism, thereby realizing management of million-level terminal devices.
In addition, according to the optional embodiment provided by the application, through a direct transmission protocol between the terminal device and the data server, the terminal device can control the data transmission process, for example, metadata information is carried during transmission, so that the utilization rate of network bandwidth can be effectively improved, and in addition, on the data server, the search management of data and metadata of a plurality of terminal devices can be realized, so that the lower network bandwidth can be realized, the method and the system are applied to scenes such as new retail, video monitoring and the like, and the network cost is greatly reduced.
Example 2
In accordance with an embodiment of the present application, there is provided an embodiment of a data processing method, it should be noted that the steps shown in the flowchart of the figure may be performed in a computer system such as a set of computer executable instructions, and that while a logical order is shown in the flowchart, in some cases the steps shown or described may be performed in an order different than here.
The method provided by the embodiment of the application can be executed in a mobile terminal, a computer terminal or a similar operation device. Fig. 3 shows a hardware configuration block diagram of a computer terminal (or mobile device) for implementing the data processing method. As shown in fig. 3, the computer terminal 10 (or mobile device 10) may include one or more (shown as 102a, 102b, … …, 102 n) processors 102 (the processors 102 may include, but are not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission module 106 for communication functions. Besides, the method can also comprise the following steps: a display, an input/output interface (I/O interface), a Universal Serial Bus (USB) port (which may be included as one of the ports of the I/O interface), a network interface, a power source, and/or a camera. It will be understood by those skilled in the art that the structure shown in fig. 3 is only an illustration and is not intended to limit the structure of the electronic device. For example, the computer terminal 10 may also include more or fewer components than shown in FIG. 3, or have a different configuration than shown in FIG. 3.
It should be noted that the one or more processors 102 and/or other data processing circuitry described above may be referred to generally herein as "data processing circuitry". The data processing circuitry may be embodied in whole or in part in software, hardware, firmware, or any combination thereof. Further, the data processing circuit may be a single stand-alone processing module, or incorporated in whole or in part into any of the other elements in the computer terminal 10 (or mobile device). As referred to in the embodiments of the application, the data processing circuit acts as a processor control (e.g. selection of a variable resistance termination path connected to the interface).
The memory 104 may be used to store software programs and modules of application software, such as program instructions/data storage devices corresponding to the data processing method in the embodiment of the present application, and the processor 102 executes various functional applications and data processing by executing the software programs and modules stored in the memory 104, that is, implementing the data processing method of the application program. The memory 104 may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory located remotely from the processor 102, which may be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission device 106 is used for receiving or transmitting data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network adapter (NIC) that can be connected to other Network devices through a base station to communicate with the internet. In one example, the transmission device 106 can be a Radio Frequency (RF) module, which is used to communicate with the internet in a wireless manner.
The display may be, for example, a touch screen type Liquid Crystal Display (LCD) that may enable a user to interact with a user interface of the computer terminal 10 (or mobile device).
Under the above operating environment, the present application provides a data processing method as shown in fig. 4. It should be noted that, in the data processing method embodiment provided in the embodiment of the present application, the application scenario and the technical scenario may be, but are not limited to, the same as those described in embodiment 1, for example, but are not limited to, being applied to a scenario of new retail, video monitoring, and the like, and the purpose of greatly reducing the network cost may be achieved.
Fig. 4 is a flowchart of a data processing method according to an embodiment of the present application, and as shown in fig. 4, the method further includes the following method steps:
step S102, receiving a data processing request from the terminal device, wherein the data processing request is used for requesting to process the data to be processed.
Specifically, in this embodiment of the present application, the execution subject of step S102 to step S108 may be, but is not limited to, a management server, for example, any optional or preferred management server in embodiment 1 above.
As an alternative embodiment, the terminal device may be any artificial intelligence AI device, for example, an AI camera device, an AI smartphone, an AI wearable device, and the like.
The terminal device has an artificial intelligence AI processing capability, for example, an artificial intelligence AI chip may be integrated into the terminal device, so that functions such as face recognition, character recognition, voice recognition, NLP, and the like may be implemented, and all data (for example, audio data, image data, video data, and the like) need not be uploaded to a data server for processing, but the data may be processed locally at the terminal device.
In an alternative embodiment, the management server may be a management server for managing data collaboration between the terminal device and the data server, where the management server has, but is not limited to, a terminal device registration function and a storage space allocation function of the data server, and may also manage a plurality of terminal devices and implement data collaboration between the terminal device and the data server.
Step S104, responding to the data processing request, sending a space allocation request to a data server, so that the data server allocates a storage space for the to-be-processed data based on the space allocation request.
In another alternative embodiment, the data server refers to a cloud server that can directly perform data interaction with the terminal device, and may be, for example, any cloud data server, where the data server may be, but is not limited to, capable of understanding AI processing capability information and metadata format information of the terminal device.
It should be noted that, the management server and the data server are implemented by different software modules, and may be deployed by using different servers, so that respective performance and isolation may be better ensured, and the management server and the data server may also be deployed on the same server.
And step S106, receiving the storage space address returned by the data server.
Step S108, sending the address of the storage space to the terminal device, so that the terminal device sends the to-be-processed data to the data server and stores the to-be-processed data in the storage space.
In an alternative embodiment, the terminal device may register on the management server in advance, and after the registration is completed, the terminal device may perform data collaboration with the data server.
As an optional embodiment, if data coordination between the terminal device and the data server needs to be implemented, a connection between the terminal device and the data server needs to be established, and the terminal device initiates a data processing request to the management server, where the data processing request carries a device ID of the terminal device, data characteristic information (e.g., AI processing capability information, metadata format information, and the like), and the like. The management server responds to the data processing request and sends a space allocation request to the data server, the data server allocates a storage space for the data to be stored, the address of the storage space is returned to the management server, and the management server returns the address of the storage space to the terminal equipment; and the terminal equipment uploads the data to be processed to the data server according to the received address of the storage space and stores the data to the corresponding storage space.
In an optional embodiment, the data server allocates a block of storage space to the terminal device in the data coordination in response to the space allocation request sent by the management server, and when allocating the storage space, one of the existing storage spaces may be selected, or a block of storage space may be newly created.
In the optional embodiment, the space allocation request carries a cloud account number for accessing the data server by the management server, the management server stores an association relationship between the device ID of the terminal device and the cloud account number, and the management server determines the cloud account number according to the device ID in the data processing request and requests the data server to allocate a storage space by using the cloud account number.
It should be noted that the terminal device, the management server, and the data server may perform communication connection by accessing a Wireless Local Area Network (WLAN), the internet, bluetooth, RFID, Zigbee, NFC, and the like, so as to perform data interaction and data processing.
It should be noted that, in the control plane, the management server may, but is not limited to, implement registration of the terminal device, and may manage millions of devices; the ability to allocate data access channels to the terminal device on the data server is achieved through the management server. In a data layer, the management server can also realize a terminal device direct transmission data server, wherein the metadata and the data share a transmission channel, so that the efficiency of data interaction between the terminal device and the data server can be effectively improved.
Based on the schemes defined in the steps S102 to S106, it can be known that a data processing request from a terminal device is received through a management server, where the data processing request is used for requesting processing of data to be processed; responding to the data processing request, sending a space allocation request to a data server so that the data server allocates a storage space for the to-be-processed data based on the space allocation request; receiving a storage space address returned by the data server; and sending the storage space address to the terminal equipment so that the terminal equipment can send the data to be processed to the data server and store the data to the storage space.
Through the scheme provided by the embodiment of the application, the purpose of directly carrying out data interaction processing between the terminal equipment and the data server can be achieved, so that the technical effects of effectively reducing network bandwidth and data storage areas and reducing artificial intelligence AI analysis cost are achieved, and the technical problem that the data interaction processing cannot be directly carried out between the terminal equipment and the data server in the prior art is solved.
In an optional embodiment, the data processing request includes: data characteristic information; the above method further comprises the following method steps:
step S202, in response to the data processing request, sending the data feature information to the data server, so that after the terminal device sends the data to be processed to the data server, the data server verifies the data to be processed according to the data feature information.
Optionally, the data characteristic information may be, but is not limited to: AI processing capability information and metadata format information of the terminal device, and the like.
In the above optional embodiment, the management server may further, but is not limited to, send the data feature information of the terminal device to the data server while applying for a storage space accessed by the terminal device to the data server by using the cloud account.
In an alternative embodiment, the process of verifying the to-be-processed data by the data server according to the data characteristic information may be, but is not limited to, verifying the validity of the to-be-processed data, for example, the data server may verify the validity of the to-be-processed data by using the data characteristic information, and store the to-be-processed data in a storage space when the validity of the to-be-processed data passes the verification.
As an optional embodiment, after the terminal device sends the data to be processed to the data server and stores the data in the storage space, the management server may further receive feedback information sent by the data server, where the feedback information includes: whether the process information was successfully stored and stored. That is, after receiving the data to be processed, the data server returns feedback information to the management server to feed back whether the data to be processed of the terminal device is successfully stored or not to the management server in real time, and stores the storage process of the data to be processed.
In an optional embodiment, fig. 5 is a flowchart of an optional data processing method according to an embodiment of the present application, and as shown in fig. 5, before receiving the data processing request from the terminal device, the method further includes:
step S302, receiving a registration request from the terminal device, where the registration request includes: a terminal device identifier;
step S304, responding to the registration request, allocating the data server to the terminal device;
step S306, storing the association relationship between the terminal device identifier and the data server identifier.
It should be noted that, the terminal device is registered and activated on the management server, and the management server may manage the terminal device, for example, allocate a storage space of the data server to the terminal device, so as to implement data coordination between the terminal device and the data server.
Based on the optional embodiments provided in steps S302 to S306, the terminal device may register on the management server by sending a registration request to the management server, and the management server may respond to the received registration request, for example, may return the following response information: and confirming the reception of the registration request, and registering and activating the terminal equipment.
Moreover, since the registration request sent by the terminal device includes the terminal device identifier, after the management server receives the registration request, the management server allocates a data server to the terminal device and stores the association relationship between the terminal device identifier and the data server identifier, and before the management server subsequently sends a space allocation request to the data server, the data server corresponding to the terminal device can be determined according to the stored association relationship.
In another alternative embodiment, fig. 6 is a flowchart of an alternative data processing method according to an embodiment of the present application, and as shown in fig. 6, before receiving the data processing request from the terminal device, the method further includes:
step S402, receiving a registration request from the terminal device, where the registration request includes: a terminal device identifier and a data server identifier;
step S404, storing the association relationship between the terminal device identifier and the data server identifier.
Unlike the optional embodiments provided in steps S302 to S306, in the optional embodiments provided in steps S402 to S404, the registration request sent by the terminal device may further include a terminal device identifier and a data server identifier, after the management server receives the registration request, the management server does not need to allocate a data server to the terminal device, directly stores the association relationship between the terminal device identifier and the data server identifier, and may determine the data server corresponding to the terminal device according to the stored association relationship before the management server subsequently sends a space allocation request to the data server.
As an alternative embodiment, fig. 7 is an interaction flowchart of an alternative data processing method according to an embodiment of the present application, and as shown in fig. 7, the method further includes the following method steps:
in step S501, the terminal device sends a registration request to the management server.
In step S501, the terminal device may be any artificial intelligence AI device, such as an AI camera device, an AI smartphone, an AI wearable device, and the like.
In an alternative embodiment, the management server is a management side for managing cooperation between the terminal device and the data server, and for example, the management server may be a management server, the management server has, but is not limited to, a device registration function and a storage space allocation function of the data server, may manage a plurality of terminal devices, and implement data cooperation between the terminal device and the data server.
Step S502, the management server returns response information to the terminal equipment.
As an alternative embodiment, the terminal device may register on the management server by sending a registration request, and the management server may respond to the received registration request information, for example, may return a response message, confirm that the registration request information is received, and register and activate the terminal device.
It should be noted that, the terminal device is registered and activated in the management server, and the management server can manage the terminal device, for example, allocate a storage space of the data server to the terminal device.
Step S503, the terminal device sends a data processing request to the management server, where the data processing request is used to request processing of the data to be processed.
Step S504, the management server sends a space allocation request to the data server in response to the data processing request, so that the data server allocates a storage space for the to-be-processed data based on the space allocation request.
Step S505, the management server receives the address of the storage space returned by the data server.
Step S506, the management server sends the storage space address to the terminal device.
And step S507, the terminal device sends the data to be processed to the data server, and stores the data to the storage space.
As an alternative embodiment, the management server responds to the data processing request and sends a space allocation request to the data server, the data server allocates a storage space for the data to be stored, and returns the address of the storage space to the management server, and the management server returns the address of the storage space to the terminal device.
As an alternative embodiment, the data processing request includes: data characteristic information; the management server may also, but is not limited to, send the data characteristic information to the data server in response to the data processing request, so that after the terminal device sends the data to be processed to the data server, the data server verifies the data to be processed according to the data characteristic information.
Step S508, the management server receives the feedback information sent by the data server.
In step S508, the feedback information includes: whether the process information was successfully stored and stored.
It should be noted that, the optional embodiments provided in the foregoing steps S501 to S508 mainly relate to a control plane, and the management server may implement, but is not limited to, registration of an intelligent device and management of a million-class device; through the management server provided by the embodiment of the application, the technical effect of distributing the data access channel on the data server for the intelligent equipment can be realized.
As another alternative embodiment, fig. 8 is an interaction flowchart of an alternative data processing method according to an embodiment of the present application, and as shown in fig. 8, the method further includes the following method steps:
step S601, the data server receives a data processing request from the terminal device.
Optionally, in step S601, the data server may receive the data processing request, for example, a data storage request, via a storage space allocated to the terminal device; the data to be processed comprises user data and metadata associated with the user data, and the data processing request at least carries the user data to be stored and the metadata associated with the user data.
In an optional embodiment, after receiving the storage space address returned by the management server, the intelligent device may further perform intelligent analysis on the user data to extract the metadata; and packaging the user data and the metadata into the data processing request, and sending the data processing request to a data server.
Optionally, the above-mentioned packing processing may be, but is not limited to, a data interaction format Protocol Buffer technology, and the packing processing is performed.
As an alternative embodiment, the terminal device may send the data processing request, and the data server may receive the data processing request.
Step S602, the data server verifies the validity of the data processing request.
In the above step S602, since the management server has allocated the storage space of the data server for the corresponding terminal device, in the case where the terminal device accesses the data server, the data server can verify the validity of the data processing request of the terminal device, for example, by verifying whether the data processing request is made through the corresponding storage space.
It should be noted that the legal data processing request may carry user information, and the validity of the data processing request of the terminal device may be determined by verifying the user information carried in the data processing request; in addition, the validity of the data processing request of the terminal device can be determined by verifying the signature carried in the data processing request.
Step S603, in case of passing the verification, the data server extracts the user data and the metadata from the data processing request, respectively;
step S604, the data server adopts the data characteristic information to verify the legality of the user data and the metadata;
in step S605, if the user data is verified, the data server stores the user data in the first storage area.
Based on the above optional embodiment, after extracting the user data, the data server may verify the validity of the user data by using the data feature information, and store the user data in the first storage area if the user data is verified.
Step S606, the data server returns the user data writing success information to the terminal device.
In step S607, in the case that the metadata passes the verification, the data server stores the metadata to the second storage area.
Based on the above alternative embodiment, after extracting the metadata, the data server may verify the validity of the metadata by using the data feature information, and store the metadata in the second storage area if the metadata is verified.
In an optional embodiment, taking the terminal device as an AI camera device as an example, the AI camera device may capture a picture of a face existing in a video, where the picture is user data, and may analyze the picture through an intelligent camera of the AI camera device to obtain face information in the picture, for example, position information of the face in the picture, age information and gender information corresponding to the face, where the face information is metadata. And then the terminal equipment can directly transmit the user data (the face picture) and the metadata (the face information in the picture) to a data server.
Optionally, the first storage area and the second storage area are operation support system OSS cloud storage spaces for which the data server applies to the upper cloud. In another optional embodiment, the data server stores the user data in a first storage area, and the data server stores the metadata in a second storage area, that is, the data server stores the user data and the metadata in different or the same OSS cloud storage spaces.
In step S608, the data server returns metadata writing success information to the terminal device.
Step S609, the data server returns the data processing request success information to the terminal device.
Based on the above optional embodiment, after the user data and the metadata both pass the verification and the data server returns the user data and metadata writing success information to the terminal device, the data server returns the data processing request success information to the terminal device.
Step S610, the management server obtains the cloud service access statistical information from the data server.
In the step S610, the cloud service access statistical information is used to record an information interaction process between the terminal device and the storage space.
As an optional embodiment, in the process of data interaction between the data server and the terminal device, the information interaction process between the terminal device and the storage space may be recorded to obtain the cloud service access statistical information, and the cloud service access statistical information is stored locally in the data server.
After the management server returns the address information of the storage space to the terminal device, cloud service access statistical information can be acquired from the data server, so that management of millions of devices is facilitated, the capacity of distributing data access channels on the data server for the intelligent device is achieved, and the efficiency of data interaction between the intelligent device and the data server is improved.
It should be noted that, the optional embodiments provided in steps S601 to S610 are mainly, but not limited to, related to a data plane, and the management server may further implement an intelligent device direct transmission data server, where metadata and data share a transmission channel, so that efficiency of data interaction between the intelligent device and the data server may be effectively improved.
It should be noted that, reference may be made to the relevant description in embodiment 1 for a preferred implementation of this embodiment, and details are not described here again.
Example 3
According to embodiments of the present application, there is also provided another data processing method, where it is noted that the steps shown in the flowchart of the figure may be executed in a computer system such as a set of computer executable instructions, and that while a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in an order different from that shown or described herein.
Under the same or similar operating environment as that of embodiment 2 above, the present application provides a data processing method as shown in fig. 9. Fig. 9 is a flow chart of another data processing method according to an embodiment of the present application, as shown in fig. 9, the method further includes the following method steps:
step S902, sending a data processing request to a management server, so that the management server requests to allocate a storage space to the data server, where the data processing request is used to request to process data to be processed, and the storage space is used to store the data to be processed.
Specifically, the execution subject of steps S902 to S906 in this embodiment may be, but is not limited to, a terminal device, and for example, may be any optional or preferred terminal device in embodiment 1 described above.
As an alternative embodiment, the terminal device may be any artificial intelligence AI device, for example, an AI camera device, an AI smartphone, an AI wearable device, and the like.
In an alternative embodiment, the management server may be a management server for managing data collaboration between the terminal device and the data server, where the management server has, but is not limited to, a terminal device registration function and a storage space allocation function of the data server, and may also manage a plurality of terminal devices and implement data collaboration between the terminal device and the data server.
In another alternative embodiment, the data server refers to a cloud server that can directly perform data interaction with the terminal device, and may be, for example, any cloud data server, where the data server may be, but is not limited to, capable of understanding AI processing capability information and metadata format information of the terminal device.
It should be noted that, the management server and the data server are implemented by different software modules, and may be deployed by using different servers, so that respective performance and isolation may be better ensured, and the management server and the data server may also be deployed on the same server.
As an alternative embodiment, the terminal device may register on the management server in advance, and after the registration is completed, the terminal device may perform data collaboration with the data server.
Step S904, receiving the address of the storage space returned by the data server and forwarded by the management server.
Step S906, sending the data to be processed to the data server, and storing the data to the storage space.
In an optional embodiment, after the terminal device initiates a data processing request to the management server, the data processing request carries a device ID, data characteristic information (e.g., AI processing capability information, metadata format information, etc.) and the like of the terminal device; the management server responds to the data processing request and sends a space allocation request to the data server, the data server allocates a storage space for the data to be stored, the address of the storage space is returned to the management server, and the management server returns the address of the storage space to the terminal equipment; and the terminal equipment uploads the data to be processed to the data server according to the received address of the storage space and stores the data to the corresponding storage space.
In an optional embodiment, the data server allocates a block of storage space to the terminal device in the data coordination in response to the space allocation request sent by the management server, and when allocating the storage space, one of the existing storage spaces may be selected, or a block of storage space may be newly created.
In the optional embodiment, the space allocation request carries a cloud account number for accessing the data server by the management server, the management server stores an association relationship between the device ID of the terminal device and the cloud account number, and the management server determines the cloud account number according to the device ID in the data processing request and requests the data server to allocate a storage space by using the cloud account number.
It should be noted that the terminal device, the management server, and the data server may perform communication connection by accessing a Wireless Local Area Network (WLAN), the internet, bluetooth, RFID, Zigbee, NFC, and the like, so as to perform data interaction and data processing.
It should be noted that, in the control plane, the management server may, but is not limited to, implement registration of the terminal device, and may manage millions of devices; the ability to allocate data access channels to the terminal device on the data server is achieved through the management server. In a data layer, the management server can also realize a terminal device direct transmission data server, wherein the metadata and the data share a transmission channel, so that the efficiency of data interaction between the terminal device and the data server can be effectively improved.
Based on the schemes defined in the steps S902 to S906, it can be known that a data processing request is sent to a management server through a terminal device, so that the management server requests to allocate a storage space to the data server, where the data processing request is used to request to process data to be processed, and the storage space is used to store the data to be processed; receiving the storage space address returned by the data server and forwarded by the management server; and sending the data to be processed to the data server and storing the data to the storage space.
Through the scheme provided by the embodiment of the application, the purpose of directly carrying out data interaction processing between the terminal equipment and the data server can be achieved, so that the technical effects of effectively reducing network bandwidth and data storage areas and reducing artificial intelligence AI analysis cost are achieved, and the technical problem that the data interaction processing cannot be directly carried out between the terminal equipment and the data server in the prior art is solved.
In an optional embodiment, the data processing request includes: data characteristic information, which is used for verifying the data to be processed according to the data characteristic information by the data server after the data to be processed is sent to the data server.
Optionally, the data characteristic information may be, but is not limited to: AI processing capability information and metadata format information of the terminal device, and the like.
In the above optional embodiment, the management server may further, but is not limited to, send the data feature information of the terminal device to the data server while applying for a storage space accessed by the terminal device to the data server by using the cloud account.
In an alternative embodiment, the process of verifying the to-be-processed data by the data server according to the data characteristic information may be, but is not limited to, verifying the validity of the to-be-processed data, for example, the data server may verify the validity of the to-be-processed data by using the data characteristic information, and store the to-be-processed data in a storage space when the validity of the to-be-processed data passes the verification.
As an alternative embodiment, before sending the data processing request to the management server, the method further includes:
step S1000, sending a registration request to the management server so that the management server allocates the data server to the terminal device, where the registration request includes: and identifying the terminal equipment.
It should be noted that, the terminal device is registered and activated on the management server, and the management server may manage the terminal device, for example, allocate a storage space of the data server to the terminal device, so as to implement data coordination between the terminal device and the data server.
In the optional embodiment, the terminal device may register on the management server by sending a registration request to the management server, and the management server may respond to the received registration request, for example, may return the following response information: and confirming the reception of the registration request, and registering and activating the terminal equipment.
Moreover, since the registration request sent by the terminal device includes the terminal device identifier, after the management server receives the registration request, the management server allocates a data server to the terminal device and stores the association relationship between the terminal device identifier and the data server identifier, and before the management server subsequently sends a space allocation request to the data server, the data server corresponding to the terminal device can be determined according to the stored association relationship.
As another optional embodiment, before sending the data processing request to the management server, the method further includes:
step S1002, sending a registration request to the management server, so that the management server obtains a terminal device identifier and a data server identifier from the registration request, and stores an association relationship between the terminal device identifier and the data server identifier.
Step S1004, receiving a data processing request from the management server, where the data processing request at least carries a cloud identifier, and the cloud identifier corresponds to an equipment identifier of the artificial intelligence terminal equipment.
Unlike the optional embodiment provided in step S1000, in the optional embodiment provided in steps S1002 to S1004, the registration request sent by the terminal device may further include a terminal device identifier and a data server identifier, after the management server receives the registration request, the management server does not need to allocate a data server to the terminal device, but directly stores the association relationship between the terminal device identifier and the data server identifier, and may determine the data server corresponding to the terminal device according to the stored association relationship before the management server subsequently sends a space allocation request to the data server.
It should be noted that, in the control plane, the management server may, but is not limited to, implement registration of the terminal device and manage million-level devices; the ability to allocate data access channels to the terminal device on the data server is achieved through the management server.
It should be noted that, in the data plane, the management server may also implement a terminal device direct transmission data server, where the metadata and the data share a transmission channel, which may effectively improve the efficiency of data interaction between the terminal device and the data server.
In an optional embodiment, after receiving the address of the storage space sent by the management server, the method further includes:
step S1102 is to transmit a data processing request to the data server.
In an optional embodiment, the data processing request carries at least user data to be processed and metadata associated with the user data.
In another alternative embodiment, the terminal device may send the data processing request, and the data server may receive the data processing request.
And step S1104, receiving an access response message returned by the data server.
As an alternative embodiment, the access response message is used to indicate that both the user data and the metadata have been successfully stored in the data server.
In an optional embodiment, the data server receives a data processing request from the terminal device, where the data processing request includes: storing a space address, AI processing capacity, user data to be written and metadata; the data server verifies whether the format information of the storage space address, the AI processing capability and the metadata in the data processing request is consistent with the corresponding storage space address, the AI processing capability and the format information of the metadata when the storage space is allocated, if so, the data server stores the user data, otherwise, the data server refuses to store the user data, and after the data server successfully stores the user data, the data server respectively sends feedback messages to the terminal equipment and the management server.
In the above optional embodiment, the data server extracts user data and the metadata from the data processing request, verifies the validity of the user data by using the data feature information, and stores the user data in the first storage area if the user data passes the verification; and verifying the validity of the metadata by using the data characteristic information, and storing the metadata in the second storage area when the metadata passes the verification.
Optionally, the policy refers to format definitions of user data and metadata, and the policy control refers to that the data server needs to detect whether the user data and the metadata uploaded by the terminal device conform to corresponding format definitions, and if a field that does not match occurs, an error is indicated, so that whether data uploaded by the terminal device is in error or not and whether the data is attacked by malicious software or not can be detected.
In an optional embodiment, taking the terminal device as an AI camera device as an example, the AI camera device may capture a picture of a face existing in a video, where the picture is user data, and may analyze the picture through an intelligent camera of the AI camera device to obtain face information in the picture, for example, position information of the face in the picture, age information and gender information corresponding to the face, where the face information is metadata. And then the terminal equipment can directly transmit the user data (the face picture) and the metadata (the face information in the picture) to a data server.
Optionally, the first storage area and the second storage area are operation support system OSS cloud storage spaces for which the data server applies to the upper cloud. In another optional embodiment, the data server stores the user data in a first storage area, and the data server stores the metadata in a second storage area, that is, the data server stores the user data and the metadata in different or the same OSS cloud storage spaces.
In an alternative embodiment, before sending the data processing request to the data server, the method further includes:
step S1202, intelligently analyzing the user data and extracting the metadata;
step S1204, performing a packaging process on the user data and the metadata, and encapsulating the user data and the metadata in the data processing request.
In an optional embodiment, after receiving the address information of the storage space returned by the management server, before sending the data processing request to the data server, the terminal device may further perform intelligent analysis on the user data to extract the metadata; and packaging the user data and the metadata into the data processing request, and sending the data processing request to a data server.
Optionally, the above-mentioned packing processing may be, but is not limited to, a data interaction format Protocol Buffer technology, and the packing processing is performed.
The "data processing request" in embodiment 3 may be the same as or different from the "data processing request" in embodiment 2. The terms "first", "second", and the like in the embodiments and drawings of the present application are used for distinguishing similar objects, and are not particularly limited.
It should be noted that, reference may be made to the relevant description in embodiments 1 and 2 for a preferred implementation of this embodiment, and details are not described here again.
Example 4
There is also provided in accordance with an embodiment of the present application a further data processing method, in which the steps illustrated in the flowchart of the figure may be performed in a computer system such as a set of computer executable instructions, and where a logical order is illustrated in the flowchart, in some cases the steps illustrated or described may be performed in an order different than here.
Under the same or similar operating environment as that of embodiment 2 described above, the present application provides a data processing method as shown in fig. 10. Fig. 10 is a flowchart of a further data processing method according to an embodiment of the present application, and as shown in fig. 10, the method further includes the following method steps:
in step S1302, a space allocation request is received from the management server.
Specifically, the execution subjects of step S1302 to step S1308 may be, but are not limited to, data servers, for example, any optional or preferred data server in embodiment 1 above.
In the optional embodiment, the space allocation request carries a cloud account number for accessing the data server by the management server, the management server stores an association relationship between the device ID of the terminal device and the cloud account number, and the management server determines the cloud account number according to the device ID in the data processing request and requests the data server to allocate a storage space by using the cloud account number.
Step S1304, responding to the space allocation request, allocating a storage space for the data to be processed of the terminal device.
As an alternative embodiment, the terminal device may be any artificial intelligence AI device, for example, an AI camera device, an AI smartphone, an AI wearable device, and the like.
The terminal device has an artificial intelligence AI processing capability, for example, an artificial intelligence AI chip may be integrated into the terminal device, so that functions such as face recognition, character recognition, voice recognition, NLP, and the like may be implemented, and all data (for example, audio data, image data, video data, and the like) need not be uploaded to a data server for processing, but the data may be processed locally at the terminal device.
In an alternative embodiment, the management server may be a management server for managing data collaboration between the terminal device and the data server, where the management server has, but is not limited to, a terminal device registration function and a storage space allocation function of the data server, and may also manage a plurality of terminal devices and implement data collaboration between the terminal device and the data server.
Step S1306 returns a storage space address to the management server so that the management server transmits the storage space address to the terminal device.
Step S1308, receiving the to-be-processed data sent by the terminal device, and storing the to-be-processed data in the storage space.
In another alternative embodiment, the data server refers to a cloud server that can directly perform data interaction with the terminal device, and may be, for example, any cloud data server, where the data server may be, but is not limited to, capable of understanding AI processing capability information and metadata format information of the terminal device.
It should be noted that, the management server and the data server are implemented by different software modules, and may be deployed by using different servers, so that respective performance and isolation may be better ensured, and the management server and the data server may also be deployed on the same server.
In an alternative embodiment, the terminal device may register on the management server in advance, and after the registration is completed, the terminal device may perform data collaboration with the data server.
As an optional embodiment, if data coordination between the terminal device and the data server needs to be implemented, a connection between the terminal device and the data server needs to be established, the terminal device initiates a data processing request to the management server, where the data processing request carries a device ID, data feature information, and the like of the terminal device, where the data feature information may include AI processing capability information, metadata format information, and the like.
The management server responds to the data processing request and sends a space allocation request to the data server, the data server allocates a storage space for the data to be stored, the address of the storage space is returned to the management server, and the management server returns the address of the storage space to the terminal equipment; and the terminal equipment uploads the data to be processed to the data server according to the received address of the storage space and stores the data to the corresponding storage space.
It should be noted that the terminal device, the management server, and the data server may perform communication connection by accessing a Wireless Local Area Network (WLAN), the internet, bluetooth, RFID, Zigbee, NFC, and the like, so as to perform data interaction and data processing.
It should be noted that, in the control plane, the management server may, but is not limited to, implement registration of the terminal device, and may manage millions of devices; the ability to allocate data access channels to the terminal device on the data server is achieved through the management server. In a data layer, the management server can also realize a terminal device direct transmission data server, wherein the metadata and the data share a transmission channel, so that the efficiency of data interaction between the terminal device and the data server can be effectively improved.
Based on the schemes defined in the above steps S1302 to S1308, it can be known that the space allocation request from the management server is received by the data server; responding to the space allocation request, and allocating a storage space for the data to be processed of the terminal equipment; returning a storage space address to the management server so that the management server can send the storage space address to the terminal equipment; and receiving the data to be processed sent by the terminal equipment, and storing the data to be processed into the storage space.
Through the scheme provided by the embodiment of the application, the purpose of directly carrying out data interaction processing between the terminal equipment and the data server can be achieved, so that the technical effects of effectively reducing network bandwidth and data storage areas and reducing artificial intelligence AI analysis cost are achieved, and the technical problem that the data interaction processing cannot be directly carried out between the terminal equipment and the data server in the prior art is solved.
In an alternative embodiment, the method further comprises the following method steps:
step S1402, receiving the data characteristic information sent by the management server;
in step S1404, the data to be processed is verified according to the data feature information.
Optionally, the data characteristic information may be, but is not limited to: AI processing capability information and metadata format information of the terminal device, and the like.
In the above optional embodiment, the management server may further, but is not limited to, send the data feature information of the terminal device to the data server while applying for a storage space accessed by the terminal device to the data server by using the cloud account.
In an optional embodiment, the to-be-processed data includes user data and metadata associated with the user data, and the step of verifying the to-be-processed data according to the data feature information may be implemented by the following method steps: judging whether the format of the metadata is consistent with the data characteristic information; if yes, the data to be processed is stored in the storage space after passing the verification; if not, the verification fails, and the to-be-processed data is refused to be stored.
In an alternative embodiment, the process of verifying the to-be-processed data by the data server according to the data feature information may be, but is not limited to, verifying the validity of the to-be-processed data, for example, the data server may determine whether the format of the metadata is consistent with the data feature information, and if the determination result is yes, determine that the verification is passed, and store the to-be-processed data in the storage space.
In an optional embodiment, after receiving the to-be-processed data sent by the terminal device and storing the to-be-processed data in the storage space, the method further includes:
step S1502 returns feedback information to the management server, where the feedback information includes: whether the process information was successfully stored and stored.
In the step S1502, after receiving the to-be-processed data, the data server returns feedback information to the management server, so as to feed back whether to successfully store the to-be-processed data of the terminal device to the management server in real time, and store a storage process of the to-be-processed data.
The "data processing request" in embodiment 4 may be the same as or different from the "data processing request" in embodiments 2 and 3 described above. The terms "first", "second", and the like in the embodiments and drawings of the present application are used for distinguishing similar objects, and are not particularly limited.
It should be noted that, reference may be made to the relevant description in embodiments 1, 2 and 3 for a preferred implementation of this embodiment, and details are not described here again.
Example 5
According to an embodiment of the present application, there is further provided an embodiment of a data processing apparatus for implementing the data processing method, and fig. 11 is a schematic structural diagram of the data processing apparatus according to the embodiment of the present application, and as shown in fig. 11, the apparatus includes: a first receiving module 110, a first responding module 112, a second receiving module 114, and a first transmitting module 116, wherein:
a first receiving module 110, configured to receive a data processing request from a terminal device, where the data processing request is used to request to process data to be processed; a first response module 112, configured to send a space allocation request to a data server in response to the data processing request, so that the data server allocates a storage space for the to-be-processed data based on the space allocation request; a second receiving module 114, configured to receive a storage space address returned by the data server; a first sending module 116, configured to send the address of the storage space to the terminal device, so that the terminal device sends the data to be processed to the data server, and stores the data to the storage space.
It should be noted here that the first receiving module 110, the first responding module 112, the second receiving module 114 and the first sending module 116 correspond to steps S102 to S108 in embodiment 2, and the four modules are the same as the corresponding steps in the implementation example and application scenario, but are not limited to the disclosure in embodiment 2. It should be noted that the above modules as a part of the apparatus may be operated in the computer terminal 10 provided in the embodiment.
It should be noted that, reference may be made to the relevant description in embodiment 2 for a preferred implementation of this embodiment, and details are not described here again.
Example 6
According to an embodiment of the present application, there is provided another embodiment of a data processing apparatus for implementing the data processing method, and fig. 12 is a schematic structural diagram of the data processing apparatus according to the embodiment of the present application, and as shown in fig. 12, the apparatus includes: a second sending module 120, a third receiving module 122, and a third sending module 124, wherein:
a second sending module 120, configured to send a data processing request to a management server, so that the management server requests to allocate a storage space to the data server, where the data processing request is used to request to process data to be processed, and the storage space is used to store the data to be processed; a third receiving module 122, configured to receive the address of the storage space returned by the data server and forwarded by the management server; a third sending module 124, configured to send the to-be-processed data to the data server, and store the to-be-processed data in the storage space.
It should be noted here that the second sending module 120, the third receiving module 122, and the third sending module 124 correspond to steps S902 to S906 in embodiment 3, and the three modules are the same as the corresponding steps in the implementation example and the application scenario, but are not limited to the disclosure in embodiment 3. It should be noted that the above modules as a part of the apparatus may be operated in the computer terminal 10 provided in the embodiment.
It should be noted that, reference may be made to the relevant description in embodiment 3 for a preferred implementation of this embodiment, and details are not described here again.
Example 7
According to an embodiment of the present application, there is further provided another embodiment of a data processing apparatus for implementing the data processing method, and fig. 13 is a schematic structural diagram of the data processing apparatus according to the embodiment of the present application, and as shown in fig. 13, the apparatus includes: a fourth receiving module 130, a second responding module 132, a returning module 134, and a fifth receiving module 136, wherein:
a fourth receiving module 130, configured to receive a space allocation request from the management server; a second response module 132, configured to respond to the space allocation request, allocate a storage space for the to-be-processed data of the terminal device; a returning module 134, configured to return a storage space address to the management server, so that the management server sends the storage space address to the terminal device; a fifth receiving module 136, configured to receive the to-be-processed data sent by the terminal device, and store the to-be-processed data in the storage space.
It should be noted here that the fourth receiving module 130, the second responding module 132, the returning module 134, and the fifth receiving module 136 correspond to steps S1302 to S1308 in embodiment 4, and the four modules are the same as the corresponding steps in the implementation example and application scenario, but are not limited to the disclosure in embodiment 4. It should be noted that the above modules as a part of the apparatus may be operated in the computer terminal 10 provided in the embodiment.
It should be noted that, reference may be made to the relevant description in embodiment 4 for a preferred implementation of this embodiment, and details are not described here again.
Example 8
The embodiment of the application can provide a computer terminal, and the computer terminal can be any one computer terminal device in a computer terminal group. Optionally, in this embodiment, the computer terminal may also be replaced with a terminal device such as a mobile terminal.
Optionally, in this embodiment, the computer terminal may be located in at least one network device of a plurality of network devices of a computer network.
The method provided by the embodiment of the application can be executed in a mobile terminal, a computer terminal or a similar operation device. As shown in fig. 3, the computer terminal 10 (or mobile device 10) may include one or more (shown as 102a, 102b, … …, 102 n) processors 102 (the processors 102 may include, but are not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission module 106 for communication functions.
The processor 102 may call the information and applications stored in the memory through the transmission device to perform the following steps: receiving a data processing request from terminal equipment, wherein the data processing request is used for requesting to process data to be processed; responding to the data processing request, sending a space allocation request to a data server so that the data server allocates a storage space for the to-be-processed data based on the space allocation request; receiving a storage space address returned by the data server; and sending the storage space address to the terminal equipment so that the terminal equipment can send the data to be processed to the data server and store the data to the storage space.
The memory 104 may be used to store software programs and modules of application software, such as program instructions/data storage devices corresponding to the data processing method in the embodiment of the present application, and the processor 102 executes various functional applications and data processing by executing the software programs and modules stored in the memory 104, that is, implementing the data processing method described above. The memory 104 may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory located remotely from the processor 102, which may be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission device 106 is used for receiving or transmitting data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network adapter (NIC) that can be connected to other Network devices through a base station to communicate with the internet. In one example, the transmission device 106 can be a Radio Frequency (RF) module, which is used to communicate with the internet in a wireless manner.
The display may be, for example, a touch screen type Liquid Crystal Display (LCD) that may enable a user to interact with the user interface of the computer terminal 10.
It should be noted here that in some alternative embodiments, the computer terminal 10 shown in fig. 8 described above may include hardware elements (including circuitry), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware and software elements. It should be noted that fig. 8 is only one example of a particular specific example and is intended to illustrate the types of components that may be present in the computer terminal 10 described above.
In this embodiment, the computer terminal may execute program codes of the following steps in the data processing method of the application program: receiving a data processing request from terminal equipment, wherein the data processing request is used for requesting to process data to be processed; responding to the data processing request, sending a space allocation request to a data server so that the data server allocates a storage space for the to-be-processed data based on the space allocation request; receiving a storage space address returned by the data server; and sending the storage space address to the terminal equipment so that the terminal equipment can send the data to be processed to the data server and store the data to the storage space.
Optionally, the processor may further execute the program code of the following steps: receiving a data processing request from terminal equipment, wherein the data processing request is used for requesting to process data to be processed; responding to the data processing request, sending a space allocation request to a data server so that the data server allocates a storage space for the to-be-processed data based on the space allocation request; receiving a storage space address returned by the data server; and sending the storage space address to the terminal equipment so that the terminal equipment can send the data to be processed to the data server and store the data to the storage space.
Optionally, the processor may further execute the program code of the following steps: and sending the data characteristic information to the data server in response to the data processing request, so that the data server verifies the data to be processed according to the data characteristic information after the terminal equipment sends the data to be processed to the data server.
Optionally, the processor may further execute the program code of the following steps: receiving feedback information sent by the data server, wherein the feedback information includes: whether the process information was successfully stored and stored.
Optionally, the processor may further execute the program code of the following steps: receiving a registration request from the terminal device, wherein the registration request includes: a terminal device identifier; responding to the registration request, and distributing the data server to the terminal equipment; and storing the association relationship between the terminal equipment identification and the data server identification.
Optionally, the processor may further execute the program code of the following steps: receiving a registration request from the terminal device, wherein the registration request includes: a terminal device identifier and a data server identifier; and storing the association relationship between the terminal equipment identification and the data server identification.
Optionally, the processor may further execute the program code of the following steps: and determining the data server according to the association relation.
Optionally, the processor may further execute the program code of the following steps: sending a data processing request to a management server so that the management server requests to allocate a storage space to the data server, wherein the data processing request is used for requesting to process data to be processed, and the storage space is used for storing the data to be processed; receiving the storage space address returned by the data server and forwarded by the management server; and sending the data to be processed to the data server and storing the data to the storage space.
Optionally, the processor may further execute the program code of the following steps: sending a registration request to the management server so that the management server allocates the data server to the terminal device, wherein the registration request includes: and identifying the terminal equipment.
Optionally, the processor may further execute the program code of the following steps: and sending a registration request to the management server so that the management server can acquire a terminal device identifier and a data server identifier from the registration request and store the association relationship between the terminal device identifier and the data server identifier.
Optionally, the processor may further execute the program code of the following steps: receiving a space allocation request from a management server; responding to the space allocation request, and allocating a storage space for the data to be processed of the terminal equipment; returning a storage space address to the management server so that the management server can send the storage space address to the terminal equipment; and receiving the data to be processed sent by the terminal equipment, and storing the data to be processed into the storage space.
Optionally, the processor may further execute the program code of the following steps: receiving data characteristic information sent by the management server; and verifying the data to be processed according to the data characteristic information.
Optionally, the processor may further execute the program code of the following steps: and returning feedback information to the management server, wherein the feedback information comprises: whether the process information was successfully stored and stored.
By adopting the embodiment of the application, a data processing scheme is provided. Receiving a data processing request from an Artificial Intelligence (AI) device, wherein the data processing request at least carries a device identifier of the AI device; determining a cloud end identification according to the equipment identification, and applying a storage space for the AI equipment to access to a data server by adopting the cloud end identification; and returning the address information of the storage space to the AI device, thereby achieving the purpose of realizing data interaction between the intelligent device and the data server, and further solving the technical problem that data interaction processing cannot be directly realized between the terminal device and the data server in the prior art.
It can be understood by those skilled in the art that the structure shown in fig. 3 is only an illustration, and the computer terminal may also be a terminal device such as a smart phone (e.g., an Android phone, an iOS phone, etc.), a tablet computer, a palmtop computer, a Mobile Internet Device (MID), a PAD, and the like. Fig. 3 is a diagram illustrating the structure of the electronic device. For example, the computer terminal 10 may also include more or fewer components (e.g., network interfaces, display devices, etc.) than shown in FIG. 3, or have a different configuration than shown in FIG. 3.
Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by a program instructing hardware associated with the terminal device, where the program may be stored in a computer-readable storage medium, and the storage medium may include: flash disks, Read-Only memories (ROMs), Random Access Memories (RAMs), magnetic or optical disks, and the like.
Example 9
Embodiments of the present application also provide an embodiment of a storage medium. Optionally, in this embodiment, the storage medium may be configured to store program codes executed by the data processing method provided in the foregoing embodiment.
Optionally, in this embodiment, the storage medium may be located in any one of computer terminals in a computer terminal group in a computer network, or in any one of mobile terminals in a mobile terminal group.
Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: receiving a data processing request from terminal equipment, wherein the data processing request is used for requesting to process data to be processed; responding to the data processing request, sending a space allocation request to a data server so that the data server allocates a storage space for the to-be-processed data based on the space allocation request; receiving a storage space address returned by the data server; and sending the storage space address to the terminal equipment so that the terminal equipment can send the data to be processed to the data server and store the data to the storage space.
Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: and sending the data characteristic information to the data server in response to the data processing request, so that the data server verifies the data to be processed according to the data characteristic information after the terminal equipment sends the data to be processed to the data server.
Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: receiving feedback information sent by the data server, wherein the feedback information includes: whether the process information was successfully stored and stored.
Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: receiving a registration request from the terminal device, wherein the registration request includes: a terminal device identifier; responding to the registration request, and distributing the data server to the terminal equipment; and storing the association relationship between the terminal equipment identification and the data server identification.
Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: receiving a registration request from the terminal device, wherein the registration request includes: a terminal device identifier and a data server identifier; and storing the association relationship between the terminal equipment identification and the data server identification.
Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: and determining the data server according to the association relation.
Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: sending a data processing request to a management server so that the management server requests to allocate a storage space to the data server, wherein the data processing request is used for requesting to process data to be processed, and the storage space is used for storing the data to be processed; receiving the storage space address returned by the data server and forwarded by the management server; and sending the data to be processed to the data server and storing the data to the storage space.
Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: sending a registration request to the management server so that the management server allocates the data server to the terminal device, wherein the registration request includes: and identifying the terminal equipment.
Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: and sending a registration request to the management server so that the management server can acquire a terminal device identifier and a data server identifier from the registration request and store the association relationship between the terminal device identifier and the data server identifier.
Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: receiving a space allocation request from a management server; responding to the space allocation request, and allocating a storage space for the data to be processed of the terminal equipment; returning a storage space address to the management server so that the management server can send the storage space address to the terminal equipment; and receiving the data to be processed sent by the terminal equipment, and storing the data to be processed into the storage space.
Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: receiving data characteristic information sent by the management server; and verifying the data to be processed according to the data characteristic information.
Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: and returning feedback information to the management server, wherein the feedback information comprises: whether the process information was successfully stored and stored.
The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments.
In the above embodiments of the present application, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one type of division of logical functions, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.
The foregoing is only a preferred embodiment of the present application and it should be noted that those skilled in the art can make several improvements and modifications without departing from the principle of the present application, and these improvements and modifications should also be considered as the protection scope of the present application.

Claims (20)

1. A method of data processing, comprising:
receiving a data processing request from terminal equipment, wherein the data processing request is used for requesting to process data to be processed;
responding to the data processing request, and sending a space allocation request to a data server so that the data server allocates a storage space for the data to be processed based on the space allocation request;
receiving a storage space address returned by the data server;
and sending the storage space address to the terminal equipment so that the terminal equipment can send the data to be processed to the data server and store the data to the storage space.
2. The method of claim 1, wherein the data processing request comprises: data characteristic information; the method further comprises the following steps:
and responding to the data processing request, and sending the data characteristic information to the data server so that the data server verifies the data to be processed according to the data characteristic information after the terminal equipment sends the data to be processed to the data server.
3. The method according to claim 1, wherein after the terminal device sends the data to be processed to the data server and stores the data to the storage space, the method further comprises:
receiving feedback information sent by the data server, wherein the feedback information comprises: whether the process information was successfully stored and stored.
4. The method of claim 1, wherein prior to receiving the data processing request from the terminal device, further comprising:
receiving a registration request from the terminal device, wherein the registration request comprises: a terminal device identifier;
responding to the registration request, and distributing the data server for the terminal equipment;
and storing the association relation between the terminal equipment identification and the data server identification.
5. The method of claim 1, wherein prior to receiving the data processing request from the terminal device, further comprising:
receiving a registration request from the terminal device, wherein the registration request comprises: a terminal device identifier and a data server identifier;
and storing the association relation between the terminal equipment identification and the data server identification.
6. The method of claim 4 or 5, wherein prior to sending the space allocation request to the data server, further comprising:
and determining the data server according to the incidence relation.
7. A method of data processing, comprising:
sending a data processing request to a management server so that the management server requests to allocate a storage space to the data server, wherein the data processing request is used for requesting to process data to be processed, and the storage space is used for storing the data to be processed;
receiving a storage space address returned by the data server and forwarded by the management server;
and sending the data to be processed to the data server and storing the data to be processed in the storage space.
8. The method of claim 7, wherein the data processing request comprises: and the data characteristic information is used for verifying the data to be processed according to the data characteristic information by the data server after the data to be processed is sent to the data server.
9. The method of claim 7, wherein prior to sending the data processing request to the management server, further comprising:
sending a registration request to the management server so that the management server allocates the data server to the terminal device, wherein the registration request includes: and identifying the terminal equipment.
10. The method of claim 7, wherein prior to sending the data processing request to the management server, further comprising:
and sending a registration request to the management server so that the management server can acquire the terminal equipment identifier and the data server identifier from the registration request and store the association relationship between the terminal equipment identifier and the data server identifier.
11. A method of data processing, comprising:
receiving a space allocation request from a management server;
responding to the space allocation request, and allocating a storage space for the data to be processed of the terminal equipment;
returning a storage space address to the management server so that the management server can send the storage space address to the terminal equipment;
and receiving the data to be processed sent by the terminal equipment, and storing the data to be processed to the storage space.
12. The method of claim 11, wherein the method further comprises:
receiving data characteristic information sent by the management server;
and verifying the data to be processed according to the data characteristic information.
13. The method of claim 12, wherein the to-be-processed data comprises user data and metadata associated with the user data, and the verifying the to-be-processed data according to the data characteristic information comprises:
judging whether the format of the metadata is consistent with the data characteristic information;
if so, the data to be processed is verified to be passed, and the data to be processed is stored in the storage space;
if not, the verification fails, and the to-be-processed data is refused to be stored.
14. The method of claim 11, wherein after receiving the data to be processed sent by the terminal device and storing the data to be processed in the storage space, the method further comprises:
returning feedback information to the management server, wherein the feedback information comprises: whether the process information was successfully stored and stored.
15. A data processing apparatus comprising:
the first receiving module is used for receiving a data processing request from the terminal equipment, wherein the data processing request is used for requesting to process data to be processed;
a first response module, configured to send a space allocation request to a data server in response to the data processing request, so that the data server allocates a storage space for the to-be-processed data based on the space allocation request;
the second receiving module is used for receiving the storage space address returned by the data server;
and the first sending module is used for sending the storage space address to the terminal equipment so that the terminal equipment can send the data to be processed to the data server and store the data to the storage space.
16. A data processing apparatus comprising:
the second sending module is used for sending a data processing request to the management server so that the management server can request the data server to allocate a storage space, wherein the data processing request is used for requesting to process data to be processed, and the storage space is used for storing the data to be processed;
a third receiving module, configured to receive the address of the storage space returned by the data server and forwarded by the management server;
and the third sending module is used for sending the data to be processed to the data server and storing the data to the storage space.
17. A data processing apparatus comprising:
the fourth receiving module is used for receiving the space allocation request from the management server;
the second response module is used for responding to the space allocation request and allocating storage space for the data to be processed of the terminal equipment;
the return module is used for returning a storage space address to the management server so that the management server can send the storage space address to the terminal equipment;
and the fifth receiving module is used for receiving the data to be processed sent by the terminal equipment and storing the data to be processed into the storage space.
18. A storage medium comprising a stored program, wherein the program, when executed, controls a device on which the storage medium is located to perform the data processing method of any one of claims 1-6, or performs the data processing method of any one of claims 7-10, or performs the data processing method of any one of claims 11-14.
19. A computing device, comprising: a memory storing program instructions and at least one processor, wherein the processor is configured to execute the program instructions, wherein the program instructions when executed cause the computing device to perform the data processing method of any of claims 1-6, or to perform the data processing method of any of claims 7-10, or to perform the data processing method of any of claims 11-14.
20. A data processing system comprising:
a management server and a data server, the management server being connected to the data server, wherein the management server is adapted to perform the data processing method of any of claims 1-6, and the data server is adapted to perform the data processing method of any of claims 11-14.
CN201810821368.2A 2018-07-24 2018-07-24 Data processing method, device and system Active CN110750206B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810821368.2A CN110750206B (en) 2018-07-24 2018-07-24 Data processing method, device and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810821368.2A CN110750206B (en) 2018-07-24 2018-07-24 Data processing method, device and system

Publications (2)

Publication Number Publication Date
CN110750206A true CN110750206A (en) 2020-02-04
CN110750206B CN110750206B (en) 2023-05-02

Family

ID=69275563

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810821368.2A Active CN110750206B (en) 2018-07-24 2018-07-24 Data processing method, device and system

Country Status (1)

Country Link
CN (1) CN110750206B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111930346A (en) * 2020-07-15 2020-11-13 北京百度网讯科技有限公司 Artificial intelligence information processing method and device, electronic equipment and storage medium
WO2021258370A1 (en) * 2020-06-24 2021-12-30 北京小米移动软件有限公司 Communication processing method, communication processing apparatus and storage medium
WO2023245512A1 (en) * 2022-06-22 2023-12-28 北京小米移动软件有限公司 Air interface processing method and apparatus, and storage medium

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101047610A (en) * 2007-04-30 2007-10-03 华为技术有限公司 Data storage, reading, transmission method and management server and network node
US20090157694A1 (en) * 2007-12-14 2009-06-18 Electronics And Telecommunications Research Institute Method and system for managing file metadata transparent about address changes of data servers and movements of their disks
CN102047642A (en) * 2008-06-02 2011-05-04 阿尔卡特朗讯 Method and device for storing online data
CN102255888A (en) * 2010-05-21 2011-11-23 英特尔公司 Method and apparatus for secure scan of Data storage device from remote server
CN105554431A (en) * 2016-02-17 2016-05-04 努比亚技术有限公司 Video call recording method, device and system
CN107181825A (en) * 2017-07-07 2017-09-19 广东中星电子有限公司 The on-line processing method of terminal device data
WO2017219852A1 (en) * 2016-06-24 2017-12-28 中兴通讯股份有限公司 Data information sharing method and apparatus, and terminal
CN107707644A (en) * 2017-09-27 2018-02-16 北京趣拿软件科技有限公司 Processing method, device, storage medium, processor and the terminal of request message
CN107749862A (en) * 2017-11-23 2018-03-02 爱国者安全科技(北京)有限公司 A kind of data encryption centrally stored method, server, user terminal and system
WO2018041078A1 (en) * 2016-08-29 2018-03-08 腾讯科技(深圳)有限公司 Method, system, proxy server, and computer storage medium for authentication

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101047610A (en) * 2007-04-30 2007-10-03 华为技术有限公司 Data storage, reading, transmission method and management server and network node
US20090157694A1 (en) * 2007-12-14 2009-06-18 Electronics And Telecommunications Research Institute Method and system for managing file metadata transparent about address changes of data servers and movements of their disks
CN102047642A (en) * 2008-06-02 2011-05-04 阿尔卡特朗讯 Method and device for storing online data
CN102255888A (en) * 2010-05-21 2011-11-23 英特尔公司 Method and apparatus for secure scan of Data storage device from remote server
CN105554431A (en) * 2016-02-17 2016-05-04 努比亚技术有限公司 Video call recording method, device and system
WO2017219852A1 (en) * 2016-06-24 2017-12-28 中兴通讯股份有限公司 Data information sharing method and apparatus, and terminal
WO2018041078A1 (en) * 2016-08-29 2018-03-08 腾讯科技(深圳)有限公司 Method, system, proxy server, and computer storage medium for authentication
CN107181825A (en) * 2017-07-07 2017-09-19 广东中星电子有限公司 The on-line processing method of terminal device data
CN107707644A (en) * 2017-09-27 2018-02-16 北京趣拿软件科技有限公司 Processing method, device, storage medium, processor and the terminal of request message
CN107749862A (en) * 2017-11-23 2018-03-02 爱国者安全科技(北京)有限公司 A kind of data encryption centrally stored method, server, user terminal and system

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
HAKAN FERHATOSMANOGLU等: "Data space mapping for efficient I/O in large multi-dimensional databases" *
张震伟;马雁云;: "基于超元数据缓存的高速数据访问系统" *
郑小燕;: "分布式文件系统中元数据管理策略" *
韦理;周悦芝;夏楠;: "用于网络存储系统的存储空间动态分配方法" *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021258370A1 (en) * 2020-06-24 2021-12-30 北京小米移动软件有限公司 Communication processing method, communication processing apparatus and storage medium
CN111930346A (en) * 2020-07-15 2020-11-13 北京百度网讯科技有限公司 Artificial intelligence information processing method and device, electronic equipment and storage medium
CN111930346B (en) * 2020-07-15 2023-04-25 北京百度网讯科技有限公司 Artificial intelligence information processing method and device, electronic equipment and storage medium
CN116679907A (en) * 2020-07-15 2023-09-01 北京百度网讯科技有限公司 Artificial intelligence information processing method and device, electronic equipment and storage medium
CN116679907B (en) * 2020-07-15 2024-02-27 北京百度网讯科技有限公司 Artificial intelligence information processing method and device, electronic equipment and storage medium
WO2023245512A1 (en) * 2022-06-22 2023-12-28 北京小米移动软件有限公司 Air interface processing method and apparatus, and storage medium

Also Published As

Publication number Publication date
CN110750206B (en) 2023-05-02

Similar Documents

Publication Publication Date Title
CN109068179B (en) Multi-platform live broadcast method, computer device and computer readable storage medium
CN110061871B (en) Base station opening method and device, computer storage medium and equipment
US20140355519A1 (en) Sharing wireless traffic
CN108881228A (en) Cloud registration activation method, device, equipment and storage medium
CN110750206B (en) Data processing method, device and system
CN110493028A (en) A kind of clustered deploy(ment) method, system, device and computer readable storage medium
CN109522198A (en) Processing method, device, electronic equipment and the readable storage medium storing program for executing of application program
CN106549989A (en) A kind of data transmission method and its system, user terminal, application server
CN105592019A (en) Method for bidirectional access to application between dual execution environments
CN104901937A (en) Data processing method and system thereof, terminal, server
US20200259900A1 (en) Data transmission method, server, storage system, terminal device, and system
KR20210043529A (en) System for authenticating image based on blockchain and hash encryption technique and method thereof
CN106302123A (en) The extraction of a kind of data, the method for gluing treatment, device and system thereof
US20170171339A1 (en) Advertisement data transmission method, electrnoic device and system
CN107231275B (en) Method for connection configuration of user equipment and household equipment
CN114389868B (en) Cloud resource allocation method, system, device and storage medium
CN115016779A (en) Audio and video function processing method and device, storage medium and processor
CN110557750B (en) Information transmission method, gateway device, core network device, and storage medium
RU2666283C1 (en) Method for distribution of advertising content through technologies of nfc-points and qr-code
US20140315600A1 (en) Communication interface method for se equipped on mobile terminal and se using the same
CN109361730A (en) Card information acquisition methods, system and Cloud Server based on face
CN108990039A (en) Data transmission method and relevant apparatus
CN109962962A (en) A kind of method and apparatus of socket connection
CN109361729A (en) Card information acquisition methods, system and Cloud Server based on fingerprint
CN102801813B (en) A kind of device and method automatically uploading audio/video file from terminal equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant