WO2018095037A1

WO2018095037A1 - Method and device for obtaining data in cloud storage system

Info

Publication number: WO2018095037A1
Application number: PCT/CN2017/091480
Authority: WO
Inventors: 浦世亮; 刘锋; 许爱秋
Original assignee: 杭州海康威视数字技术股份有限公司
Priority date: 2016-11-24
Filing date: 2017-07-03
Publication date: 2018-05-31
Also published as: CN108111557A; CN108111557B

Abstract

Embodiments of the present application provide a method and a device for obtaining data in a cloud storage system. The method comprises: receiving condition information of target data to be analyzed transmitted by a big data analysis server; in model data in all pre-stored data, obtaining target model information matching the condition information; and transmitting a target URL in target model data comprising the target model information to the big data analysis server, such that the big data analysis server obtains data corresponding to the target URL from a cloud storage system. By using the embodiments of the present application, target data stored in a cloud storage system is pre-analyzed, such that when a big data analysis server analyzes the target data, it is not necessary to extract all possible target data in the cloud storage system, reducing hardware requirements for the big data analysis server and improving data processing efficiency of the big data analysis server.

Description

Method and device for acquiring data in cloud storage system

The present application claims priority to Chinese Patent Application No. 201611053415.0 entitled "A Method and Apparatus for Acquiring Data in a Cloud Storage System" on November 24, 2016, the entire contents of which are incorporated by reference. In this application.

Technical field

The present application relates to the field of cloud storage technologies, and in particular, to a method and apparatus for acquiring data in a cloud storage system.

Background technique

Big data analysis is often associated with cloud computing, through the real-time data analysis, to complete the filtering of massive data. The role and value of big data analysis in various industries has become more and more important and more and more obvious.

A cloud storage system is a distributed storage system that connects different types of storage devices to each other to realize unified management of massive data. At present, a system combining cloud storage and big data processing is shown in FIG. 1 , and includes: a cloud storage system 110 and a big data analysis server 120 . The cloud storage system 110 includes a management server 111 and a storage server 112. Specifically, the management server 111 is responsible for managing a plurality of storage servers 112 for providing external data storage and service access. The big data analysis server 120 is configured to obtain data from the cloud storage system 110 and analyze the obtained data. deal with.

In the prior art, the cloud storage system stores massive data in the storage server. When the big data analysis server analyzes the target data (picture data, document data, etc.), it needs to read all possible data from the storage server of the cloud storage system. And then analyze all possible data to get the data the user needs. Especially in the field of video surveillance, a large amount of image data obtained from an image acquisition device of a video surveillance front end is stored in a storage server; when a user needs to obtain target image data, the big data analysis server reads all from the storage server. The possible image data is analyzed, and finally the target image data required by the user is obtained.

It can be seen that the prior art big data analysis server needs to read all possible data from the storage server when acquiring the data required by the user, and analyze all the possible data to obtain the data required by the user. This way of obtaining data consumes a lot of time and processing efficiency. Very low.

Summary of the invention

The purpose of the embodiments of the present application is to provide a method and an apparatus for acquiring data in a cloud storage system, so as to quickly and accurately obtain data required by a user from a cloud storage system. The specific technical solutions are as follows:

In a first aspect, the embodiment of the present application discloses a method for acquiring data in a cloud storage system, which is applied to a cloud storage system, and includes:

Receiving condition information of the target data to be analyzed sent by the big data analysis server;

In the model data of all the pre-stored data, target model information matching the condition information is obtained; wherein each data corresponds to one model data, and each model data includes a uniform resource locator URL and model information of the corresponding data. Each model information is determined by the cloud storage system according to the key information of the data when storing the corresponding data;

Sending a target URL in the target model data including the target model information to the big data analysis server, so that the big data analysis server acquires data corresponding to the target URL from the cloud storage system.

Optionally, the target data is picture data; the condition information includes: at least one of an acquisition time, a collection location, and an object attribute;

The model information includes: an acquisition time, an acquisition location, and an object attribute.

Optionally, the target data is text data; the condition information includes: at least one of a storage time, a text data type, and a size of the storage space;

The model information includes: storage time, text data type, and size of the storage space.

Optionally, before the receiving the condition information of the target data to be analyzed sent by the big data analysis server, the method further includes:

Storing target data, and generating a URL of the target data according to a storage location of the storage target data;

Decoding key information from the target data;

Obtaining target model information of the target data according to the key information, and constructing model data corresponding to the target data, where the model data corresponding to the target data includes a URL and a location of the target data. Describe the target model information.

Optionally, the big data analysis server obtains data corresponding to the target URL from the cloud storage system, including:

Determining a current data analysis rate of the big data analysis server;

Obtaining data corresponding to the target URL from the cloud storage system according to the determined current data analysis rate.

Optionally, the obtaining the data corresponding to the target URL from the cloud storage system according to the determined current data analysis rate includes:

Determining whether the determined current data analysis rate is less than a first threshold;

If the determination result is yes, the data corresponding to the target URL is suspended from the cloud storage system;

If the determination result is no, the data corresponding to the target URL is requested to be obtained from the cloud storage system.

Determining whether the determined current data analysis rate is less than a second threshold;

If the determination result is yes, the data corresponding to the target URL is extracted from the cloud storage system at a first rate;

If the determination result is no, the data corresponding to the target URL is requested to be obtained from the cloud storage system at a second rate;

Wherein the first rate is less than the second rate.

In a second aspect, the embodiment of the present application discloses an apparatus for acquiring data in a cloud storage system, which is applied to a cloud storage system, and includes:

a receiving unit, configured to receive a condition letter of the target data to be analyzed sent by the big data analysis server interest;

An obtaining unit, configured to obtain target model information that matches the condition information in model data of all pre-stored data; wherein each data corresponds to one model data, and each model data includes uniform resource positioning of corresponding data URL and model information, each model information is determined by the cloud storage system when storing the corresponding data according to the key information of the data;

a sending unit, configured to send a target URL in the target model data including the target model information to the big data analysis server, so that an acquiring unit in the big data analyzing server acquires the target URL from a cloud storage system Corresponding data.

Optionally, the device further includes:

a generating unit, configured to store target data, and generate a target URL of the target data according to the storage location of the storage target data;

a decoding unit, configured to decode key information from the target data;

a processing unit, configured to obtain, by using a preset algorithm, target model information of the target data according to the key information; and construct model data corresponding to the target data, where the model data corresponding to the target data includes the target The URL of the data and the target model information.

Optionally, the acquiring unit in the big data analysis server includes:

Determining a subunit for determining a current data analysis rate of the big data analysis server;

The sub-unit is configured to obtain data corresponding to the target URL from the cloud storage system according to the determined current data analysis rate.

Optionally, the searching subunit is specifically configured to:

Wherein the first rate is less than the second rate.

In a third aspect, the application provides an electronic device, including:

a processor, a memory, a communication interface, and a bus;

The processor, the memory, and the communication interface are connected by the bus and complete communication with each other;

The memory stores executable program code;

The processor runs a program corresponding to the executable program code by reading executable program code stored in the memory for performing a method of acquiring cloud storage according to the first aspect of the present application at runtime The method of data in the system.

In a fourth aspect, the present application provides a storage medium, where the storage medium is configured to store executable program code, and the executable program code is configured to execute a acquiring cloud according to the first aspect of the present application at runtime A method of storing data in a system.

In a fifth aspect, the application provides an application, wherein the application is configured to perform a method for acquiring data in a cloud storage system according to the first aspect of the present application at runtime.

A method and device for acquiring data in a cloud storage system provided by an embodiment of the present application, the method includes: receiving condition information of a target data to be analyzed sent by a big data analysis server; obtaining, in a model data of all pre-stored data And the target model information matched by the condition information; sending a target URL in the target model data including the target model information to the big data analysis server, so that the big data analysis server acquires the macro data analysis system from the cloud storage system The data corresponding to the target URL.

In the embodiment of the present application, all the data stored in the cloud storage system is analyzed and processed in advance, and when the big data analysis server acquires the target data required by the user, it is not necessary to extract all possible data in the cloud storage system, and only needs to be based on The condition information input by the user extracts target data corresponding to the target model information that matches the condition information. The method for obtaining data in the cloud storage system quickly and accurately acquires data required by the user, reduces hardware requirements for the big data analysis server, and improves the efficiency of processing the data by the big data analysis server.

DRAWINGS

In order to more clearly illustrate the embodiments of the present application and the technical solutions of the prior art, the following description of the embodiments and the drawings used in the prior art will be briefly introduced. Obviously, the drawings in the following description are only Some embodiments of the application may also be used to obtain other figures from those of ordinary skill in the art without departing from the scope of the invention.

1 is a schematic structural diagram of a system combining cloud storage and big data processing provided by the prior art;

2 is a system structural diagram of acquiring picture data in a cloud storage system according to an embodiment of the present application;

FIG. 3 is a schematic flowchart of a method for acquiring data in a cloud storage system according to an embodiment of the present disclosure;

FIG. 4 is a schematic structural diagram of an apparatus for acquiring data in a cloud storage system according to an embodiment of the present disclosure;

FIG. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

detailed description

The technical solutions in the embodiments of the present application are clearly and completely described in the following with reference to the drawings in the embodiments of the present application. It is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present application without departing from the inventive scope are the scope of the present application.

The embodiment of the present application provides a method and device for acquiring data in a cloud storage system, so as to quickly and accurately obtain data required by a user from a cloud storage system.

A method for obtaining data in a cloud storage system provided by an embodiment of the present application is first introduced.

It should be noted that the execution entity of the method for obtaining data in the cloud storage system provided by the embodiment of the present application may be an apparatus for acquiring data in the cloud storage system. Specifically, the device for acquiring data in the cloud storage system may be function software in a device for processing a big data analysis server request in the cloud storage system, where the device for processing the big data analysis server request, such as the cloud storage A management server or storage server in the system.

It can be understood that various types of data are stored in the cloud storage system, including: picture data or text data, and of course, it is not limited thereto. The image data is usually generated by the front camera or the capture machine, or uploaded by the user to the cloud storage system through electronic devices (for example, computers, mobile phones, devices with storage functions, etc.); the text data may be uploaded to the cloud by the user through the electronic device. Storage system. For example, FIG. 2 is a schematic structural diagram of a system for acquiring picture data in a cloud storage system according to an embodiment of the present disclosure. As shown in FIG. 2, the system may include: a front-end camera (or capture machine) 210, a cloud storage system 220, and The big data analysis server 230; specifically, the front camera 210 is used to generate picture data, the cloud storage system 220 is used to process and store picture data, and the big data analysis server 230 is used to obtain picture data from the cloud storage system 220 and obtain the picture data. The image data is analyzed and processed.

Referring to FIG. 3, a method for obtaining data in a cloud storage system provided by an embodiment of the present application may include the following steps:

S301. Receive condition information of the target data to be analyzed sent by the big data analysis server.

The target data may be picture data or text data, and is of course not limited thereto. Generally, the user sets condition information according to the target data required by the user, and uploads the set condition information to the big data analysis server, so that the big data analysis server obtains the cloud data system according to the set condition information. Required target data. When the condition information set by the user is different, the target data obtained from the cloud storage system is also different.

It should be emphasized that the condition information of the target data to be analyzed sent by the big data analysis server is the same as the condition information sent by the big data analysis server in the prior art. For example, when the target data is picture data, the condition information may be collected. At least one of the time, the collection location, and the object attribute is of course not limited thereto; when the target data is text data, the condition information may be at least one of storage time, text data type, and storage space size. Of course, it is not limited to this.

S302: Obtain target model information that matches the condition information in the model data of all the pre-stored data; wherein each data corresponds to one model data, and each model data includes a uniform resource locator URL of the corresponding data and Model information, each model information is determined by the cloud storage system according to the key information of the data when storing the corresponding data;

After receiving the condition information of the target data to be analyzed sent by the big data analysis server, the device that obtains the data in the cloud storage system does not directly feed back all the possible large amounts of data to the big data analysis server, but all of the pre-stored data. In the model data of the data, the target model information matching the condition information is obtained, and then a small amount of accurate data is fed back to the big data analysis server based on the target model information.

It should be noted that before receiving the condition information sent by the big data analysis server, the model data of all the data is pre-stored in the cloud storage system. Here, each data stored in the cloud storage system corresponds to one model data. Specifically, in a specific implementation manner, in the cloud storage system, the process of generating model data corresponding to each data may include: the cloud storage system The key information of each stored data is obtained by using a preset algorithm to obtain model information of each data, and the corresponding model data is constructed by using the model information. Here, the model data of each data includes a uniform resource locator URL and model information of the data, wherein the uniform resource locator URL of the data indicates a storage location of the data in the cloud storage system, and the model information is when the data is stored. , the key information of the data is determined.

For example, when the target data is picture data, in the cloud storage system, the picture key information of the picture data is decoded, and the picture key information is obtained by using a preset algorithm to obtain picture model information of the picture data, and the picture model information and The URL constructs the picture model data corresponding to the picture data. Among them, the key information of the picture is the main information of the front-end device (front-end camera, capture machine and other devices with coding function) when generating picture data, and the key information of the picture can be input through the screening algorithm. Row filtering, the filtered information is used as image model information. For example, when the front-end device generates image data, the acquisition time, the collection location, and the object attributes and other related image information are encoded into the image data, and the cloud storage system is from the image data. The key information of the picture is decoded, and the key information of the picture is filtered, and the image model information such as the collection time, the collection location, and the object attribute can be obtained. It should be noted that the collection time refers to the time when the front-end device collects the image data, and the collection location refers to the location of the image data collected by the front-end device. The corresponding attribute refers to the format, size, resolution, and total pixels of the image data.

It should be emphasized that the key information is various attribute information of the data, and the model information is information determined based on the key information and convenient for subsequent query (ie, capable of matching analysis with the condition information), and therefore, when determining the model information, The determination can be made based on the condition information that can be obtained.

S303. Send a target URL in the target model data that includes the target model information to the big data analysis server, so that the big data analysis server acquires data corresponding to the target URL from the cloud storage system.

When the target model information matching the condition information is obtained, the target URL is searched in the target model data including the target model information, and the obtained target URL is sent to the big data analysis server, so that the big data analysis server is configured according to The target URL extracts data corresponding to the target URL from the cloud storage system. For example, when the target data is image data, specifically, when the cloud storage system sends the image URL of the image data to the big data analysis server, the big data analysis server may extract the image URL corresponding to the image from the cloud storage system according to the image URL. Picture data.

It should be noted that the target data in the embodiment of the present application may be picture data or text data. When the user needs to obtain the image data in the cloud storage system, the target data is the image data; when the user needs to obtain the text data in the cloud storage system, the target data is the text data. Applying the foregoing embodiment, receiving condition information of the target data to be analyzed sent by the big data analysis server, obtaining target model information matching the condition information in the model data of all pre-stored data, and transmitting the target information The target URL in the target model data of the model information is sent to the big data analysis server, so that the big data analysis server acquires data corresponding to the target URL from the cloud storage system. It can be seen that, in the embodiment of the present application, all data stored in the cloud storage system is analyzed and processed in advance, and when the big data analysis server acquires the target data required by the user, it is not required to extract all possible data in the cloud storage system. The embodiment reduces the score on big data The hardware requirements of the server are analyzed, and the efficiency of processing data by the big data analysis server is improved, and the data required by the user is quickly and accurately obtained from the cloud storage system.

In a possible implementation manner of the embodiment of the present application, when the target data is picture data, the condition information includes at least one of an acquisition time, a collection location, and an object attribute, that is, the condition information may be collection. One type or condition information in time, collection location, and object attributes may include at least two types of acquisition time, collection location, and object attributes. The big data analytics server can upload one or at least two of the condition information to the cloud storage system. The acquisition time here is the time when the front camera or the capture machine generates image data, or the collection time is the time when the user uploads the image data in the cloud storage system. Here, the collection location can be understood as the location where the front camera is located, that is, the location where the image data is generated, or the location where the user is located when the user uploads the image data in the cloud storage system. Object attributes include: vehicle type, vehicle color, license plate number, speed of movement, number of vehicles, and the like. In addition, when the target data is picture data, each picture data corresponds to one picture model data, and each picture model data includes a uniform resource locator and picture model information of the picture data, where the picture model information of the picture data includes the collection. Time, collection location, and object properties.

In a possible implementation manner of the embodiment of the present application, when the target data is text data, the condition information includes at least one of a storage time, a text data type, and a storage space size. That is, the condition information may be one of storage time, text data type, and size of the occupied storage space, or the condition information may include at least two of storage time, text data type, and size of the occupied storage space. The big data analytics server can upload one or at least two of storage time, text data type, and storage size to the cloud storage system. Here, the storage time is the time for storing the data text in the cloud storage system, and the text data types include: txt, doc, docx, wps, and the like. In addition, when the target data is text data, each text data corresponds to a text model data, and each text model data includes a uniform resource locator of the text data and text model information of the text data, where the text model of the text data Information includes: storage time, text data type, and size of storage space. It should be noted that the model information of the text data includes condition information.

In a possible implementation manner of the embodiment of the present application, before receiving the condition information of the target data to be analyzed sent by the big data analysis server, the method further includes: storing the target data, according to the storage location of the storage target data. , generating a URL of the target data; from the number of targets Decoding the key information according to the key information, obtaining the target model information of the target data by using a preset algorithm; and constructing model data corresponding to the target data, wherein the model data corresponding to the target data includes The URL of the target data and the target model information.

Specifically, the cloud storage system stores the target data generated by the device such as the front camera or the target data uploaded by the user to a preset storage location of the cloud storage system, and generates a URL of the target data according to the storage location of the target data. For example, the cloud storage system stores the image generated by the device such as the front camera or the user uploaded image data to a preset storage location, and generates a URL of the image data according to the storage location of the image data.

It should be noted that the target data includes key information of the target data, and when the target data is processed, the key information of the target data is decoded. The key information is that when the target data is generated, the front-end device encodes the key information of the target data into the target data. For example, when the current end device generates the image data, the key information of the number of cars, the model of the car, the license plate, and the like in the image data is encoded. Go to the image data.

After the target data is stored in the cloud storage system, the key information in the target data is decoded; then, the target model information is obtained through the preset algorithm for the key information, and the target model data corresponding to the target data is constructed from the target model information. At the same time, the target model data includes the URL of the target data and the target model information. That is, the target model data is constructed by the target model information, and the URL of the target data is added to the target model data to obtain the final target model data. When the condition information is obtained, the target model information matching the condition information may be searched in the target model data, and when the target model information matching the target model information exists in the model data, the target URL in the model data is obtained, The target URL is sent to the big data analysis server, and the big data analysis server extracts data corresponding to the target URL from the cloud storage system according to the received target URL. For example, after the picture data is stored in the cloud storage system, the key information in the picture data is decoded, and the key information of the picture data is analyzed and processed by using a preset algorithm to obtain picture model information of the picture data, and the picture is constructed from the picture model information. The model data corresponding to the data, and the URL of the image data is added to the image model data, such that the image model data includes the URL of the image model information and the image data.

In a possible implementation manner of the embodiment of the present application, the big data analysis server acquires data corresponding to the target URL from the cloud storage system, including: determining the big data analysis service. Current data analysis rate of the device; extracting data corresponding to the target URL from the cloud storage system according to the determined current data analysis rate.

Specifically, when the big data analysis server obtains the data corresponding to the target URL from the cloud storage system, the data analysis rate of the big data analysis server is considered. For example, when the current data analysis rate of the big data analysis server is fast, the data corresponding to the target URL is obtained from the cloud storage system, and when the current data analysis rate of the big data analysis server is slow, the stop from the cloud storage system is stopped. Get the data corresponding to the target URL. In this way, the pressure of analyzing data of the big data analysis server is reduced, and the load of the big data analysis server is effectively balanced.

Specifically, the obtaining, according to the determined current data analysis rate, the data corresponding to the target URL from the cloud storage system, comprising: determining whether the determined current data analysis rate is less than a first threshold; If the determination result is yes, the data corresponding to the target URL is extracted from the cloud storage system; if the determination result is no, the data corresponding to the target URL is requested to be obtained from the cloud storage system.

Specifically, by comparing the current data analysis rate of the big data analysis server with the first threshold, according to the comparison result, determining to extract data from the cloud storage system or suspending data extraction from the cloud storage system. Preferably, when the current data analysis rate of the big data analysis server is less than the first threshold, the data corresponding to the target URL is extracted from the cloud storage system, and when the current data analysis rate of the big data analysis server is detected to be greater than the first threshold, The data corresponding to the target URL is requested to be obtained from the cloud storage system. Here, the big data analysis server detects the data analysis rate of the big data analysis server in a preset period, determines the data corresponding to the target URL from the cloud storage system, or pauses the data according to the current data analysis rate of the detected big data analysis server. The cloud storage system extracts data corresponding to the target URL.

Optionally, the obtaining, according to the determined current data analysis rate, the data corresponding to the target URL from the cloud storage system, comprising: determining whether the determined current data analysis rate is less than a second threshold. If the result of the determination is yes, the data corresponding to the target URL is extracted from the cloud storage system at a first rate; if the determination result is no, the target URL is requested to be obtained from the cloud storage system at a second rate. Corresponding data; wherein the first rate is less than the second rate.

For example, when the current data analysis rate of the big data analysis server is less than the second threshold, The first rate is used to slowly extract the data corresponding to the target URL from the cloud storage system, or the current data analysis rate of the big data analysis server is greater than the second threshold, and request the access to the cloud storage system at the preset second rate. The data corresponding to the target URL, that is, the data corresponding to the target URL is quickly extracted from the cloud storage system.

Corresponding to the foregoing method embodiments, the embodiment of the present application further provides an apparatus for acquiring data in a cloud storage system. As shown in FIG. 4, an apparatus for acquiring data in a cloud storage system according to an embodiment of the present application may include:

The receiving unit 410 is configured to receive condition information of the target data to be analyzed sent by the big data analysis server.

The obtaining unit 420 is configured to obtain, in the model data of all the pre-stored data, target model information that matches the condition information, where each data corresponds to one model data, and each model data includes a unified resource of the corresponding data. a locator URL and model information, each model information is determined by the cloud storage system according to the key information of the data when storing the corresponding data;

a sending unit 430, configured to send a target URL in the target model data including the target model information to the big data analysis server, so that an acquiring unit in the big data analyzing server acquires the target from a cloud storage system The data corresponding to the URL.

In the embodiment of the present application, the condition information of the target data to be analyzed sent by the big data analysis server is received, and the target model information matching the condition information is obtained in the model data of all the pre-stored data, and the sending includes the The target URL in the target model data of the target model information is sent to the big data analysis server, so that the big data analysis server acquires data corresponding to the target URL from the cloud storage system. It can be seen that, in the embodiment of the present application, all data stored in the cloud storage system is analyzed and processed in advance, and when the big data analysis server acquires the target data required by the user, it is not required to extract all possible data in the cloud storage system. The embodiment reduces the hardware requirements of the big data analysis server, improves the efficiency of processing data by the big data analysis server, and realizes the fast and accurate acquisition of data required by the user from the cloud storage system.

Specifically, in an implementation manner, the target data is picture data; the condition information includes: at least one of an acquisition time, a collection location, and an object attribute;

Specifically, in another implementation manner, the target data is text data; and the condition information includes: at least one of a storage time, a text data type, and a storage space size;

The model information includes: storage time, text data type, and size of the storage space. In a possible implementation manner of the embodiment of the present application, the device further includes:

a decoding unit, configured to decode key information from the target data;

In order to reduce the pressure of the analysis data of the big data analysis server, the load of the big data analysis server is effectively balanced. Optionally, in an implementation manner, the acquiring unit in the big data analysis server may include:

Optionally, the searching subunit is specifically configured to:

Wherein the first rate is less than the second rate.

Correspondingly, as shown in FIG. 5, the embodiment of the present application further provides an electronic device, which may include:

a processor 510, a memory 520, a communication interface 530, and a bus 540;

The processor 510, the memory 520, and the communication interface 530 are connected by the bus 540 and complete communication with each other;

The memory 520 stores executable program code;

The processor 510 runs a program corresponding to the executable program code by reading the executable program code stored in the memory 520, and is configured to execute an acquisition cloud according to an embodiment of the present application at runtime. A method for storing data in a system, wherein the method for obtaining data in a cloud storage system includes:

In the embodiment of the present application, all the data stored in the cloud storage system is analyzed and processed in advance, and when the big data analysis server acquires the target data required by the user, it is not necessary to extract all possible data in the cloud storage system, and only needs to be based on The condition information input by the user extracts target data corresponding to the target model information that matches the condition information. The method for obtaining data in the cloud storage system quickly and accurately obtains data required by the user, and reduces hardware requirements for the big data analysis server. Increased the efficiency of big data analytics servers in processing data.

Correspondingly, the embodiment of the present application further provides a storage medium, where the storage medium is used to store executable program code, and the executable program code is configured to perform an acquisition according to an embodiment of the present application at runtime. A method for collecting data in a cloud storage system, wherein the method for obtaining data in a cloud storage system includes:

Correspondingly, the embodiment of the present application further provides an application, where the application is used to execute a method for acquiring data in a cloud storage system according to an embodiment of the present application, where the acquiring cloud The parties to the data in the storage system include:

For the device/electronic device/storage medium/application embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment.

It should be noted that, in this context, relational terms such as first and second are used merely to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply such entities or operations. There is any such actual relationship or order between them. Furthermore, the term "comprises" or "comprises" or "comprises" or any other variations thereof is intended to encompass a non-exclusive inclusion, such that a process, method, article, or device that comprises a plurality of elements includes not only those elements but also Other elements, or elements that are inherent to such a process, method, item, or device. An element that is defined by the phrase "comprising a ..." does not exclude the presence of additional equivalent elements in the process, method, item, or device that comprises the element.

The various embodiments in the present specification are described in a related manner, and the same or similar parts between the various embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment.

The above is only the preferred embodiment of the present application, and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc., which are made within the spirit and principles of the present application, should be included in the present application. Within the scope of protection.

Claims

A method for obtaining data in a cloud storage system, which is characterized by being applied to a cloud storage system, including:

Receiving condition information of the target data to be analyzed sent by the big data analysis server;

In the model data of all the pre-stored data, target model information matching the condition information is obtained; wherein each data corresponds to one model data, and each model data includes a uniform resource locator URL and model information of the corresponding data. Each model information is determined by the cloud storage system according to the key information of the data when storing the corresponding data;

Sending a target URL in the target model data including the target model information to the big data analysis server, so that the big data analysis server acquires data corresponding to the target URL from the cloud storage system.
The method according to claim 1, wherein the target data is picture data; the condition information comprises: at least one of an acquisition time, a collection location, and an object attribute;

The model information includes: an acquisition time, an acquisition location, and an object attribute.
The method according to claim 1, wherein the target data is text data; the condition information comprises: at least one of a storage time, a text data type, and a size of the storage space;

The model information includes: storage time, text data type, and size of the storage space.
The method according to any one of claims 1-3, wherein before the receiving the condition information of the target data to be analyzed sent by the big data analysis server, the method further comprises:

Storing target data, and generating a URL of the target data according to a storage location of the storage target data;

Decoding key information from the target data;

Obtaining target model information of the target data according to the key information, and constructing model data corresponding to the target data, where the model data corresponding to the target data includes a URL and a location of the target data. Describe the target model information.
The method according to any one of claims 1-3, wherein the big data analysis server acquires data corresponding to the target URL from the cloud storage system, including:

Determining a current data analysis rate of the big data analysis server;

Obtaining data corresponding to the target URL from the cloud storage system according to the determined current data analysis rate.
The method according to claim 5, wherein the obtaining the data corresponding to the target URL from the cloud storage system according to the determined current data analysis rate comprises:

Determining whether the determined current data analysis rate is less than a first threshold;

If the determination result is yes, the data corresponding to the target URL is suspended from the cloud storage system;

If the determination result is no, the data corresponding to the target URL is requested to be obtained from the cloud storage system.
The method according to claim 5, wherein the obtaining the data corresponding to the target URL from the cloud storage system according to the determined current data analysis rate comprises:

Determining whether the determined current data analysis rate is less than a second threshold;

If the determination result is yes, the data corresponding to the target URL is extracted from the cloud storage system at a first rate;

If the determination result is no, the data corresponding to the target URL is requested to be obtained from the cloud storage system at a second rate;

Wherein the first rate is less than the second rate.
An apparatus for obtaining data in a cloud storage system, which is characterized by being applied to a cloud storage system, including:

a receiving unit, configured to receive condition information of the target data to be analyzed sent by the big data analysis server;

An obtaining unit, configured to obtain target model information that matches the condition information in model data of all pre-stored data; wherein each data corresponds to one model data, and each model data The uniform resource locator URL and the model information of the corresponding data are included, and each model information is determined by the cloud storage system according to the key information of the data when storing the corresponding data;

a sending unit, configured to send a target URL in the target model data including the target model information to the big data analysis server, so that an acquiring unit in the big data analyzing server acquires the target URL from a cloud storage system Corresponding data.
The apparatus according to claim 8, wherein the target data is picture data; the condition information comprises: at least one of an acquisition time, a collection location, and an object attribute;

The model information includes: an acquisition time, an acquisition location, and an object attribute.
The apparatus according to claim 8, wherein the target data is text data; and the condition information comprises: at least one of a storage time, a text data type, and a size of the storage space;

The model information includes: storage time, text data type, and size of the storage space.
The device according to any one of claims 8 to 10, wherein the device further comprises:

a generating unit, configured to store target data, and generate a target URL of the target data according to the storage location of the storage target data;

a decoding unit, configured to decode key information from the target data;

a processing unit, configured to obtain, by using a preset algorithm, target model information of the target data according to the key information; and construct model data corresponding to the target data, where the model data corresponding to the target data includes the target The URL of the data and the target model information.
The device according to claim 8, wherein the acquiring unit in the big data analysis server comprises:

Determining a subunit for determining a current data analysis rate of the big data analysis server;

The sub-unit is configured to obtain data corresponding to the target URL from the cloud storage system according to the determined current data analysis rate.
The apparatus according to claim 12, wherein said lookup subunit is specifically configured to:

Determining whether the determined current data analysis rate is less than a first threshold;

If the determination result is yes, the data corresponding to the target URL is suspended from the cloud storage system;

If the determination result is no, the data corresponding to the target URL is requested to be obtained from the cloud storage system.
The apparatus according to claim 12, wherein said lookup subunit is specifically configured to:

Determining whether the determined current data analysis rate is less than a second threshold;

If the determination result is yes, the data corresponding to the target URL is extracted from the cloud storage system at a first rate;

If the determination result is no, the data corresponding to the target URL is requested to be obtained from the cloud storage system at a second rate;

Wherein the first rate is less than the second rate.
An electronic device, comprising:

a processor, a memory, a communication interface, and a bus;

The processor, the memory, and the communication interface are connected by the bus and complete communication with each other;

The memory stores executable program code;

The processor executes a program corresponding to the executable program code by reading executable program code stored in the memory for performing an acquisition cloud according to any one of claims 1-7 A method of storing data in a system.
A storage medium, characterized in that the storage medium is for storing executable program code for performing an acquisition cloud storage according to any one of claims 1-7 at runtime The method of data in the system.
An application, wherein the application is configured to perform a method of acquiring data in a cloud storage system according to any one of claims 1-7 at runtime.