CN111143417A - Data processing method, device and system, Nginx server and medium - Google Patents
Data processing method, device and system, Nginx server and medium Download PDFInfo
- Publication number
- CN111143417A CN111143417A CN201911382325.XA CN201911382325A CN111143417A CN 111143417 A CN111143417 A CN 111143417A CN 201911382325 A CN201911382325 A CN 201911382325A CN 111143417 A CN111143417 A CN 111143417A
- Authority
- CN
- China
- Prior art keywords
- data
- accessed
- local cache
- mapping relation
- nginx server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 27
- 238000003860 storage Methods 0.000 claims abstract description 127
- 238000013507 mapping Methods 0.000 claims abstract description 67
- 238000000034 method Methods 0.000 claims abstract description 26
- 238000004891 communication Methods 0.000 claims abstract description 18
- 238000012545 processing Methods 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 13
- 230000014759 maintenance of location Effects 0.000 claims description 6
- 238000005516 engineering process Methods 0.000 description 10
- 230000008569 process Effects 0.000 description 7
- 230000000694 effects Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 238000007405 data analysis Methods 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 239000003795 chemical substances by application Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/256—Integrating or interfacing systems involving database management systems in federated or virtual databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/258—Data format conversion from or to a database
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The application discloses a data processing method, device and system, a Nginx server and a medium, wherein the method is applied to the Nginx server, the Nginx server acquires data from a cloud storage through a preset interface and caches the data to a local cache, and the method comprises the following steps: receiving a data access request sent by a client; calling a first mapping relation established in advance; the mapping relation is between object storage data and local cache data; judging whether the data to be accessed of the data access request exists in a local cache or not according to the first relation; if yes, returning the data to be accessed to the terminal through the object storage format. According to the application, the Nginx server can read and cache the data stored in the cloud to the local, and can directly obtain data from the local cache to return the data for the data access request sent by the subsequent client, so that the data delay of communication from the remote end is saved, and the online video playing is more smooth.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a data processing method, apparatus, and system, an Nginx server, and a computer-readable storage medium.
Background
The cloud storage refers to a system which integrates a large number of different types of storage devices in a network through application software to cooperatively work through functions such as cluster application, a grid technology or a distributed file system, and provides data storage and service access functions to the outside, so that the safety of data is ensured, and the storage space is saved. Briefly, cloud storage is an emerging solution for putting storage resources on the cloud for human access. The user can conveniently access data at any time and any place through connecting to the cloud through any internet-connected device.
With the rapid development of cloud technology, more and more individuals or enterprises choose to put data in the cloud. Therefore, the method can not only be independent of the local storage environment, but also be used as a good backup carrier. However, since the data interaction at the cloud is accessed through the Internet, a relatively large data delay is inevitably caused. The problem of how to solve the high delay loss caused by Internet connection is also considered while bringing convenience.
Disclosure of Invention
The application aims to provide a data processing method, a device and a system, an Nginx server and a computer readable storage medium, which save data delay of communication from a far end and enable online video playing to have smoother effect.
In order to achieve the above object, the present application provides a data processing method applied to a Nginx server, where the Nginx server obtains data from a cloud storage through a preset interface and caches the data to a local cache, and the method includes:
receiving a data access request sent by a client;
calling a first mapping relation established in advance; the first mapping relation is a mapping relation between object storage data and local cache data;
judging whether the local cache has data to be accessed corresponding to the data access request or not according to the first mapping relation;
if so, returning the data to be accessed to the client through the object storage format.
Optionally, after the determining, through the first mapping relationship, whether the data to be accessed corresponding to the data access request exists in the local cache, the method further includes:
if not, calling a second mapping relation established in advance; the second mapping relation is a mapping relation between a local storage and a cloud storage of the Nginx server;
determining a data reading address of cloud storage corresponding to the data access request according to the second mapping relation;
and by accessing the data reading address, pulling the data to be accessed in the object storage format from the cloud storage, and returning the pulled data to be accessed to the client.
Optionally, after the data to be accessed in the object storage format is pulled from the cloud storage, the method further includes:
and storing the pulled data to be accessed into a local cache.
Optionally, the method further includes:
receiving a data writing request sent by the client;
determining a data writing address of cloud storage corresponding to the data writing request according to the second mapping relation;
and forwarding the data to be written corresponding to the data writing request to the data writing address so as to write the data to be written into the corresponding cloud storage, and updating the local cache by using the data to be written.
Optionally, the pulling the data to be accessed in the object storage format from the cloud storage by accessing the data reading address includes:
and if the cloud storage corresponding to the data access request is multiple, synchronously accessing multiple corresponding data reading addresses so as to pull the data to be accessed in the object storage format from the multiple cloud storages.
Optionally, the method further includes:
acquiring newly added data or modified data from the cloud storage according to a preset period;
and updating the local cache according to the newly added data or the modified data.
In order to achieve the above object, the present application provides a data processing apparatus, which is applied to a Nginx server, the Nginx server acquires data from a cloud storage through a preset interface and caches the data to a local cache, and the apparatus includes:
the receiving module is used for receiving a data access request sent by a client;
the calling module is used for calling a first mapping relation which is established in advance; the first mapping relation is a mapping relation between object storage data and local cache data;
the judging module is used for judging whether the data to be accessed corresponding to the data access request exists in the local cache or not according to the first mapping relation;
and the feedback module is used for returning the data to be accessed to the client side through an object storage format if the corresponding data to be accessed exists in the local cache.
To achieve the above object, the present application provides an Nginx server, comprising:
a memory for storing a computer program;
a processor for implementing the steps of any of the data processing methods disclosed above when executing the computer program.
To achieve the above object, the present application provides a data processing system comprising:
a Nginx server, a client, and a plurality of cloud stores as disclosed above; the Nginx server is in data communication with cloud storage through a Restful API.
To achieve the above object, the present application provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of any of the data processing methods disclosed in the foregoing.
According to the scheme, the data processing method is applied to the Nginx server, the Nginx server acquires data from cloud storage through the preset interface and caches the data to the local cache, and the method comprises the following steps: receiving a data access request sent by a client; calling a first mapping relation established in advance; the first mapping relation is a mapping relation between object storage data and local cache data; judging whether the local cache has data to be accessed corresponding to the data access request or not according to the first mapping relation; if so, returning the data to be accessed to the client through the object storage format. Therefore, the Nginx server can read and cache the data stored in the cloud to the local, and can directly obtain the data from the local cache to return the data for the data access request sent by the subsequent client, so that the data delay of communication from the far end is omitted, and the online video playing is more smooth.
The application also discloses a data processing device, a system, an Nginx server and a computer readable storage medium, which can also realize the technical effects.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a flowchart of a data processing method disclosed in an embodiment of the present application;
FIG. 2 is a flow chart of another data processing method disclosed in the embodiments of the present application;
fig. 3 is a block diagram of a data processing apparatus according to an embodiment of the present disclosure;
fig. 4 is a structural diagram of an Nginx server disclosed in an embodiment of the present application;
fig. 5 is a structural diagram of another nginnx server disclosed in the embodiment of the present application;
fig. 6 is a block diagram of a data processing system according to an embodiment of the present disclosure.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
In the prior art, cloud storage implements an emerging scheme for placing storage resources on the cloud for human access. The user can conveniently access data at any time and any place through connecting to the cloud through any internet-connected device. More and more individuals or enterprises choose to put data in the cloud, so that the dependence on a local storage environment can be removed, and the data can also be used as a good backup carrier. However, since the data interaction at the cloud is accessed through the Internet, a relatively large data delay is inevitably caused. The problem of how to solve the high delay loss caused by Internet connection is also considered while bringing convenience.
Therefore, the embodiment of the application discloses a data processing method, which omits the data delay of communication from a far end, so that the online video playing has a smoother effect.
The data processing method disclosed in the embodiment of the application is applied to a Nginx server, the Nginx server acquires data from a cloud storage through a preset interface and caches the data to a local cache, and referring to fig. 1, the method comprises the following steps:
s101: receiving a data access request sent by a client;
in the embodiment of the application, a Nginx server is used for receiving a data access request sent by a client. In a specific implementation, the Nginx server may provide a unique object storage interface for a user, and configure object storage authentication inside the server, that is, receive a data access request initiated by the user through a client using the object storage interface, and determine information such as a right of a current user based on the object storage authentication, so as to determine whether to allow a response to the data access request of the current user.
S102: calling a first mapping relation established in advance; the first mapping relation is a mapping relation between object storage data and local cache data;
it should be noted that the Nginx server is a web server developed based on the C language and having high concurrency, and can meet the requirement of front-end access through http, and also can perform data interaction with a plurality of cloud storages through a reverse proxy, and can better meet the existing service requirement. However, the Nginx server only supports local caching of static web pages and does not have the ability to identify objects to store data.
Therefore, in the embodiment of the application, the Nginx server is developed for the second time in advance so that the Nginx server supports data analysis of restful, and specifically, a mapping relationship between object storage data and local cache data can be established in advance, so that mapping from object storage access to local cache data access can be realized in the Nginx server. Specifically, the first mapping relationship in the embodiment of the present application may specifically store a mapping relationship between basic information of object storage data on the cloud and basic information of local cache data, for example, the basic information may include, but is not limited to, data content, a storage location, and the like.
S103: judging whether the local cache has data to be accessed corresponding to the data access request or not according to the first mapping relation; if yes, go to step S104;
in this step, the data access request sent by the current client may be first analyzed to determine the data to be accessed corresponding to the data access request, and the data of the local cache is read, and whether the data to be accessed exists is searched in the local cache through the first mapping relationship.
S104: and returning the data to be accessed to the client through an object storage format.
It can be understood that, if the data to be accessed corresponding to the current data access request is found in the local cache, the data to be accessed can be returned to the corresponding client through the object storage format, so as to implement the data access.
As a feasible implementation manner, the embodiment of the present application may also actively update the local cache, and the specific process may include: the Nginx server acquires new data or modified data from the cloud storage according to a preset period; and updating the local cache according to the new data or the modified data. For example, newly added data stored in the cloud is additionally recorded to the local cache, or data to be modified in the local cache corresponding to modified data stored in the cloud is determined, and then the data to be modified is modified, so that the data is updated. The preset period may be set according to an actual service requirement in an implementation process, and is not specifically limited herein. For example, if the real-time requirement for the data is high, the preset period is set to a short time period, such as one hour, three hours, etc. In a specific implementation, the Nginx server may also provide a default value for a preset period first, and a user may adjust the default value in real time during an application process.
According to the scheme, the data processing method is applied to the Nginx server, the Nginx server acquires data from cloud storage through the preset interface and caches the data to the local cache, and the method comprises the following steps: receiving a data access request sent by a client; calling a first mapping relation established in advance; the first mapping relation is a mapping relation between object storage data and local cache data; judging whether the local cache has data to be accessed corresponding to the data access request or not according to the first mapping relation; if so, returning the data to be accessed to the client through the object storage format. Therefore, the Nginx server can read and cache the data stored in the cloud to the local, and can directly obtain the data from the local cache to return the data for the data access request sent by the subsequent client, so that the data delay of communication from the far end is omitted, and the online video playing is more smooth.
The embodiment of the application discloses another data processing method, and compared with the previous embodiment, the embodiment further explains and optimizes the technical scheme. Referring to fig. 2, specifically:
s201: receiving a data access request sent by a client;
s202: calling a first mapping relation established in advance; the first mapping relation is a mapping relation between object storage data and local cache data;
s203: judging whether the local cache has data to be accessed corresponding to the data access request or not according to the first mapping relation; if yes, go to step S204; if not, go to step S205;
s204: returning the data to be accessed to the client through an object storage format;
s205: calling a second mapping relation established in advance; the second mapping relation is a mapping relation between a local storage and a cloud storage of the Nginx server;
s206: determining a data reading address of cloud storage corresponding to the data access request according to the second mapping relation;
in the embodiment of the application, if the data to be accessed corresponding to the current data access request cannot be found in the local cache, the pre-established second mapping relation is called. Specifically, a restful api of a native reverse proxy may be added to the Nginx server in advance to configure the object storage configuration of the multiple clouds in the Nginx server. And further, a mapping relation between cloud storage of the Nginx server agent and local storage of the Nginx server can be established. After a data access request initiated by the client is received and the data to be accessed corresponding to the current data access request cannot be found in the local cache, the data reading address on the cloud storage corresponding to the current data access request, namely which data storage unit in which cloud storage object the data to be accessed is located, is determined through the second mapping relation.
S207: and by accessing the data reading address, pulling the data to be accessed in the object storage format from the cloud storage, and returning the pulled data to be accessed to the client.
After the data reading address is determined, the Nginx server can access the data reading address to pull the data to be accessed in the object storage format from the cloud storage, and return the data to be accessed to the client to respond to the data access request.
It can be understood that, in the embodiment of the present application, after the data to be accessed in the object storage format is pulled from the cloud storage, the pulled data to be accessed can be further stored in the local cache, so that the passive update of the local cache is realized.
In a specific implementation, the user can also write data through the client. Specifically, a data write request sent by a client may be received; determining a data writing address stored in a cloud corresponding to the data writing request according to the second mapping relation; and forwarding the data to be written corresponding to the data writing request to a data writing address so as to write the data to be written into the corresponding cloud storage, and updating the local cache by using the data to be written.
As a preferred embodiment, if the data to be accessed corresponding to the current data access request cannot be found in the local cache and the cloud storage needs to be removed for reading, if a second mapping relationship established in advance is called to determine that a plurality of cloud storages corresponding to the current data access request are available, the corresponding plurality of data reading addresses can be accessed synchronously, so that the data to be accessed in the object storage format is pulled from the plurality of cloud storages, and the data reading efficiency is improved.
In the following, a data processing apparatus provided in an embodiment of the present application is introduced, and a data processing apparatus described below and a data processing method described above may be referred to each other.
An embodiment of the present application provides a data processing apparatus is applied to a Nginx server, the Nginx server acquires data from a cloud storage through a preset interface and caches the data to a local cache, as shown in fig. 3, the apparatus includes:
a receiving module 301, configured to receive a data access request sent by a client;
a calling module 302, configured to call a first mapping relationship established in advance; the first mapping relation is a mapping relation between object storage data and local cache data;
a determining module 303, configured to determine, according to the first mapping relationship, whether to-be-accessed data corresponding to the data access request exists in the local cache;
a feedback module 304, configured to return the data to be accessed to the client through an object storage format if the corresponding data to be accessed exists in the local cache.
For the specific implementation process of the modules 301 to 304, reference may be made to the corresponding content disclosed in the foregoing embodiments, and details are not repeated here.
The present application further provides a Nginx server, and as shown in fig. 4, an Nginx server provided in an embodiment of the present application includes:
a memory 100 for storing a computer program;
the processor 200, when executing the computer program, may implement the steps of the data processing method provided by the above embodiments.
Specifically, the memory 100 includes a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer-readable instructions, and the internal memory provides an environment for the operating system and the computer-readable instructions in the non-volatile storage medium to run. The processor 200 may be a Central Processing Unit (CPU), a controller, a microcontroller, a microprocessor or other data processing chip in some embodiments, and provides the calculation and control capability for the Nginx server, and when executing the computer program stored in the memory 100, the steps of the data processing method disclosed in any of the foregoing embodiments may be implemented.
On the basis of the above embodiment, as a preferred implementation, referring to fig. 5, the Nginx server further includes:
and an input interface 300 connected to the processor 200, for acquiring computer programs, parameters and instructions imported from the outside, and storing the computer programs, parameters and instructions into the memory 100 under the control of the processor 200. The input interface 300 may be connected to an input device for receiving parameters or instructions manually input by a user. The input device may be a touch layer covered on a display screen, or a button, a track ball or a touch pad arranged on a terminal shell, or a keyboard, a touch pad or a mouse, etc.
And a display unit 400 connected to the processor 200 for displaying data processed by the processor 200 and for displaying a visualized user interface. The display unit 400 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch panel, or the like.
And a network port 500 connected to the processor 200 for performing communication connection with each external terminal device. The communication technology adopted by the communication connection can be a wired communication technology or a wireless communication technology, such as a mobile high definition link (MHL) technology, a Universal Serial Bus (USB), a High Definition Multimedia Interface (HDMI), a wireless fidelity (WiFi), a bluetooth communication technology, a low power consumption bluetooth communication technology, an ieee802.11 s-based communication technology, and the like.
While fig. 5 shows only a nginnx server having the assembly 100-500, those skilled in the art will appreciate that the structure shown in fig. 5 is not limiting of a nginnx server and may include fewer or more components than shown, or combine certain components, or a different arrangement of components.
The present application further provides a data processing system, and as shown in fig. 6, an embodiment of the present application provides a data processing system including: the system comprises an Nginx server, a client and a plurality of cloud storages; the Nginx server is in data communication with the cloud storage through the Restful API.
Specifically, the embodiment of the application performs secondary development in the Nginx server to support data analysis of Restful API, and caches data read back from the cloud in the local. For the reading request sent by the subsequent client, the data can be directly obtained from the local cache and returned, and the delay of communication to the far end is saved. Meanwhile, the Restful API extension of the native reverse proxy is added, and the object storage configuration of a plurality of cloud ends can be simultaneously supported. Furthermore, a unique object storage interface is provided for the user by using the Nginx server, so that a plurality of cloud storage data at the rear end can be accessed only through unified object storage authentication provided by the Nginx server, independent and different user authentications do not need to be carried out for different cloud ends, and the user can access the Nginx server by using unified authentication information such as keys. The Nginx server internally realizes the mapping from object storage access to local cache data access and the mapping from local object storage access to cloud object storage access, and can update data in an active or passive mode.
Specifically, as shown in fig. 6, the local client may access three different cloud object stores simultaneously through the Nginx service address based on the unified object store authentication; after the Nginx server receives an object storage access request sent by the client, if the request is a reading request, the storage data cached in the local can be found according to the internal mapping relation and sent to the client through the format of object storage, so that the long step of accessing the data at the remote end is omitted. If the data corresponding to the read request is not in the cache, the Nginx server needs to go to the cloud end through the real background address of the proxy to store the data stored in the pull object, return the data to the client and update and store the data in the local cache. In order to ensure the consistency of the cache data, the nginnx server needs to go to the cloud end for storage periodically to perform data synchronization, or after the nginnx server receives a write request sent by the client, the local cache data is updated in time while the data corresponding to the write request is forwarded to the cloud end for storage.
The data processing system provided by the embodiment of the application realizes the accelerated proxy of data based on the Nginx server, can improve the efficiency of reading the client data by supporting the access of object storage, realizes the accelerated access of cloud storage data, and has a smoother effect on online video playing. Meanwhile, data stored in a plurality of cloud ends can be hidden in the background in a reverse proxy mode, a user does not need to care about configuration of the back-end storage, the data in the plurality of cloud ends can be accessed at one time, and great convenience is provided for the user.
The present application also provides a computer-readable storage medium, which may include: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk. The storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of the data processing method disclosed in any of the foregoing embodiments.
According to the method and the system, the Nginx server can read and cache the data stored in the cloud to the local, the data can be directly obtained from the local cache to return for the data access request sent by the subsequent client, the data delay of communication from the far end is omitted, and the online video playing is enabled to have a smoother effect.
The embodiments are described in a progressive manner in the specification, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the system disclosed by the embodiment, the description is relatively simple because the system corresponds to the method disclosed by the embodiment, and the relevant points can be referred to the method part for description. It should be noted that, for those skilled in the art, it is possible to make several improvements and modifications to the present application without departing from the principle of the present application, and such improvements and modifications also fall within the scope of the claims of the present application.
It is further noted that, in the present specification, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
Claims (10)
1. A data processing method is applied to a Nginx server, and is characterized in that the Nginx server acquires data from a cloud storage through a preset interface and caches the data to a local cache, and the method comprises the following steps:
receiving a data access request sent by a client;
calling a first mapping relation established in advance; the first mapping relation is a mapping relation between object storage data and local cache data;
judging whether the local cache has data to be accessed corresponding to the data access request or not according to the first mapping relation;
if so, returning the data to be accessed to the client through the object storage format.
2. The data processing method according to claim 1, wherein after the determining, through the first mapping relationship, whether there is data to be accessed corresponding to the data access request in a local cache, the method further comprises:
if not, calling a second mapping relation established in advance; the second mapping relation is a mapping relation between a local storage and a cloud storage of the Nginx server;
determining a data reading address of cloud storage corresponding to the data access request according to the second mapping relation;
and by accessing the data reading address, pulling the data to be accessed in the object storage format from the cloud storage, and returning the pulled data to be accessed to the client.
3. The data processing method according to claim 2, further comprising, after the pulling of the data to be accessed in the object store format from the cloud store:
and storing the pulled data to be accessed into a local cache.
4. The data processing method of claim 2, further comprising:
receiving a data writing request sent by the client;
determining a data writing address of cloud storage corresponding to the data writing request according to the second mapping relation;
and forwarding the data to be written corresponding to the data writing request to the data writing address so as to write the data to be written into the corresponding cloud storage, and updating the local cache by using the data to be written.
5. The data processing method according to claim 2, wherein the pulling of the data to be accessed in the object storage format from the cloud storage by accessing the data reading address comprises:
and if the cloud storage corresponding to the data access request is multiple, synchronously accessing multiple corresponding data reading addresses so as to pull the data to be accessed in the object storage format from the multiple cloud storages.
6. The data processing method according to any one of claims 1 to 5, further comprising:
acquiring newly added data or modified data from the cloud storage according to a preset period;
and updating the local cache according to the newly added data or the modified data.
7. The utility model provides a data processing apparatus, is applied to Nginx server, its characterized in that, Nginx server acquires data and caches to local cache from the high in the clouds storage through predetermineeing the interface, the device includes:
the receiving module is used for receiving a data access request sent by a client;
the calling module is used for calling a first mapping relation which is established in advance; the first mapping relation is a mapping relation between object storage data and local cache data;
the judging module is used for judging whether the data to be accessed corresponding to the data access request exists in the local cache or not according to the first mapping relation;
and the feedback module is used for returning the data to be accessed to the client side through an object storage format if the corresponding data to be accessed exists in the local cache.
8. An Nginx server, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the data processing method according to any one of claims 1 to 6 when executing the computer program.
9. A data processing system, comprising:
the Nginx server of claim 8, a client, and a plurality of cloud storage; the Nginx server is in data communication with cloud storage through a Restful API.
10. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, which computer program, when being executed by a processor, carries out the steps of the data processing method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911382325.XA CN111143417A (en) | 2019-12-27 | 2019-12-27 | Data processing method, device and system, Nginx server and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911382325.XA CN111143417A (en) | 2019-12-27 | 2019-12-27 | Data processing method, device and system, Nginx server and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111143417A true CN111143417A (en) | 2020-05-12 |
Family
ID=70521257
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911382325.XA Pending CN111143417A (en) | 2019-12-27 | 2019-12-27 | Data processing method, device and system, Nginx server and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111143417A (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111694868A (en) * | 2020-06-16 | 2020-09-22 | 北京同邦卓益科技有限公司 | Data caching method and device, electronic equipment and storage medium |
| CN111885098A (en) * | 2020-06-03 | 2020-11-03 | 中邮消费金融有限公司 | Proxy access method, system and computer equipment for object storage cluster |
| CN112887434A (en) * | 2021-04-13 | 2021-06-01 | 上海哔哩哔哩科技有限公司 | Data processing method and device |
| CN113242285A (en) * | 2021-04-30 | 2021-08-10 | 北京京东拓先科技有限公司 | Hotspot data processing method, device and system |
| CN114327302A (en) * | 2022-03-11 | 2022-04-12 | 阿里云计算有限公司 | Method, device and system for processing object storage access |
| CN115061952A (en) * | 2022-08-19 | 2022-09-16 | 飞狐信息技术（天津）有限公司 | Data caching method and device, electronic equipment and computer storage medium |
| CN115150422A (en) * | 2021-03-31 | 2022-10-04 | 北京轻舟智航智能技术有限公司 | Data processing apparatus and data processing method |
| CN115525686A (en) * | 2022-10-10 | 2022-12-27 | 中电金信软件有限公司 | Caching method and device for mapping configuration data |
| CN116110607A (en) * | 2023-04-13 | 2023-05-12 | 暨南大学 | Method and system for checking detection data of traditional Chinese medicine components |
| WO2023185454A1 (en) * | 2022-03-28 | 2023-10-05 | 华为技术有限公司 | Data access method and related apparatus |
| CN117724852A (en) * | 2024-02-07 | 2024-03-19 | 微网优联科技(成都)有限公司 | Cloud computer computing resource allocation method and device |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104349324A (en) * | 2013-11-15 | 2015-02-11 | 上海触乐信息科技有限公司 | Mobile terminal number intelligent protection system and method |
| CN104468807A (en) * | 2014-12-12 | 2015-03-25 | 北京小渔无限信息技术有限公司 | Processing method, cloud end device, local devices and system for webpage cache |
| CN104753966A (en) * | 2013-12-25 | 2015-07-01 | 明博教育科技有限公司 | Resource file inquiry method and system based on server and client caches |
| CN105635196A (en) * | 2014-10-27 | 2016-06-01 | 中国电信股份有限公司 | Method and system of file data obtaining, and application server |
| CN107194253A (en) * | 2017-05-23 | 2017-09-22 | 维沃移动通信有限公司 | A kind of applied program processing method, mobile terminal and Cloud Server |
| CN107329963A (en) * | 2016-04-29 | 2017-11-07 | 北京京东尚科信息技术有限公司 | Accelerate the method and apparatus of web page access |
| CN109120467A (en) * | 2017-06-22 | 2019-01-01 | 中兴通讯股份有限公司 | A kind of Reverse Proxy monitoring method, device and Reverse Proxy |
-
2019
- 2019-12-27 CN CN201911382325.XA patent/CN111143417A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104349324A (en) * | 2013-11-15 | 2015-02-11 | 上海触乐信息科技有限公司 | Mobile terminal number intelligent protection system and method |
| CN104753966A (en) * | 2013-12-25 | 2015-07-01 | 明博教育科技有限公司 | Resource file inquiry method and system based on server and client caches |
| CN105635196A (en) * | 2014-10-27 | 2016-06-01 | 中国电信股份有限公司 | Method and system of file data obtaining, and application server |
| CN104468807A (en) * | 2014-12-12 | 2015-03-25 | 北京小渔无限信息技术有限公司 | Processing method, cloud end device, local devices and system for webpage cache |
| CN107329963A (en) * | 2016-04-29 | 2017-11-07 | 北京京东尚科信息技术有限公司 | Accelerate the method and apparatus of web page access |
| CN107194253A (en) * | 2017-05-23 | 2017-09-22 | 维沃移动通信有限公司 | A kind of applied program processing method, mobile terminal and Cloud Server |
| CN109120467A (en) * | 2017-06-22 | 2019-01-01 | 中兴通讯股份有限公司 | A kind of Reverse Proxy monitoring method, device and Reverse Proxy |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111885098A (en) * | 2020-06-03 | 2020-11-03 | 中邮消费金融有限公司 | Proxy access method, system and computer equipment for object storage cluster |
| CN111885098B (en) * | 2020-06-03 | 2021-06-29 | 中邮消费金融有限公司 | Proxy access method, system and computer equipment for object storage cluster |
| CN111694868A (en) * | 2020-06-16 | 2020-09-22 | 北京同邦卓益科技有限公司 | Data caching method and device, electronic equipment and storage medium |
| CN115150422A (en) * | 2021-03-31 | 2022-10-04 | 北京轻舟智航智能技术有限公司 | Data processing apparatus and data processing method |
| CN112887434A (en) * | 2021-04-13 | 2021-06-01 | 上海哔哩哔哩科技有限公司 | Data processing method and device |
| CN112887434B (en) * | 2021-04-13 | 2023-05-23 | 上海哔哩哔哩科技有限公司 | Data processing method and device |
| CN113242285A (en) * | 2021-04-30 | 2021-08-10 | 北京京东拓先科技有限公司 | Hotspot data processing method, device and system |
| WO2023169269A1 (en) * | 2022-03-11 | 2023-09-14 | 阿里云计算有限公司 | Method, apparatus and system for processing access to object storage |
| CN114327302A (en) * | 2022-03-11 | 2022-04-12 | 阿里云计算有限公司 | Method, device and system for processing object storage access |
| WO2023185454A1 (en) * | 2022-03-28 | 2023-10-05 | 华为技术有限公司 | Data access method and related apparatus |
| CN115061952A (en) * | 2022-08-19 | 2022-09-16 | 飞狐信息技术（天津）有限公司 | Data caching method and device, electronic equipment and computer storage medium |
| CN115061952B (en) * | 2022-08-19 | 2022-12-27 | 飞狐信息技术（天津）有限公司 | Data caching method and device, electronic equipment and computer storage medium |
| CN115525686A (en) * | 2022-10-10 | 2022-12-27 | 中电金信软件有限公司 | Caching method and device for mapping configuration data |
| CN115525686B (en) * | 2022-10-10 | 2023-06-13 | 中电金信软件有限公司 | Caching method and device for mapping configuration data |
| CN116110607B (en) * | 2023-04-13 | 2023-06-09 | 暨南大学 | Method and system for checking detection data of traditional Chinese medicine components |
| CN116110607A (en) * | 2023-04-13 | 2023-05-12 | 暨南大学 | Method and system for checking detection data of traditional Chinese medicine components |
| CN117724852A (en) * | 2024-02-07 | 2024-03-19 | 微网优联科技(成都)有限公司 | Cloud computer computing resource allocation method and device |
| CN117724852B (en) * | 2024-02-07 | 2024-05-07 | 微网优联科技(成都)有限公司 | Cloud computer computing resource allocation method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111143417A (en) | Data processing method, device and system, Nginx server and medium | |
| US8645453B2 (en) | Method and system of processing cookies across domains | |
| US8438251B2 (en) | Methods and systems for implementing a virtual storage network | |
| CN101533410B (en) | Widget content buffering method facing mobile terminal | |
| US20130227047A1 (en) | Methods for managing content stored in cloud-based storages | |
| CN113010818B (en) | Access current limiting method, device, electronic equipment and storage medium | |
| KR20160069445A (en) | User apparatus for managing web application and method therefor | |
| JP2018518733A (en) | File operation method and apparatus | |
| TWI461943B (en) | Method and apparatus for multiple windows simultaneously analyzing and displaying web pages | |
| JP2016526230A (en) | Computer program product, system and method for optimizing web page loading | |
| CN104899274B (en) | A kind of memory database Efficient Remote access method | |
| US10402464B2 (en) | Methods and apparatuses for opening a webpage, invoking a client, and creating a light application | |
| US10015253B2 (en) | System and method for preemptive request processing | |
| CN106817388B (en) | Method and device for acquiring data by virtual machine and host machine and system for accessing data | |
| WO2019041670A1 (en) | Method, device and system for reducing frequency of functional page requests, and storage medium | |
| CN105138649A (en) | Data search method and device and terminal | |
| CN110222046B (en) | List data processing method, device, server and storage medium | |
| CN103150172A (en) | Method and equipment for realizing individual scene | |
| US11146860B2 (en) | Service processing method, device, and intelligent terminal for obtaining and displaying data based on texture data corresponding to a target service | |
| CN103123651A (en) | Method of rapidly searching multiple same-kind paper, device and mobile equipment | |
| CN108173898A (en) | The method, apparatus and terminal device of service are accessed for resource | |
| US20150295991A1 (en) | Method and device for browsing network data, and storage medium | |
| CN103685472A (en) | Method and equipment used for providing resource information corresponding to mobile equipment | |
| US11288336B2 (en) | Systems and methods for providing content items in situations involving suboptimal network conditions | |
| CN113722007A (en) | Configuration method, device and system of VPN branch equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |