WO2021036909A1

WO2021036909A1 - Picture write-in method and apparatus

Info

Publication number: WO2021036909A1
Application number: PCT/CN2020/110330
Authority: WO
Inventors: 严波; 陈鹏
Original assignee: 杭州海康威视系统技术有限公司
Priority date: 2019-08-23
Filing date: 2020-08-20
Publication date: 2021-03-04
Also published as: CN111399753A; CN111399753B

Abstract

Provided are a picture write-in method and apparatus, falling within the technical field of storage machines. The method is applied to a target storage node in a cloud storage system, and the target storage node is configured with a memory, a magnetic disk and an SSD. The method comprises: a target storage node receiving a write-in request regarding a target picture; determining whether a picture resource pool to which the target storage node belongs supports accelerated write-in; when the picture resource pool to which the target storage node belongs supports accelerated write-in, if the data volume of the target picture is less than a target value, acquiring a configured cache type in configuration information of the storage node; if the cache type is memory acceleration, the target storage node writing the target picture into a memory of the target storage node; and if the cache type is SSD acceleration, the target storage node writing the target picture into an SSD of the target storage node. By means of the present disclosure, the picture write-in efficiency can be improved.

Description

Method and device for writing pictures

This disclosure claims the priority of a Chinese patent application filed on August 23, 2019, with the application number 201910786664.8 and the invention title "Method and Apparatus for Writing Pictures", the entire contents of which are incorporated into this disclosure by reference.

Technical field

The present disclosure relates to the field of storage technology, and in particular to a method and device for writing pictures.

Background technique

With the development of computer technology and network technology, more and more application scenarios need to store pictures and extract pictures. For example, a camera set up on a road needs to store the picture after taking the picture, and when the manager views the picture, the picture needs to be extracted.

In the related art, after a picture is captured and stored, the picture is generally stored in a disk, and subsequently read from the disk.

Since the writing speed of the disk is slow, the writing efficiency of the picture will be lower.

Summary of the invention

In order to solve the related art problems, embodiments of the present disclosure provide a method and apparatus for writing pictures. The technical solution is as follows:

In the first aspect, a method for writing pictures is provided, the method is applied to a target storage node in a cloud storage system, and the target storage node is configured with a disk, a memory, and an SSD (Solid State Drives); The methods include:

The target storage node receives the write request of the target picture, and determines whether the picture resource pool to which the target storage node belongs supports accelerated writing;

When the picture resource pool to which the target storage node belongs supports accelerated writing, if the data amount of the target picture is less than the target value, the configured cache type is obtained from the configuration information of the storage node, wherein the cache Type is the cache type of the target picture when the target storage node writes;

If the cache type is memory acceleration, the target storage node writes the target picture into the memory of the target storage node;

If the cache type is SSD acceleration, the target storage node writes the target picture into the SSD of the target storage node.

In this way, when the amount of data of the picture is relatively small, the picture can be stored in the memory or SSD. Since the writing speed of the memory or SSD is faster, when the amount of data of the picture is relatively small, the writing speed of the picture can be relatively improved. Improve writing efficiency. Moreover, the pictures are stored in the memory or SSD. When reading the pictures, the pictures can be read from the memory or SSD. Since the reading speed of the memory or SSD is faster, the efficiency of reading pictures can also be improved.

Optionally, the method further includes:

Writing the target picture to the disk of the storage node.

In this way, the picture can be prevented from being lost.

Optionally, the writing the target picture into the memory of the target storage node includes:

If the current memory margin of the target storage node is greater than or equal to the data amount of the target picture, writing the target picture into the memory of the target storage node;

If the current memory margin of the target storage node is less than the data amount of the target picture, deleting the picture whose writing time point is the longest from the current time point in the memory;

If the memory margin after deleting the picture is greater than or equal to the data amount of the target picture, then the picture is written into the memory, and if the memory margin after deleting the picture is less than the data amount of the target picture, execute The process of deleting the picture whose writing time point is the longest from the current time point in the memory.

In this way, it can be stored in the memory as much as possible.

Optionally, the method further includes:

The target storage node receives the read request of the target picture sent by the requester, and determines whether the picture resource pool to which the target storage node belongs supports accelerated writing, wherein the read request carries the target picture Index information stored on disk;

If the picture resource pool to which the target storage node belongs supports accelerated writing, read the target picture according to the cache type corresponding to the target picture and the read request;

Sending the target picture to the requesting party.

In this way, the picture can be read quickly.

Optionally, the reading the target picture according to the cache type corresponding to the target picture and the read request includes:

If the cache type corresponding to the target picture is memory acceleration, determining the first index of the target picture stored in the memory of the target storage node according to the index information;

If the first index is found in the index table corresponding to the memory of the target storage node, read the target picture from the memory of the target storage node according to the first index; if in the If the first index is not found in the index table corresponding to the memory of the target storage node, the target picture is read from the disk;

If the cache type corresponding to the target picture is SSD acceleration, the second index of the target picture stored in the SSD is determined according to the index information, and if the index table corresponding to the SSD configured on the target storage node is searched To the second index, read the target picture from the SSD configured on the target storage node according to the second index;

If the second index is not found in the index table corresponding to the SSD configured on the target storage node, the target picture is read from the disk according to the read request.

In this way, since the picture can also be read in the memory or SSD, the picture can be read quickly.

In a second aspect, a device for writing pictures is provided, the device is applied to a target storage node in a cloud storage system, the target storage node is configured with a disk, a memory, and an SSD; the device includes:

The receiving module is used to receive the write request of the target picture;

A judging module for judging whether the image resource pool to which the target storage node belongs supports accelerated writing;

The determining module is configured to obtain the configured cache type from the configuration information of the storage node when the image resource pool to which the target storage node belongs supports accelerated writing, if the data amount of the target picture is less than the target value, Wherein, the cache type is the cache type of the target picture when the target storage node writes;

A writing module, configured to write the target picture into the memory of the target storage node if the cache type is memory acceleration;

If the cache type is SSD acceleration, the target picture is written into the SSD of the target storage node.

Optionally, the writing module is also used for:

Writing the target picture to the disk of the target storage node.

Optionally, the writing module is used for:

Optionally, the receiving module is further configured to receive a read request of a target picture sent by a requesting party, wherein the read request carries index information of the target picture stored on a disk;

The judgment module is also used to judge whether the image resource pool to which the target storage node belongs supports accelerated writing;

The device also includes:

A reading module, configured to read the target picture according to the cache type corresponding to the target picture and the read request if the picture resource pool to which the target storage node belongs supports accelerated writing;

The sending module is used to send the target picture to the requester.

Optionally, the reading module is used for:

In a third aspect, a method for reading pictures is provided, the method is applied to a target storage node in a cloud storage system, the target storage node is configured with a disk, a memory, and a solid-state hard drive (SSD); the method includes:

The target storage node receives the read request of the target picture sent by the requester, and determines whether the picture resource pool to which the target storage node belongs supports accelerated writing;

If the image resource pool to which the target storage node belongs supports accelerated writing, and the cache type is memory acceleration, the target image is read in the memory configured by the target storage node according to the read request, where: The cache type is the cache type of the target picture when the target storage node writes;

If the picture resource pool to which the target storage node belongs supports accelerated writing, and the cache type is SSD accelerated, reading the target picture from the SSD configured on the target storage node according to the read request;

Sending the target picture to the requesting party.

In this way, the pictures are stored in the memory or SSD, so the pictures can be read from the memory or SSD. Since the read speed of the memory or SSD is faster, the efficiency of reading pictures can be improved.

Optionally, the read request carries index information of the target picture stored on the disk;

The reading the target picture in the memory configured by the target storage node according to the reading request includes:

Determine the first index of the target picture stored in the memory of the target storage node according to the index information;

If the first index is found in the index table corresponding to the memory of the target storage node, the target picture is read from the memory of the target storage node according to the first index.

The reading the target picture in the SSD configured on the target storage node according to the reading request includes:

Determine the second index of the target picture stored in the SSD according to the index information;

If the second index is found in the index table corresponding to the SSD configured on the target storage node, the target picture is read from the SSD configured on the target storage node according to the second index.

In a fourth aspect, a storage node for writing pictures is provided, the storage node includes a processor and a memory, and at least one instruction is stored in the memory, and the instruction is loaded and executed by the processor to achieve the above The method of writing pictures described in the first aspect.

In a fifth aspect, a computer-readable storage medium is provided, and at least one instruction is stored in the storage medium, and the instruction is loaded and executed by a processor to implement the method for writing a picture as described in the first aspect.

The beneficial effects brought about by the technical solutions provided by the embodiments of the present disclosure include at least:

In the embodiment of the present disclosure, the target storage node in the cloud storage system is configured with a disk, a memory, and an SSD. When writing a target picture in the cloud storage system, the target storage node can receive the write request of the target picture and determine whether the picture resource pool to which the target storage node belongs supports accelerated writing, and if the picture resource pool to which the storage node belongs supports accelerated writing , The data amount of the target picture and the size of the target value are judged. If the data amount of the target picture is less than the target value, the configured cache type is obtained from the configuration information of the storage node. If the cache type is memory acceleration, the target storage node writes the target image into the memory of the target storage node. If the cache type is SSD acceleration, the target storage node writes the target image into the SSD configured by the target storage node. In this way, when the amount of data of the picture is relatively small, the picture can be stored in the memory or SSD. Since the writing speed of the memory or SSD is faster, when the amount of data of the picture is relatively small, the writing speed of the picture can be relatively improved. Improve writing efficiency. And the picture is stored in the memory or SSD, and the picture can be read from the memory or SSD. Since the reading speed of the memory or SSD is faster, the reading efficiency of the picture can also be improved.

Description of the drawings

FIG. 1 is a schematic diagram of a cloud storage system provided by an embodiment of the present disclosure;

FIG. 2 is a flowchart of a method for writing pictures according to an embodiment of the present disclosure;

FIG. 3 is a schematic diagram of setting a cache type according to an embodiment of the present disclosure;

FIG. 4 is a schematic diagram of setting a target value provided by an embodiment of the present disclosure;

FIG. 5 is a schematic diagram of setting an acceleration strategy according to an embodiment of the present disclosure;

FIG. 6 is a flowchart of a method for extracting pictures provided by an embodiment of the present disclosure;

FIG. 7 is a schematic structural diagram of an apparatus for writing pictures according to an embodiment of the present disclosure;

FIG. 8 is a schematic structural diagram of an apparatus for writing pictures according to an embodiment of the present disclosure;

FIG. 9 is a schematic structural diagram of a storage node provided by an embodiment of the present disclosure.

detailed description

In order to make the objectives, technical solutions, and advantages of the present disclosure clearer, the following further describes the embodiments of the present disclosure in detail with reference to the accompanying drawings.

The embodiment of the present disclosure provides a method for writing a picture. The execution subject of the method may be a storage node, and the storage node may be a server.

The storage node may be provided with a processor, a memory, a transceiver, etc., and the processor may be used to process the process of writing pictures. For example, the processor is used to write pictures in the storage node, read pictures from the storage node, and so on. The memory can be used for the data needed and generated during the process of writing pictures. For example, the memory is used for storing written pictures. The transceiver can be used for receiving and sending data. For example, it can be used for sending or receiving pictures. It can also be used to receive picture write requests, picture read requests, and so on.

Before implementation, first introduce the application scenarios involved in the embodiments of the present disclosure:

The method for writing pictures mentioned in the embodiments of the present disclosure is applied to a storage node of a cloud storage system, and there are multiple storage nodes. The storage nodes in the cloud storage system can be divided into different picture resource pools, and the same storage node will not exist in different picture resource pools. In this way, it can be considered that the cloud storage system includes multiple image resource pools, and each image resource pool can be configured to support accelerated writing or not to support accelerated writing. The storage node in the image resource pool that supports accelerated writing is configured with disk, memory, and SSD. The storage node in the image resource pool that does not support accelerated writing can be configured with disk, memory, and SSD, or only with disk and memory. . What needs to be explained here is that the image resource pool is a virtual storage space.

As shown in Figure 1, the cloud storage system may include an application layer and a device layer. The application layer includes a management node and a storage node. Multiple storage nodes form a picture resource pool, and the management node can be used to manage storage nodes in the picture resource pool. The device layer includes external storage components of storage nodes, such as disks and SSDs. In addition, a virtual file system layer is also included between the application layer and the device layer to connect the device layer and the application layer.

As shown in FIG. 2, an embodiment of the present disclosure provides a method for writing a picture, taking a picture taken by a camera for storage as an example for description, and the processing flow of the method may be as follows:

Step 201: The target storage node receives the write request of the target picture, and judges whether the picture resource pool to which the target storage node belongs supports accelerated writing.

Among them, the target picture can be any picture.

In implementation, after the camera captures the target picture, it can generate a write request and add the target picture to it, and the camera determines the management node to connect to. Then send the write request to the management node. After receiving the write request, the management node can view the image resource pool connected to the camera ID, and determine the storage capacity of each storage node in the storage nodes included in the image resource pool, and the total storage space of each storage node. For any storage node, compare the storage capacity of the storage node with the total storage space capacity of the storage node to obtain the storage occupancy rate of the storage node. The management node may select the storage node with the least occupancy rate (hereinafter may be referred to as the target storage node), and send a write request of the target picture to the target storage node.

The target storage node may receive the write request sent by the management node, and parse the write request to obtain the target picture.

The target storage node may obtain the corresponding relationship between the storage node and the picture resource pool, and from the corresponding relationship, determine the picture resource pool to which the target storage node belongs, or the configuration information of the target storage node includes the identity of the picture resource pool to which it belongs, The target storage node can directly obtain the identification of the picture resource pool to which it belongs from the configuration information. Then the target storage node obtains the configuration file of the image resource pool, obtains the writing flag of the image resource pool from the configuration file, and determines whether the image resource pool supports accelerated writing according to the value of the writing flag.

In addition, the foregoing is based on the storage occupancy rate of the storage node to determine the target storage node. In the embodiments of the present disclosure, it may also be based on the weighting of the storage occupancy rate of the storage node and the central processing unit (CPU) occupancy rate of the storage node. Value to determine the target storage node. For example, the management node determines the storage node with the smallest weighted value of the storage occupancy rate and the CPU occupancy rate as the target storage node. The weight coefficients corresponding to the storage occupancy rate and the CPU occupancy rate are different, and the sum is 1, and the weight coefficient corresponding to the storage occupancy rate is greater than the weight coefficient corresponding to the CPU occupancy rate.

Step 202: When the image resource pool to which the target storage node belongs supports accelerated writing, if the data amount of the target image is less than the target value, obtain the configured cache type from the configuration information of the storage node.

Among them, the target value can be set and stored in the storage node. The target value generally belongs to 0～500k and can be set to 100k. Cache types are generally divided into non-accelerated, memory accelerated, and SSD accelerated.

In implementation, when the value of the write flag of the picture resource pool to which the target storage node belongs is the first value, the target storage node may determine that the picture resource pool supports accelerated writing. When the value of the write flag of the picture resource pool to which the target storage node belongs is the second value, the target storage node may determine that the picture resource pool does not support accelerated writing. The first value and the second value are different values. For example, the first value may be 1, and the second value may be zero.

Then the target storage node can determine the relationship between the data volume of the target image and the target value. If the data volume of the target image is less than the target value, it can obtain the configured cache type from the configuration information of the target storage node, and then set the cache type Set to the cache type corresponding to the target image. In this way, when the data volume of the picture is less than the target value, it can be accelerated by means of memory or SSD to increase the writing speed. In this way, because the target value is set, when the amount of image data is relatively small, it can be stored in SSD or memory, and when the amount of image data is relatively large, it can be stored in the disk, not only without losing the image, but also improving The reading speed of pictures with a relatively small amount of data.

Optionally, in order to prevent the loss of image data, you can also perform the following processing:

The target storage node writes the target image to the disk of the target storage node.

In implementation, after receiving the write request, the target storage node may first directly write the target image to the disk of the target storage node, and then determine whether the image resource pool to which the target storage node belongs supports accelerated writing. Alternatively, after receiving the write request of the target picture, the target storage node may first determine whether the picture resource pool to which the target storage node belongs supports accelerated writing, and then write the target picture to the disk. In this way, if the image resource pool does not support acceleration or supports acceleration, the image can also be written to the disk, which can prevent the loss of image data.

In addition, in step 202, if the data amount of the target picture is greater than or equal to the target value, since the target picture has been written to the disk of the target storage node, subsequent processing is not required, and the writing process of the target picture is completed. In addition, if the target picture does not support accelerated writing, the target picture is written to the disk of the target storage node, no subsequent processing is required, and the writing process of the target picture is completed.

Step 203: If the cache type is memory acceleration, the target image is written into the memory of the storage node; if the cache type is solid state drive SSD acceleration, the target image is written into the SSD of the storage node.

In implementation, if the cache type of the target image is memory acceleration, the target storage node can write the target image into the memory of the target storage node, and create the index value of the target image for subsequent retrieval from the memory of the target storage node Read the target picture. The index value here can be the identification of the target picture. Corresponding to the memory, the target storage node maintains an index table. The index table includes the corresponding relationship between the index value and the storage location information. The target storage node also records the corresponding relationship between the index value and the block identifier of the target image stored on the disk. In this way, Later, you can use the identifier of the block in the disk to find the storage location information of the target picture in the memory.

If the cache type is SSD acceleration, the target storage node can determine its own SSD, write the target picture into its own SSD, and create the index value of the target picture for subsequent reading of the target picture from the SSD. The index value here can be the identification of the target picture. Corresponding to the SSD, the target storage node maintains an index table. The index table includes the corresponding relationship between the index value and the storage location information. The target storage node also records the corresponding relationship between the index value and the block identifier of the picture stored on the disk. In this way, the subsequent You can use the identifier of the block in the disk to find the storage location information of the target image stored in the SSD.

In this way, due to the memory storage and SSD storage, for pictures with a small amount of data, the speed of writing pictures and reading pictures can be made faster.

Optionally, when the target image is written into the memory of the target storage node, the data amount of the target image and the size of the memory margin are also determined, and the processing can be as follows:

If the current memory margin of the target storage node is greater than or equal to the data amount of the target image, the target image is written into the memory of the target storage node; if the current memory margin of the target storage node is less than the data amount of the target image, then the target image is written in the memory Delete the picture that is the longest from the current time point in the writing time point; if the memory margin after deleting the picture is greater than or equal to the data amount of the target picture, then the picture is written into the memory, if the memory margin after deleting the picture is less than the target For the amount of data of the picture, the process of deleting the picture whose writing time point is the longest from the current time point in the memory is executed.

In implementation, the target storage node can determine the memory margin of the current memory, and the memory margin refers to the space in the current memory that has not yet stored data. Then the target storage node can compare the memory margin with the size of the data volume of the target picture. If the memory margin is greater than or equal to the data amount of the target picture, the target picture can be directly written into the memory of the target storage node.

If the memory margin is less than the data amount of the target picture, the target storage node can determine the picture with the longest time point from the current time point in the memory, and then delete the picture. Then determine the current memory margin, if the memory margin is greater than or equal to the data amount of the target picture, you can directly write the target picture into the memory. If the current memory margin is still less than the data amount of the target picture, you can continue to delete the picture in the current memory and write the picture with the longest time point from the current time point until the memory margin is greater than or equal to the data amount of the target picture. Write to the memory of the target storage node.

In addition, in the embodiment of the present disclosure, after the target storage node writes the target image to the disk of the target storage node, it may return index information to the management node, where the index information is the identifier of the disk to which the target image is written and the identifier of the block. .

It should be noted that the process shown in FIG. 2 may be executed by an application on the target storage node. After the application is started, the configuration information of the target storage node can be read, and configuration items such as the cache type, memory size, and target value can be obtained from it. If the application detects that the memory cache is enabled in the configuration item, it can create a memory for caching pictures according to the size of the memory, and initialize the memory, and then use the memory to cache pictures. If the memory allocation fails, the application fails to start, and the memory cannot be used to cache pictures subsequently.

In addition, in the process shown in Figure 2, it can be executed by an application on the target storage node. In the application, there are two processes, one is A process, the other is B process, A and Process B is connected through a unix domain socket link. In the process shown in Figure 2, process A can receive the requester’s write request and inform process B through the unix domain socket link. Process B can perform the above step 202 To 203. Then after writing the target picture, the index information (the index information is the identifier of the disk and the identifier of the block stored in the disk) is returned to the A process through the unix domain socket link. Process A can feed back to the requesting party.

In addition, in the embodiment of the present disclosure, a process for the target storage node to obtain the cache type is also provided:

As shown in Figure 3, the user can click the option to set the cache type on the cloud storage system management page displayed on the terminal to trigger the terminal to display the storage node that has not set the cache type, find the target storage node in it, and select the cache for the target storage node Types of. Then the user can click the confirmation option to trigger the generation of the setting information of the cache type (the setting information carries the set cache type), and send the setting information of the cache type to the target storage node (a1). The target storage node can update the cache type set by the user to the configuration file (a2), and then return success to the display terminal of the cloud storage system management page (a3).

In addition, in the embodiments of the present disclosure, a process for the target storage node to obtain the target value is also provided:

As shown in Figure 4, the user can click the threshold setting option on the cloud storage system management page displayed on the terminal, and the terminal will display a threshold input box, and the user can enter the target value. Then the user can click the OK option to trigger the terminal to send the threshold setting information (the setting information carries the target value) (b1). The management node inserts the target value into the database of the management node (b2), and updates the target value to the zookeeper node (b3). The target storage node can monitor the zookeeper node, obtain the target value, write it into the memory, and write it to its configuration file (b4). In addition, other management nodes can synchronize the target value to their own memory and database by monitoring the target value of the management node.

In addition, in the embodiment of the present disclosure, a process for the target storage node to obtain an indication message of whether the image resource pool supports acceleration is also provided:

As shown in Figure 5, the user can click the acceleration policy setting option on the cloud storage system management page of the terminal to trigger the terminal to display the image resource pool for which the acceleration policy is not set, and find the image resource pool to which the target storage node belongs, as the target Select the acceleration strategy for the image resource pool to which the storage node belongs. Then the user can click the confirmation option to trigger the terminal to generate acceleration indication information, which can be used to indicate any acceleration strategy of acceleration support and non-support acceleration. Then the terminal sends the acceleration instruction information to the management node (c1), and the management node can insert the acceleration strategy in the acceleration instruction information into the database of the management node (c2), and update the acceleration strategy to the zookeeper node (c3). The target storage node can monitor the zookeeper node, obtain the acceleration strategy, write to the memory, and write to its own configuration file (c4). In addition, other management nodes can synchronize the acceleration strategy to their own memory and database by monitoring the acceleration strategy of the management node.

Optionally, the above-mentioned management node receives the acceleration indication information and the target value through CLI (command-line interface) interface. Similarly, when the above-mentioned target storage node receives the cache type, it also uses CLI to receive it. of.

In addition, in the embodiments of the present disclosure, the cache type of the picture resource pool can also be modified. The user can select the image resource pool to which the target storage node belongs in the cloud storage system management page of the terminal, and re-select the acceleration strategy for the image resource pool to which the target storage node belongs. Then the user can click the confirmation option to trigger the terminal to generate update acceleration indication information, which can be used to indicate the modified acceleration strategy. Then the terminal sends the update acceleration instruction information to the management node, and the management node can use the acceleration strategy in the update acceleration instruction information to update the currently existing acceleration strategy. The target storage node can monitor the zookeeper node, obtain the updated acceleration strategy, write the updated acceleration strategy into the memory, and update the existing acceleration strategy in its configuration file.

Optionally, as shown in FIG. 6, in the embodiment of the present disclosure, processing of reading the target picture is also provided, and the corresponding processing flow may be as follows:

Step 601: Receive a read request for a target picture sent by the requesting party, where the read request carries index information of the target picture stored on the disk.

In implementation, when the target picture is to be viewed, the target picture can be read, and the user viewing the target picture can operate the terminal (that is, the requester) to send a read request of the target picture to the target storage node. The read request carries the index information of the target picture stored in the disk, which may be the identifier of the disk and the identifier of the block in the disk.

The target storage node may receive the read request of the target picture, and parse the index information of the target picture stored on the disk, that is, the identifier of the stored disk and the identifier of the block in the disk.

Step 602: Determine whether the picture resource pool to which the target storage node belongs supports accelerated writing. If the picture resource pool to which the target storage node belongs supports accelerated writing, read the target picture according to the cache type and read request corresponding to the target picture.

In implementation, the target storage node can obtain the corresponding relationship between the storage node and the picture resource pool, and from the corresponding relationship, determine the picture resource pool to which the target storage node belongs, or the configuration information of the target storage node includes the picture resource to which it belongs The identification of the pool, the target storage node can directly obtain the picture resource pool to which it belongs from the configuration information. Then obtain the configuration file of the picture resource pool, obtain the write flag of the picture resource pool from the configuration file, and determine the picture resource pool according to the value of the write flag if the value is the first value Support accelerated writing. If the value is the second value, it is determined that the image resource pool does not support accelerated writing.

If the image resource pool to which the target storage node belongs supports accelerated writing, the target storage node can obtain the configuration item of the cache type in the configuration file, and determine the cache type according to the value of the configuration item. The cache type is no acceleration, memory acceleration, and SSD In any of the acceleration, the cache type in the configuration file is determined as the cache type corresponding to the target picture (this process has been discussed above, and will not be repeated here). For example, if the value of the configuration item is 0, the cache type is determined to be no acceleration, if the value of the configuration item is 1, the cache type is determined to be SSD acceleration, and if the value of the configuration item is 2, the cache type is determined to be memory acceleration.

The target storage node can use the cache type corresponding to the target picture and the index information of the target picture stored on the disk to read the target picture from the target storage node.

Optionally, the target image can be read as follows:

If the cache type corresponding to the target image is memory acceleration, the first index of the target image stored in the memory of the target storage node is determined according to the index information. If the first index is found in the index table corresponding to the memory of the target storage node, According to the first index, the target picture is read from the memory of the target storage node; if the first index is not found in the index table corresponding to the memory of the target storage node, the target picture is read from the disk. If the target picture corresponds to If the cache type of is SSD acceleration, the second index of the target image stored in the SSD is determined according to the index information. If the second index is found in the index table corresponding to the SSD configured on the target storage node, the second index is used from The target image is read from the SSD configured on the target storage node. If the second index is not found in the index table corresponding to the SSD configured on the target storage node, the target image is read from the disk according to the read request.

In implementation, if the cache type corresponding to the target picture is memory acceleration, you can use the index information to find the correspondence between the index value and the identifier of the block stored in the picture on the disk, and determine the target picture stored in the memory of the target storage node. The first index, and then look for the first index in the index table corresponding to the memory of the target storage node. If the first index exists, the target image can be read at the position corresponding to the first index in the memory of the target storage node .

If the first index does not exist in the index table corresponding to the memory of the target storage node, you can directly read the target image from the disk, using the disk identifier and block identifier (in this case, the target image is generally in the target storage node). The storage time in the configured memory is relatively long and has been deleted).

If the cache type corresponding to the target image is not accelerated, the target image can be read directly from the disk using the disk identifier and the block identifier.

If the cache type corresponding to the target image is SSD acceleration, the corresponding relationship between the index value and the identifier of the block stored in the disk of the image can be found according to the index information, and the second index of the target image stored in the SSD of the target storage node can be determined. If the second index can be found in the index table of the SSD of the target storage node, the target image can be read at the position corresponding to the second index in the SSD of the target storage node.

If the second index cannot be found in the index table of the SSD of the target storage node, the disk identifier and block identifier in the index information can be used to read the target picture (in this case, the target picture is generally stored in the target storage). The storage time in the SSD of the node configuration is relatively long and has been deleted).

It should be noted that the read request carries a MYSQL (Structured Query Language, structured query language) database index (that is, the identifier of the disk and the identifier of the block), which is used to locate the position of the image data on the disk. The first index is generated when the data of the picture is stored in the memory, and can be called a memory index, which is used to locate the address of the data of the picture in the memory. The second index is generated when the data of the picture is stored in the SSD, and can be called a DB (databases, database) index, which is used to locate the address of the data of the picture in the SSD.

In addition, it is also possible to first determine whether the cache type corresponding to the target image is memory acceleration. If it is memory acceleration, then read the target image in the memory, if it reads the target image, then go to step 603, if the target image is not read, then read the target image from the disk, if it is not memory acceleration, then The target picture is read in the SSD. If the target picture is read, step 603 is executed. If the target picture is not read, the target picture is read from the disk.

Step 603: Send the target picture to the requesting party.

In implementation, after the target storage node reads the target picture, it can send the target picture to the requesting party.

In addition, in the process shown in Figure 6, it can be executed by an application on the target storage node. In the application, there are two processes, one process is A process, the other process can be B process, A process It is connected to process B through a unix domain socket link. In the process shown in Figure 6, process A can receive the requester's read request and notify process B through the unix domain socket link. Process B can perform the above steps 602 to 603, and then send the read target picture to the A process through the unix domain socket link, and the A process can feed back to the requester. Here, the unix domain socket link established by the A process and the B process can be connected all the time, and through the heartbeat message, it is detected whether the unix domain socket link is in a connected state. If it is not in a connected state, a unix domain socket link is established. If it is in the connected state, continue to use the heartbeat message to detect whether the unix domain socket link is in the connected state

In the embodiment of the present disclosure, the storage node in the cloud storage system is configured with a disk, a memory, and an SSD. When writing a target picture in the cloud storage system, the target storage node can receive the write request of the target picture and determine whether the picture resource pool to which the target storage node belongs supports accelerated writing, and if the picture resource pool to which the storage node belongs supports accelerated writing , The data amount of the target picture and the size of the target value are judged. If the data amount of the target picture is less than the target value, the configured cache type is obtained from the configuration information of the storage node. If the cache type is memory acceleration, the target storage node writes the target image into the memory of the target storage node. If the cache type is SSD acceleration, the target storage node writes the target image into the SSD configured by the target storage node. In this way, when the amount of data of the picture is relatively small, the picture can be stored in the memory or SSD. Since the writing speed of the memory or SSD is faster, when the amount of data of the picture is relatively small, the writing speed of the picture can be relatively improved. Improve picture writing efficiency. And the picture is stored in the memory or SSD, and the picture can be read from the memory or SSD. Since the reading speed of the memory or SSD is faster, the reading efficiency of the picture can also be improved.

In addition, the embodiment of this application also provides a process of reading pictures: the target storage node receives the read request of the target picture sent by the requester, and judges whether the picture resource pool to which the target storage node belongs supports accelerated writing; if the target storage node The image resource pool to which the node belongs supports accelerated writing, and the cache type is memory acceleration. According to the read request, the target image is read in the memory configured by the target storage node, where the cache type is that the target image is written in the target storage node If the image resource pool to which the target storage node belongs supports accelerated writing, and the cache type is SSD acceleration, then according to the read request, read the target image from the SSD configured on the target storage node; send the target to the requester image.

In the embodiment, after the target storage node receives the read request of the target picture sent by the requester, it determines whether the picture resource pool to which the target storage node belongs supports accelerated writing, and determines the cache type as described in the foregoing. If the image resource pool to which the target storage node belongs supports accelerated writing, and the cache type is memory acceleration, you can use a read request to read the target image in the memory. If the image resource pool to which the target storage node belongs supports accelerated writing, And the cache type is SSD acceleration, then the target image is read in the SSD according to the read request. After reading the target picture, the target storage node sends the target picture to the requester. In this way, when the amount of data of the picture is relatively small, the picture can be stored in the memory or SSD, and the picture can be stored in the memory or SSD, so the picture can be read from the memory or SSD. Since the reading speed of the memory or SSD is faster, Can improve the efficiency of image reading.

Optionally, the read request carries index information of the target image stored on the disk; according to the index information, determine the first index of the target image stored in the memory of the target storage node; if it is in the index table corresponding to the memory of the target storage node If the first index is found in, the target image is read from the memory of the target storage node according to the first index. This process has been explained in the previous article, so I won't repeat it here.

Optionally, the read request carries index information of the target image stored on the disk; according to the index information, determine the second index of the target image stored in the SSD; if it is found in the index table corresponding to the SSD configured on the target storage node The second index reads the target picture from the SSD configured on the target storage node according to the second index. This process has been explained in the previous article, so I won't repeat it here.

In addition, if the target storage node does not find the target image in the memory and SSD configured on the target storage node, the target image can be read from the disk configured by the target storage node. The detailed process is described in the foregoing and will not be repeated here.

Based on the same technical concept, the embodiments of the present disclosure also provide a device for writing pictures, the device is applied to a target storage node of a cloud storage system, and the target storage node is configured with a disk, a memory, and an SSD. As shown in Figure 7, the device includes:

The receiving module 710 is configured to receive a write request of the target picture;

The judging module 720 is configured to judge whether the image resource pool to which the target storage node belongs supports accelerated writing;

The determining module 730 is configured to obtain the configured cache type from the configuration information of the storage node when the image resource pool to which the target storage node belongs supports accelerated writing, if the data amount of the target picture is less than the target value , Wherein the cache type is the cache type of the target picture when the target storage node writes;

The writing module 740 is configured to write the target picture into the memory of the target storage node if the cache type is memory acceleration; if the cache type is SSD acceleration, write the target picture The SSD of the target storage node.

Optionally, the writing module 740 is also used for:

Writing the target picture to the disk of the target storage node.

Optionally, the writing module 740 is configured to:

Optionally, the receiving module 710 is further configured to receive a read request of a target picture sent by a requesting party, wherein the read request carries index information of the target picture stored on a disk;

The judgment module 720 is further configured to judge whether the image resource pool to which the target storage node belongs supports accelerated writing;

As shown in Figure 8, the device further includes:

A reading module 750, configured to read the target picture according to the cache type corresponding to the target picture and the read request if the picture resource pool to which the target storage node belongs supports accelerated writing;

The sending module 760 is configured to send the target picture to the requester.

Optionally, the reading module 750 is configured to:

In the embodiment of the present disclosure, the storage node in the cloud storage system is configured with a disk, a memory, and an SSD. When writing a target picture in the cloud storage system, the target storage node can receive the write request of the target picture and determine whether the picture resource pool to which the target storage node belongs supports accelerated writing, and if the picture resource pool to which the storage node belongs supports accelerated writing , The data amount of the target picture and the size of the target value are judged. If the data amount of the target picture is less than the target value, the configured cache type is obtained from the configuration information of the storage node. If the cache type is memory acceleration, the target storage node writes the target image into the memory of the target storage node. If the cache type is SSD acceleration, the target storage node writes the target image into the SSD configured by the target storage node. In this way, when the amount of data of the picture is relatively small, the picture can be stored in the memory or SSD. Since the writing speed of the memory or SSD is faster, when the amount of data of the picture is relatively small, the writing speed of the picture can be relatively improved. Improve writing efficiency. And the picture is stored in the memory or SSD, and the picture can be read from the memory or SSD. Since the reading speed of the memory or SSD is faster, the reading efficiency of the picture can also be improved.

It should be noted that when the device for writing pictures provided in the above embodiments writes pictures, only the division of the above-mentioned functional modules is used as an example for illustration. In actual applications, the above-mentioned functions can be allocated to different functional modules according to needs. Complete, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above. In addition, the device for writing a picture provided in the above-mentioned embodiment belongs to the same concept as the embodiment of the method for writing a picture, and the specific implementation process is detailed in the method embodiment, and will not be repeated here.

FIG. 9 is a schematic structural diagram of a storage node provided by an embodiment of the present disclosure. The storage node 900 may have relatively large differences due to different configurations or performance, and may include one or more processors (central processing units, CPU) 901 And one or more memories 902, wherein at least one instruction is stored in the memory 902, and the at least one instruction is loaded and executed by the processor 901 to implement the above-mentioned method steps of determining to write a picture.

The embodiments of the present disclosure also provide a storage node for writing pictures, the storage node includes a processor and a memory, and at least one instruction is stored in the memory, and the instruction is loaded and executed by the processor to realize The above-mentioned method of writing pictures.

The embodiment of the present disclosure also provides a computer-readable storage medium, in which at least one instruction is stored, and the instruction is loaded and executed by a processor to implement the method for writing a picture as described above.

A person of ordinary skill in the art can understand that all or part of the steps in the above embodiments can be implemented by hardware, or by a program to instruct relevant hardware. The program can be stored in a computer-readable storage medium. The storage medium mentioned can be a read-only memory, a magnetic disk or an optical disk, etc.

The above descriptions are only preferred embodiments of the present disclosure and are not intended to limit the present disclosure. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present disclosure shall be included in the protection of the present disclosure. Within range.

Claims

A method for writing pictures, characterized in that the method is applied to a target storage node in a cloud storage system, and the target storage node is configured with a disk, a memory, and a solid-state hard drive (SSD); the method includes:

The target storage node receives the write request of the target picture, and determines whether the picture resource pool to which the target storage node belongs supports accelerated writing;

When the picture resource pool to which the target storage node belongs supports accelerated writing, if the data amount of the target picture is less than the target value, the configured cache type is obtained from the configuration information of the storage node, wherein the cache Type is the cache type of the target picture when the target storage node writes;

If the cache type is memory acceleration, the target storage node writes the target picture into the memory of the target storage node;

If the cache type is SSD acceleration, the target storage node writes the target picture into the SSD of the target storage node.
The method according to claim 1, wherein the method further comprises:

Writing the target picture to the disk of the target storage node.
The method according to claim 1, wherein the writing the target picture into the memory of the target storage node comprises:

If the current memory margin of the target storage node is greater than or equal to the data amount of the target picture, writing the target picture into the memory of the target storage node;

If the current memory margin of the target storage node is less than the data amount of the target picture, deleting the picture whose writing time point is the longest from the current time point in the memory;

If the memory margin after deleting the picture is greater than or equal to the data amount of the target picture, then the picture is written into the memory, and if the memory margin after deleting the picture is less than the data amount of the target picture, execute The process of deleting the picture whose writing time point is the longest from the current time point in the memory.
The method according to any one of claims 1 to 3, wherein the method further comprises:

The target storage node receives the read request of the target picture sent by the requester, and determines whether the picture resource pool to which the target storage node belongs supports accelerated writing, wherein the read request carries the target picture Index information stored on disk;

If the picture resource pool to which the target storage node belongs supports accelerated writing, read the target picture according to the cache type corresponding to the target picture and the read request;

Sending the target picture to the requesting party.
The method according to claim 4, wherein the reading the target picture according to the cache type corresponding to the target picture and the read request comprises:

If the cache type corresponding to the target picture is memory acceleration, then according to the index information, determine the first index of the target picture stored in the memory of the target storage node; if it corresponds to the memory of the target storage node If the first index is found in the index table of the target storage node, the target picture is read from the memory of the target storage node according to the first index; if it is in the index table corresponding to the memory of the target storage node If the first index is not found, read the target picture from the disk;

If the cache type corresponding to the target picture is SSD acceleration, the second index of the target picture stored in the SSD is determined according to the index information, and if the index table corresponding to the SSD configured on the target storage node is searched To the second index, read the target picture from the SSD configured on the target storage node according to the second index; if it is not found in the index table corresponding to the SSD configured on the target storage node The second index reads the target picture from the disk according to the read request.
A device for writing pictures, characterized in that the device is applied to a target storage node in a cloud storage system, and the target storage node is configured with a magnetic disk, a memory, and a solid-state hard drive (SSD); the device includes:

The receiving module is used to receive the write request of the target picture;

A judging module for judging whether the image resource pool to which the target storage node belongs supports accelerated writing;

The determining module is configured to obtain the configured cache type from the configuration information of the storage node when the image resource pool to which the target storage node belongs supports accelerated writing, if the data amount of the target picture is less than the target value, Wherein, the cache type is the cache type of the target picture when the target storage node writes;

A writing module, configured to write the target picture into the memory of the target storage node if the cache type is memory acceleration; if the cache type is SSD acceleration, write the target picture into the memory In the SSD of the target storage node.
The device according to claim 6, wherein the writing module is further used for:

Writing the target picture to the disk of the target storage node.
The device according to claim 7, wherein the writing module is configured to:

If the current memory margin of the target storage node is greater than or equal to the data amount of the target picture, writing the target picture into the memory of the target storage node;

If the current memory margin of the target storage node is less than the data amount of the target picture, deleting the picture whose writing time point is the longest from the current time point in the memory;

If the memory margin after deleting the picture is greater than or equal to the data amount of the target picture, then the picture is written into the memory, and if the memory margin after deleting the picture is less than the data amount of the target picture, execute The process of deleting the picture whose writing time point is the longest from the current time point in the memory.
The device according to any one of claims 6 to 8, wherein the receiving module is further configured to receive a read request for a target picture sent by a requesting party, wherein the read request carries the target Index information of pictures stored on disk;

The judgment module is also used to judge whether the image resource pool to which the target storage node belongs supports accelerated writing;

The device also includes:

A reading module, configured to read the target picture according to the cache type corresponding to the target picture and the read request if the picture resource pool to which the target storage node belongs supports accelerated writing;

The sending module is used to send the target picture to the requester.
The device according to claim 9, wherein the reading module is configured to:

If the cache type corresponding to the target picture is memory acceleration, then according to the index information, determine the first index of the target picture stored in the memory of the target storage node; if it corresponds to the memory of the target storage node If the first index is found in the index table of the target storage node, the target picture is read from the memory of the target storage node according to the first index; if it is in the index table corresponding to the memory of the target storage node If the first index is not found, read the target picture from the disk;

If the cache type corresponding to the target picture is SSD acceleration, the second index of the target picture stored in the SSD is determined according to the index information, and if the index table corresponding to the SSD configured on the target storage node is searched To the second index, read the target picture from the SSD configured on the target storage node according to the second index; if it is not found in the index table corresponding to the SSD configured on the target storage node The second index reads the target picture from the disk according to the read request.
A storage node for writing pictures, comprising a processor and a memory; the memory is used to store at least one instruction; the processor is used to execute at least one instruction stored in the memory to implement The method step of any one of claims 1-5.
A computer-readable storage medium, characterized in that a computer program is stored in the storage medium, and when the computer program is executed by a processor, the method steps according to any one of claims 1 to 5 are realized.
A method for reading pictures, characterized in that the method is applied to a target storage node in a cloud storage system, and the target storage node is configured with a disk, a memory, and a solid-state hard drive (SSD); the method includes:

The target storage node receives the read request of the target picture sent by the requester, and determines whether the picture resource pool to which the target storage node belongs supports accelerated writing;

If the image resource pool to which the target storage node belongs supports accelerated writing, and the cache type is memory acceleration, the target image is read in the memory configured by the target storage node according to the read request, where: The cache type is the cache type of the target picture when the target storage node writes;

If the picture resource pool to which the target storage node belongs supports accelerated writing, and the cache type is SSD accelerated, reading the target picture from the SSD configured on the target storage node according to the read request;

Sending the target picture to the requesting party.
The method according to claim 13, wherein the read request carries index information of the target picture stored on a disk;

The reading the target picture in the memory configured by the target storage node according to the reading request includes:

Determine the first index of the target picture stored in the memory of the target storage node according to the index information;

If the first index is found in the index table corresponding to the memory of the target storage node, the target picture is read from the memory of the target storage node according to the first index.
The method according to claim 13 or 14, wherein the read request carries index information of the target picture stored on a disk;

The reading the target picture in the SSD configured on the target storage node according to the reading request includes:

Determine the second index of the target picture stored in the SSD according to the index information;

If the second index is found in the index table corresponding to the SSD configured on the target storage node, the target picture is read from the SSD configured on the target storage node according to the second index.