CN116150431A - Information processing method, apparatus, electronic device, and computer-readable storage medium - Google Patents

Information processing method, apparatus, electronic device, and computer-readable storage medium Download PDF

Info

Publication number
CN116150431A
CN116150431A CN202111383230.7A CN202111383230A CN116150431A CN 116150431 A CN116150431 A CN 116150431A CN 202111383230 A CN202111383230 A CN 202111383230A CN 116150431 A CN116150431 A CN 116150431A
Authority
CN
China
Prior art keywords
object storage
storage tank
information
target
target object
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111383230.7A
Other languages
Chinese (zh)
Inventor
刘斌
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to CN202111383230.7A priority Critical patent/CN116150431A/en
Publication of CN116150431A publication Critical patent/CN116150431A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/901Indexing; Data structures therefor; Storage structures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/903Querying
    • G06F16/90335Query processing
    • G06F16/90348Query processing by searching ordered data, e.g. alpha-numerically ordered data
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/903Querying
    • G06F16/9035Filtering based on additional data, e.g. user or group profiles
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

The embodiment of the invention discloses an information processing method, an information processing device, electronic equipment and a computer readable storage medium; after receiving an object reading request, the embodiment of the invention identifies target object storage tanks corresponding to object storage tank identifiers carried in the object reading request in a preset object storage tank, then respectively acquires object storage information of candidate objects stored in each target object storage tank, screens out target objects to be read from the candidate objects based on the object storage information to obtain target object information of the target object storage tank, and then sorts the target object information according to the object storage tank identifiers of the target object storage tanks to obtain object response information corresponding to the object reading request; the scheme can improve the processing efficiency of information processing in the process of batch sequential reading of the objects.

Description

Information processing method, apparatus, electronic device, and computer-readable storage medium
Technical Field
The present invention relates to the field of communications technologies, and in particular, to an information processing method, an apparatus, an electronic device, and a computer readable storage medium.
Background
In recent years, with the rapid development of internet technology, object storage has become more and more convenient. In the object storage, the object is stored in an object storage bucket mainly in a cloud storage mode. The information processing task of batch sequential reading of the objects stored in the object storage bucket can be realized through the core interface in the cloud storage. The existing information processing method for batch sequential reading of objects generally adopts a plurality of inquiry requests to read target objects from all objects stored in an object storage bucket, and then returns object indexes of the target objects according to dictionary sequences.
In the research and practice process of the prior art, the inventor of the present invention finds that the objects stored in the object storage bucket are all physically concentrated in the existing information processing method, when the query rate (QPS) per unit time in the object storage bucket exceeds a preset threshold, frequency control occurs, and the reading efficiency of the target object is affected, so that the processing efficiency of the information processing in the process of batch sequential reading of the objects is reduced.
Disclosure of Invention
The embodiment of the invention provides an information processing method, an information processing device, electronic equipment and a computer readable storage medium, which can improve the processing efficiency of information processing in the batch sequential reading process of objects.
An information processing method, comprising:
receiving an object reading request, wherein the object reading request carries at least one object storage tank identifier;
identifying a target object storage tank corresponding to the object storage tank identifier in a preset object storage bucket according to the object reading request;
object storage information of candidate objects stored in each target object storage tank is respectively obtained;
screening target objects to be read from the candidate objects based on the object storage information to obtain target object information of the target object storage tank;
and sequencing the target object information according to the object storage tank identification of the target object storage tank to obtain the object response information corresponding to the object reading request.
Accordingly, an embodiment of the present invention provides an information processing apparatus including:
the receiving unit is used for receiving an object reading request, wherein the object reading request carries at least one object storage tank identifier;
the identification unit is used for identifying a target object storage tank corresponding to the object storage tank identifier in a preset object storage tank according to the object reading request;
the acquisition unit is used for respectively acquiring object storage information of the candidate objects stored in each target object storage groove;
The screening unit is used for screening target objects to be read from the candidate objects based on the object storage information to obtain target object information of the target object storage tank;
and the ordering unit is used for ordering the target object information according to the object storage tank identification of the target object storage tank so as to obtain the object response information corresponding to the object reading request.
Optionally, in some embodiments, the filtering unit may be specifically configured to extract, from the object storage information, an object identifier of the candidate object; identifying an ending object identifier in the object identifiers, wherein the ending object identifier is used for restricting the object identifier range of the target object; and screening target objects to be read from the candidate objects based on the end object identification to obtain target object information of the target object storage tank.
Optionally, in some embodiments, the screening unit may be specifically configured to screen a largest object identifier from the object identifiers to obtain a target object identifier corresponding to each target object storage slot; and screening out the smallest target object identifier from the target object identifiers to obtain an ending object identifier.
Optionally, in some embodiments, the filtering unit may specifically be configured to compare an object identifier of the candidate object stored in each target object storage slot with the end object identifier; screening candidate objects of which the object identifications do not exceed the end object identifications from the candidate objects to obtain target objects to be read; and fusing the object identifiers of the target objects to obtain target object information of the target object storage tank.
Optionally, in some embodiments, the sorting unit may be specifically configured to obtain a target object storage slot identifier of each version included in the target object storage slot; sorting the target object information according to the number of the target object storage tank identifiers; and fusing the target object information based on the ordering information to obtain object response information corresponding to the object reading request.
Optionally, in some embodiments, the sorting unit may be specifically configured to sort, when the number of the target object storage tank identifiers is one, the target object information according to the identifier size of the target object storage tank identifier; and when the number of the target object storage tank identifiers is a plurality of, sorting the target object information based on the versions of the target storage tank identifiers.
Optionally, in some embodiments, the sorting unit may be specifically configured to identify, according to the version of the target storage tank identifier, object information corresponding to each version in the target object information, and obtain current object information of the target storage tank identifier; sorting the current object information according to the identification size of the target storage tank identification; the merging the target object information based on the ordering information to obtain object response information corresponding to the object reading request comprises the following steps: and fusing the current object information based on the ordering information to obtain object response information corresponding to the object request.
Optionally, in some embodiments, the information processing apparatus may further include a storage unit, where the storage unit may be specifically configured to obtain a set of objects to be stored, where the set of objects to be stored includes at least one object to be stored and an original object identifier of the object to be stored; according to the original object identification, an object storage tank corresponding to each object to be stored in the object set to be stored is screened out from object storage tanks in the preset object storage bucket, and a basic object storage tank corresponding to the object to be stored is obtained; and adding the object to be stored to a corresponding basic object storage tank based on the original object identification.
Optionally, in some embodiments, the storage unit may be specifically configured to perform hash computation on the original object identifier to obtain a hash value of the original object identifier; obtaining the number of object storage tanks in the preset object storage barrel, and taking a model of the number of object storage tanks based on the hash value to obtain an object storage tank identification set corresponding to the object set to be stored; and determining a basic object storage tank corresponding to each object to be stored in the object storage tanks in the preset object storage barrel according to the object storage tank identification set.
Optionally, in some embodiments, the storage unit may be specifically configured to screen out a base object storage tank identifier of the base object storage tank from the base object storage tank identifier set; adding the basic object storage tank identifier to the original object identifier of the corresponding object to be stored to obtain an updated object to be stored; and storing the updated object to be stored into a corresponding basic object storage tank.
Optionally, in some embodiments, the identifying unit may specifically be configured to create, according to an object reading request, a query thread group identified for the object storage slot, where the query thread group includes a parallel query thread for each candidate object storage slot in the preset object storage bucket; inquiring the object storage identification in the candidate object storage tank based on the parallel inquiry thread to obtain inquiry response information of the candidate object storage tank; and identifying a target object storage tank corresponding to the object storage tank identification in the candidate object storage tanks according to the query response information.
Optionally, in some embodiments, the identifying unit may be specifically configured to generate an object storage identification query request based on the parallel query thread, and send the object storage identification query request to the candidate object storage slot; when receiving the return information of the candidate storage tank, taking the return information as the query response information of the candidate storage tank; and when the return information of the candidate object storage tank is not received within the preset time or the query error information is not received, returning to execute the step of sending the object storage identification query request to the candidate object storage tank until the return information of the candidate object storage tank is received.
Optionally, in some embodiments, the identifying unit may be specifically configured to identify, in the query response information, a query result of the candidate storage slot; and screening candidate object storage tanks with the object storage tank identification from the candidate object storage tanks based on the query result to obtain a target object storage tank.
Optionally, in some embodiments, the obtaining unit may be specifically configured to screen query response information corresponding to the target object storage slot from the query response information to obtain target query response information;
Identifying object storage information of candidate objects stored in the target object storage tank in the target query response information;
the step of screening the target object to be read from the candidate objects based on the object storage information to obtain target object information in the target object storage tank, includes: when at least one object storage information corresponding to the target object storage tank is extracted, screening target objects to be read from the candidate objects based on the object storage information to obtain target object information in the target object storage tank.
In addition, the embodiment of the invention also provides electronic equipment, which comprises a processor and a memory, wherein the memory stores an application program, and the processor is used for running the application program in the memory to realize the information processing method provided by the embodiment of the invention.
In addition, the embodiment of the invention also provides a computer readable storage medium, which stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor to execute the steps in any information processing method provided by the embodiment of the invention.
After receiving an object reading request, the embodiment of the invention identifies target object storage tanks corresponding to object storage tank identifiers carried in the object reading request in a preset object storage tank, then respectively acquires object storage information of candidate objects stored in each target object storage tank, screens out target objects to be read from the candidate objects based on the object storage information to obtain target object information of the target object storage tank, and then sorts the target object information according to the object storage tank identifiers of the target object storage tanks to obtain object response information corresponding to the object reading request; in the scheme, a plurality of object storage tanks exist in the preset object storage tank, the object storage tanks are identified in the preset object storage tank through the object storage tank identification, and the object to be read is screened out from the object storage tanks and is ordered, so that independent object reading can be carried out on each object storage tank, the QPS of the object storage tank is greatly increased, and therefore, the processing efficiency of information processing in the process of object batch sequential reading can be improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic view of a scenario of an information processing method provided by an embodiment of the present invention;
FIG. 2 is a schematic flow chart of an information processing method according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a parallel query object storage slot identification using a set of query threads according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of object identifiers of objects stored in an object storage slot according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of object identifiers of screening target objects according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of merging object information under multiple versions according to an embodiment of the present invention;
FIG. 7 is another flow chart of an information processing method according to an embodiment of the present invention;
fig. 8 is a schematic diagram of the structure of an information processing apparatus provided by an embodiment of the present invention;
fig. 9 is another schematic structural view of an information processing apparatus provided in an embodiment of the present invention;
fig. 10 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to fall within the scope of the invention.
The embodiment of the invention provides an information processing method, an information processing device, electronic equipment and a computer readable storage medium. The information processing apparatus may be integrated in an electronic device, which may be a server or a terminal.
The server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, network acceleration services (Content Delivery Network, CDN), basic cloud computing services such as big data and an artificial intelligent platform. The terminal may be, but is not limited to, a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, etc. The terminal and the server may be directly or indirectly connected through wired or wireless communication, which is not limited herein.
For example, referring to fig. 1, taking an example that an information processing apparatus is integrated in an electronic device, after receiving an object reading request, the electronic device identifies a target object storage tank corresponding to an object storage tank identifier carried in the object reading request in a preset object storage tank, then obtains object storage information of candidate objects stored in each target object storage tank, screens out target objects to be read from the candidate objects based on the object storage information to obtain target object information in the target object storage tank, and then sorts the target object information according to the object storage tank identifier of the target object storage tank to obtain object response information corresponding to the object reading request, thereby improving processing efficiency of information processing in a process of batch sequential reading of the objects.
The preset object storage bucket can be deployed on the cloud platform, and object response information corresponding to the object reading request can be stored in the cloud platform. Cloud platforms, also referred to as cloud computing platforms, refer to services based on hardware resources and software resources that provide computing, networking, and storage capabilities. Cloud computing (clouding) is a computing model that distributes computing tasks across a large pool of computers, enabling various application systems to acquire computing power, storage space, and information services as needed. The network that provides the resources is referred to as the "cloud". Resources in the cloud are infinitely expandable in the sense of users, and can be acquired at any time, used as needed, expanded at any time and paid for use as needed.
As a basic capability provider of cloud computing, a cloud computing resource pool (cloud platform for short, generally referred to as IaaS (Infrastructure as a Service, infrastructure as a service) platform) is established, in which multiple types of virtual resources are deployed for external clients to select for use.
According to the logic function division, a PaaS (Platform as a Service ) layer can be deployed on an IaaS (Infrastructure as a Service ) layer, and a SaaS (Software as a Service, software as a service) layer can be deployed above the PaaS layer, or the SaaS can be directly deployed on the IaaS. PaaS is a platform on which software runs, such as a database, web container, etc. SaaS is a wide variety of business software such as web portals, sms mass senders, etc. Generally, saaS and PaaS are upper layers relative to IaaS.
The following will describe in detail. The following description of the embodiments is not intended to limit the preferred embodiments.
The present embodiment will be described from the viewpoint of an information processing apparatus which can be integrated in an electronic device which can be a server or a terminal or the like; the terminal may include a tablet computer, a notebook computer, a personal computer (PC, personal Computer), a wearable device, a virtual reality device, or other devices such as an intelligent device that can read objects in a batch order.
An information processing method, comprising:
receiving an object reading request, wherein the object reading request carries at least one object storage tank identifier, identifying target object storage tanks corresponding to the object storage tank identifiers in a preset object storage tank according to the object reading request, respectively acquiring object storage information of candidate objects stored in each target object storage tank, screening target objects to be read from the candidate objects based on the object storage information to obtain target object information of the target object storage tanks, and sequencing the target object information according to the object storage tank identifiers of the target object storage tanks to obtain object response information corresponding to the object reading request.
As shown in fig. 2, the specific flow of the information processing method is as follows:
101. an object read request is received.
The receiving object reading request is used for batch reading of object information arranged according to a certain sequence. The received object request carries at least one object storage slot identification. The object slot identifier is used to indicate an object slot in the preset object storage bucket, or may be an identifier contained in an object stored in the object slot in the preset object storage bucket.
The manner of receiving the object reading request may be various, and specifically may be as follows:
for example, the object reading request generated by triggering the object storage tank identifier input by the user through the terminal may be directly received, or when the number of the object storage tank identifiers is large, the object reading request sent by the terminal or the third party server may also be received, where the object reading request carries a storage address of the object storage tank identifier, and the object storage tank identifier is obtained based on the storage address.
102. And identifying a target object storage tank corresponding to the object storage tank identifier in a preset object storage tank according to the object reading request.
The preset object storage bucket can be a carrier stored by the preset object, is similar to a table of a database, is a "container" for storing the object, and has no capacity upper limit. The preset object storage bucket comprises a plurality of object storage tanks, at least one object is independently stored in each object storage tank, and the names of the objects stored in the object storage tanks all carry the object storage tank identification specific to the object storage tank.
The manner of identifying the target object storage slot corresponding to the object storage slot identifier in the preset object storage bucket may be various, and may specifically be as follows:
for example, a query thread group for object storage tank identifiers can be created according to the object reading request, and the object storage identifiers are queried in candidate object storage tanks in a preset object storage bucket based on parallel query threads in the query thread group, so as to obtain query response information of the candidate object storage tanks. And identifying a target object storage tank corresponding to the object storage tank identification in the candidate object storage tanks according to the query response information.
Wherein the query thread group includes parallel query threads for each candidate storage slot in the preset object store. The form of the query thread group may be various, for example, may be an Actor thread group, one Actor thread group may include a plurality of parallel query threads (ListReq), and the Actor thread group may perform multi-line Cheng Piliang to initiate a query request, and query whether an object storage tank identifier exists in the candidate object storage tank through the query request, so as to obtain query response information returned by the candidate object storage tank, which may be specifically shown in fig. 3. There are various ways of querying the object storage identity in the candidate storage tank based on the parallel query thread, for example, the object storage identity query request may be generated based on the parallel query thread, and the object storage identity query request is sent to the candidate storage tank, when the return information of the candidate storage tank is received, the return information is used as the query response information of the candidate storage tank, when the return information of the candidate storage tank is not received within a preset time, or when the query error information is not received, the step of sending the object storage identity query request to the candidate storage tank is performed in a returning manner until the return information of the candidate storage tank is received, and the return information is used as the query response information of the candidate storage tank.
It should be noted that, in the process of querying the object storage identifier through the parallel query thread, a "one-way retry mechanism" is adopted to perform timeout and service return monitoring on all the one-way parallel query threads, so when the return information of the candidate object storage tank is received within a preset time, or no query error information is received, the candidate object storage tank with the above problem is retried to resend the object storage identifier query request until the return information of the candidate object storage tank is received, the preset time may be any time, for example, may be 3s or other time, because 99% of the object storage identifier query requests will receive the return information of the candidate object storage tank within 100ms, and the maximum timeout after the retry is 50s or other time. The query error information may be understood as another explicit error code or the like that explicitly indicates that there is an abnormality in the candidate storage slot. The method has better fault tolerance capability for delay Shi Mao caused by network jitter, disk jitter and the like through the 'one-way retry mechanism'.
After obtaining the query response information of the candidate storage tank, the target object storage tank corresponding to the object storage tank identifier can be identified in the candidate storage tank according to the query response information, and various manners of identifying the target object storage tank can be adopted, for example, the query result of the candidate storage tank can be identified in the query response information, and the candidate storage tank only identified by the object storage is screened out from the candidate storage tank based on the query result, so that the target object storage tank is obtained.
The query result is used to indicate whether the object storage tank identifier exists in the candidate object storage tank, and various manners of identifying the query result of the candidate object storage tank in the query response information may be used, for example, the query response information may be used to query the object storage tank in the query response information, when the object storage information exists, the query result of the candidate object storage tank may be determined to be the object storage identifier, and when the object storage information does not exist, the query result of the candidate object storage tank may be determined to be the object storage tank identifier which does not exist.
Optionally, before identifying the target object storage slot corresponding to the object storage slot identifier in the preset object storage bucket according to the object reading request, the object to be stored may be stored in each object storage slot of the preset object storage bucket, and the storage manner may be various, for example, an object set to be stored may be obtained, where the object set to be stored includes at least one object to be stored and an original object identifier of the object to be stored, according to the original object identifier, an object storage slot corresponding to each object to be stored in the object set to be stored is selected from the object storage slots in the preset object storage bucket, so as to obtain a basic object storage slot corresponding to the object to be stored, and based on the original object identifier, the object to be stored is added to the corresponding basic object storage slot.
The method includes that an original object identifier of an object to be stored can be an original object name of the object to be stored, a plurality of modes for screening object storage tanks corresponding to each object to be stored in a set of objects to be stored in a preset object storage bucket can be adopted based on the original object identifier, for example, hash calculation can be carried out on the original object identifier to obtain a hash value of the original object identifier, the number of the object storage tanks in the preset object storage bucket is obtained, the number of the object storage tanks is modulo based on the hash value, an object storage tank identifier set corresponding to the set of the object to be stored is obtained, and a basic object storage tank corresponding to each object to be stored in the object storage tanks in the preset object storage bucket is determined according to the object storage tank identifier set.
The object storage tank identifier set may be understood as a set formed by object storage tank identifiers corresponding to each object to be stored in the object set to be stored. The method of determining the basic object storage tank corresponding to each object to be stored in the object storage tank in the preset object storage barrel according to the object storage tank identification set may be various, for example, the basic object storage identification of the basic object storage tank is screened out from the basic object storage tank identification set, the basic object storage tank identification is added to the original object identification of the corresponding object to be stored, the updated object to be stored is obtained, and then the updated object to be stored is stored in the corresponding basic object storage tank.
It should be noted that, by performing hash calculation on the object identifier of the object to be stored and taking a modulus of the number of object storage slots based on the hash value, the object to be stored in the object set to be stored can be uniformly stored in different object storage slots in the preset object storage bucket, and taking the preset object storage bucket containing 32 object storage slots as an example, when performing batch sequential reading of the objects, the request quantity for continuous object identifiers can be uniformly scattered on different 32 processes and disks, so that the QPS uploaded and downloaded by a single storage bucket is improved by 32 times. In addition, the object storage tank identifier added in the object identifier of each object to be stored may be understood as a slot number prefix, through which the object storage tank stored by the object to be stored may be determined, and when the object to be stored is stored, the slot number prefixes in one object storage tank may be the same or different, but the slot number prefixes in different object storage tanks are different, and since the slot number prefixes are obtained by performing hash calculation based on the original object identifier, the object storage tank may also be referred to as a hash bucket, and in each version, the naming rule of the slot number prefix of the object to be stored in the version may be the same, but the naming rule of the slot number prefix of different versions may be the same or different, and thus, there may be objects in the same object storage tank (hash bucket) that contain the object storage tank identifier (slot number prefix) under different naming rules. The object slot identification (slot number prefix) may be an inherent identification of the object slot (hash bucket), or may be a sequential identification indicating the order of objects to be stored in the set of objects to be stored, or other identification. Taking the set of objects to be stored as the jpg image set as an example, the prefix of the slot number of the objects to be stored in the object storage slot may be as shown in fig. 4, and the identifier may determine the corresponding object storage slot (hash bucket).
103. Object storage information of candidate objects stored in each target object storage tank is acquired respectively.
The object storage information may be an object list of candidate objects stored in the storage object slot or other storage information.
The manner of respectively obtaining the object storage information of the candidate object stored in each target object storage slot may be various, and specifically may be as follows:
for example, query response information corresponding to the target object storage tank may be screened from the query response information to obtain target query response information, and object storage information of the candidate object stored in the target object storage tank is identified in the target query response information.
104. And screening target objects to be read from the candidate objects based on the object storage information to obtain target object information in a target object storage tank.
The target object information may be information after the object indexes of the target objects to be read in the target object storage tank are ordered according to the order in the target object storage tank.
The method for screening the target object to be read from the candidate objects based on the object storage information may be various, and specifically may be as follows:
For example, object identifiers of candidate objects are extracted from object storage information, end object identifiers are identified in the object identifiers, and target objects to be read are screened out of the candidate objects based on the end object identifiers, so that target object information of a target object storage tank is obtained.
The end object identifier is used for restricting the object identifier range of the target object, and taking the object identifiers of the candidate objects as 52, 25, 71 and 59 as examples, and the calculated end object identifier is 59, the object identifier not smaller than the end object identifier can be screened from the object identifiers, so that the object identifiers of the target object can be 52, 25 and 59. The manner of identifying the ending object identifier in the object identifiers may be various, for example, the largest object identifier is screened out from the object identifiers to obtain the object identifier corresponding to each target object storage tank, and the smallest object identifier is screened out from the object identifiers to obtain the ending object identifier, which may be specifically shown in formula (1):
endmarker=min { max (object identification of candidate object in target object slot) } (1)
The endmarker is an ending object identifier, and the object identifier of the candidate object in the target object storage tank may be an original object identifier of the candidate object, or may be a basic object identifier after the object storage tank identifier is added to the original object identifier.
After the end object identifier is identified, the target object to be read can be screened out from the candidate objects based on the end object identifier, and various screening modes can be adopted, for example, the object identifier of the candidate object stored in each target object storage tank can be compared with the end object identifier, the candidate object whose object identifier does not exceed the end object identifier is screened out from the candidate objects, the target object to be read is obtained, and the object identifiers of the target objects are fused, so that the target object information of the target object storage tank is obtained.
In the process of screening the target objects, the object identifiers of the candidate objects need to be compared with the end object identifiers, so that the object identifiers of the target objects are screened, the number of target object storage tanks is 4, the object identifiers of the candidate objects stored in each target object storage tank are shown in fig. 5, the end object identifier is 59, the screening end line is identified in the candidate objects, the candidate objects corresponding to the object identifiers before the screening end line can be the target objects, it can be found that the object identifiers of the target objects to be read in the first target object storage tank are 5, 16, 49 and 52, the object identifiers of the target objects to be read in the second target object storage tank are 7, 12 and 25, the object identifiers of the target objects to be read in the third target object storage tank can be 29, 38 and 57, and the object identifiers of the target objects to be read in the fourth target object storage tank can be 9, 22, 47, 48 and 59.
The method of fusing the object identifiers of the target objects may also be various, for example, the object identifiers of the screened target objects to be read may be sorted in the target object storage tank, and the target object identifiers may be combined based on the sorting order to obtain the target object information of the target object storage tank.
It should be noted that, the condition of screening the target object in the candidate objects is that when the object storage information corresponding to at least one target object storage tank is extracted, the target object can be screened, that is, when all the object storage information corresponding to all the target object storage tanks is returned, the target object is not required to be screened, but one object storage information is returned, and the target object is screened from the candidate objects based on the object storage information, so that the CPU calculation in the sorting process is smoother.
105. And ordering the target object information according to the object storage tank identification of the target object storage tank to obtain object response information corresponding to the object reading request.
The object response information may be understood as a plurality of object indexes with sequences that are read in batches within a preset time corresponding to the object reading request, where the number of the object indexes may be the number carried in the object reading request or the maximum return number set by the system.
The sorting manner of the target object information according to the object storage tank identifier of the target object storage tank may be various, and may specifically be as follows:
for example, the target object storage tank identifier of each version contained in the target object storage tank may be obtained, the target object information may be ordered according to the number of the target object storage tank identifiers, and the target object information may be fused based on the ordering information, to obtain the object response information corresponding to the object reading request.
The target object storage tank identifier of each version may be an object storage tank identifier for each object storage tank in a preset object storage bucket, which is generated when a batch of objects to be processed are stored, or may be an object storage tank identifier for each object storage tank in the preset object storage bucket, which is generated when the objects to be stored are stored after one update. When a plurality of target objects exist in one target object storage tank, the object storage tank identifiers of a plurality of versions are contained in the target object storage tank, and when only one target object storage tank identifier exists in one target object storage tank, the object storage identifier of only one version exists in the target object storage tank. Therefore, there may be various ways of sorting the target object information according to the number of the target object storage tank identifications, for example, sorting the target object information according to the identification size of the target object storage tank identifications when the number of the target object storage tank identifications is one, and sorting the target object information based on the version of the target object storage tank identifications when the number of the target object storage tank identifications is a plurality.
The target object information may be sorted according to the identification size of the target object storage slot identification, for example, the target object information may be sorted in descending order or ascending order according to the identification size, so as to obtain the sorting position of each object index of the target object to be read. Based on the ordering information, there may be various ways to fuse the target object information, for example, each object index of the target object may be added to the object index set according to the ordering position, so as to obtain the target object index set, and the target object index set is used as the object response information corresponding to the object reading request.
The method for sorting the target object information according to the version of the target storage tank identifier may be various, for example, the object information corresponding to each version may be identified in the target object information according to the version of the target object storage tank identifier, the current object information of the target object storage tank identifier is obtained, and the current object information is sorted according to the identifier size of the target object storage tank identifier. Based on the sorting information, the current object information may be fused in various ways, for example, the fusion relates to multi-version fusion, and it is generally required to identify the sorting position of the current object information in the sorting information of each version of the current object information, then, under the same version, based on the sorting position, combine the current sorting information to obtain an object index set corresponding to each version, and then, reorder and combine the object index sets of two or more versions according to the size of the object storage slot identifier to obtain a target object index set, and use the target object index set as the object response information corresponding to the object reading request. As shown in fig. 6, two versions of object storage tank identifiers exist in the target object storage tank, the two versions are respectively a current version and a historical version, the object storage tank identifiers of the current version are respectively 1-8, the object storage tank identifiers of the historical version are respectively 2.1, 4.1, 8.1, 97.1 and 98.1, then unified sorting can be performed according to the sizes of the object storage tank identifiers, and sorting and merging are performed based on sorting positions, so that a target object index set of a target object to be read under a plurality of versions is obtained, and the target object index set is used as object response information corresponding to an object reading request. Then, the object response information is returned to the terminal or the third party server.
Optionally, the object reading request may further carry the number of object indexes of the object to be read, and the target object response information is screened out from the object response information according to the number of object indexes, where the screening manner may be multiple, for example, the object indexes with the sorting positions not exceeding the number of object indexes are screened out from the object response information, and the screened object indexes are combined to obtain the target object response information, and then the target object response information is returned to the terminal or the third party server.
It should be noted that the process of merging and sorting the target object information of the target object may be performed simultaneously, or the merging of the target object information may be performed after the sorting is completed, so as to return the object response information or the target object response information.
It should be noted that in the scheme, the physical ordered storage in the preset object storage bucket is scattered through the object storage tanks and is uniformly distributed in each object storage tank, so that the QPS performance of the preset object storage bucket is greatly improved to more than 6 ten thousand, in addition, in the object reading process, multithreading batch reading is adopted, so that the problem of hot spot concentration is effectively solved, the underlying system of the preset object storage bucket is more stable, the problem of hot spot concentration is effectively solved, and the operation efficiency of object storage in cloud storage is greatly improved.
As can be seen from the foregoing, after receiving the object reading request, the embodiment of the present application identifies a target object storage slot corresponding to an object storage slot identifier carried in the object reading request in a preset object storage bucket, then obtains object storage information of candidate objects stored in each target object storage slot, screens out target objects to be read from the candidate objects based on the object storage information, obtains target object information of the target object storage slot, and then orders the target object information according to the object storage slot identifier of the target object storage slot, so as to obtain object response information corresponding to the object reading request; in the scheme, a plurality of object storage tanks exist in the preset object storage tank, the object storage tanks are identified in the preset object storage tank through the object storage tank identification, and the object to be read is screened out from the object storage tanks and is ordered, so that independent object reading can be carried out on each object storage tank, the QPS of the object storage tank is greatly increased, and therefore, the processing efficiency of information processing in the process of object batch sequential reading can be improved.
According to the method described in the above embodiments, examples are described in further detail below.
In this embodiment, the information processing apparatus is specifically integrated in an electronic device, the electronic device is a server, the object storage slots are identified as slot prefixes, and the number of object storage slots in the preset object storage bucket is 32.
As shown in fig. 7, a specific flow of the information processing method is as follows:
201. the server receives an object read request.
For example, the server may directly receive an object reading request generated by triggering a slot prefix input by a user through the terminal, or when the number of slot prefixes is greater, may also receive an object reading request sent by the terminal or a third party server, where the object reading request carries a storage address of the slot prefix, and acquire the slot prefix based on the storage address.
202. And the server identifies a target object storage tank corresponding to the slot prefix in a preset object storage bucket according to the object reading request.
For example, the server may create 8 Actor thread groups for slot prefixes according to the object read request, create 4 parallel query threads (ListReq) per Actor thread group, generate an object store identification query request based on the parallel query threads, and send the object store identification query request to the candidate object storage slot, when receiving return information of the candidate object storage slot, take the return information as query response information of the candidate object, when receiving no return information of the candidate object storage slot within a preset time, or when receiving no query error information, return to perform the step of sending the object store identification query request to the candidate object storage slot until receiving the return information of the candidate object storage slot, and take the return information as query response information of the candidate object storage slot.
The server queries object storage information in the query response information, when the object storage information exists, the query result of the candidate object storage tank can be determined to be the existence object storage identification, and when the object storage information does not exist, the query result of the candidate object storage tank can be determined to be the absence of the slot prefix. And screening candidate object storage tanks which are only identified by the object storage from the candidate object storage tanks based on the query result to obtain target object storage tanks.
Optionally, before identifying the target object storage slot corresponding to the slot prefix in the preset object storage bucket according to the object reading request, the server may further store the object to be stored in each object storage slot of the preset object storage bucket, where the storage manner may be various, and specifically may be as follows:
the server may obtain an object set to be stored, where the object set to be stored includes at least one object to be stored and an original object identifier of the object to be stored, perform hash computation on the original object identifier to obtain a hash value of the original object identifier, and modulo 32 based on the hash value to obtain a slot prefix set (32 slots) corresponding to the object set to be stored. And screening a basic object storage identifier of a basic object storage tank from the basic slot prefix set, adding the basic slot prefix to an original object identifier of a corresponding object to be stored to obtain an updated object to be stored, and then storing the updated object to be stored into the corresponding basic object storage tank.
203. The server obtains object storage information of the candidate objects stored in each target object storage slot respectively.
For example, the server may screen the query response information corresponding to the target object storage slot from the query response information to obtain target query response information, and identify object storage information of the candidate object stored in the target object storage slot in the target query response information.
204. The server extracts the object identification of the candidate object from the object storage information.
For example, the server extracts an object identification list of candidate objects from the object storage information, and determines an object identification of each candidate object in the object identification list.
205. The server identifies an ending object identifier among the object identifiers.
For example, the largest object identifier is screened out from the object identifiers to obtain the object identifier corresponding to each object storage tank, and the smallest object identifier is screened out from the object identifiers to obtain the end object identifier, which can be specifically shown in formula (1).
206. And the server screens out target objects to be read from the candidate objects based on the end object identification to obtain target object information of the target object storage tank.
For example, the server may compare the object identifier of the candidate object stored in each target object storage slot with the end object identifier, screen out candidate objects whose object identifiers do not exceed the end object identifier from the candidate objects, obtain the target object to be read, sort the object identifiers of the screened target objects to be read in the target object storage slot, and combine the target object identifiers based on the sorting order, to obtain the target object information of the target object storage slot.
207. And the server sorts the target object information according to the slot prefix of the target object storage slot so as to obtain object response information corresponding to the object reading request.
For example, the server may obtain the target slot prefix of each version included in the target object storage slot, and when the number of the target slot prefixes is one, sort the target object information in descending order or ascending order according to the size of the identifier, so as to obtain the sort location of each object index of the target object to be read. And adding each object index of the target object to the object index set according to the ordering position to obtain the target object index set, and taking the target object index set as object response information corresponding to the object reading request.
When the number of the target slot prefixes is multiple, the server can identify object information corresponding to each version in the target object information according to the versions of the target slot prefixes, obtain current object information of the target slot prefixes, and sort the current object information according to the identification sizes of the target slot prefixes. And identifying the ordering position of the current object information in the ordering information of each version of the current object information, combining the current ordering information based on the ordering position under the same version to obtain an object index set corresponding to each version, and then ordering and combining the object index sets of two or more versions again according to the size of the slot prefix to obtain a target object index set which is used as object response information corresponding to the object reading request.
Optionally, the object reading request may further carry the number of object indexes of the object to be read, and the server may screen the target object response information from the object response information according to the number of object indexes, where the screening manner may be multiple, for example, the server screens the object indexes whose sorting positions do not exceed the number of object indexes from the object response information, and combines the screened object indexes to obtain the target object response information.
As can be seen from the foregoing, after receiving the object reading request, the server in this embodiment identifies, in the preset object storage bucket, the object storage slots corresponding to the slot prefixes carried in the object reading request, then obtains object storage information of candidate objects stored in each of the object storage slots, screens out, based on the object storage information, the target objects to be read from the candidate objects, and obtains target object information of the target object storage slots, and then orders the target object information according to the slot prefixes of the target object storage slots, so as to obtain object response information corresponding to the object reading request; in the scheme, a plurality of object storage tanks exist in the preset object storage tank, the target object storage tanks are identified in the preset object storage tank through the tank prefix, and target objects to be read are screened out from the target object storage tanks and are ordered, so that independent object reading can be carried out on each object storage tank, the QPS of the object storage tank is greatly increased, and therefore, the processing efficiency of information processing in the process of object batch sequential reading can be improved.
In order to better implement the above method, the embodiment of the present invention further provides an information processing apparatus, where the information processing apparatus may be integrated into an electronic device, such as a server or a terminal, where the terminal may include a tablet computer, a notebook computer, and/or a personal computer.
For example, as shown in fig. 8, the information processing apparatus may include a receiving unit 301, an identifying unit 302, an acquiring unit 303, a screening unit 304, and a sorting unit 305, as follows:
(1) A receiving unit 301;
the receiving unit 301 is configured to receive an object read request, where the object read request carries at least one object storage slot identifier.
For example, the receiving unit 301 may be specifically configured to receive an object read request generated by triggering an object storage tank identifier input by a user through a terminal, or when the number of object storage tank identifiers is greater, may also receive an object read request sent by the terminal or a third party server, where the object read request carries a storage address of the object storage tank identifier, and acquire the object storage tank identifier based on the storage address.
(2) An identification unit 302;
the identifying unit 302 is configured to identify, according to the object reading request, a target object storage slot corresponding to the object storage slot identifier in the preset object storage bucket.
For example, the identifying unit 302 may specifically be configured to create a query thread group for the object storage slot identifier according to the object reading request, query the object storage identifier in the candidate object storage slot in the preset object storage bucket based on the parallel query threads in the query thread group, and obtain query response information of the candidate object storage slot. And identifying a target object storage tank corresponding to the object storage tank identification in the candidate object storage tanks according to the query response information.
(3) An acquisition unit 303;
an obtaining unit 303, configured to obtain object storage information of the candidate objects stored in each target object storage slot, respectively.
For example, the obtaining unit 303 may be specifically configured to screen query response information corresponding to the target object storage slot from the query response information, obtain target query response information, and identify object storage information of the candidate object stored in the target object storage slot in the target query response information.
(4) A screening unit 304;
and the screening unit 304 is configured to screen the target object to be read from the candidate objects based on the object storage information, so as to obtain target object information of the target object storage slot.
For example, the filtering unit 304 may specifically be configured to extract an object identifier of a candidate object from the object storage information, identify an end object identifier from the object identifiers, and screen a target object to be read from the candidate object based on the end object identifier, so as to obtain target object information of the target object storage slot.
(5) A sorting unit 305;
the sorting unit 305 is configured to sort the target object information according to the object storage tank identifier of the target object storage tank, so as to obtain object response information corresponding to the object read request.
For example, the sorting unit 305 may be specifically configured to obtain a target object storage tank identifier of each version included in the target object storage tank, sort the target object information according to the number of target object storage tank identifiers, and fuse the target object information based on the sorting information to obtain object response information corresponding to the object reading request.
Optionally, the information processing apparatus may further include a storage unit 306, as shown in fig. 9, specifically may be as follows:
the storage unit 306 is configured to store each object to be stored in the set of objects to be stored in the object storage slot of the preset object storage bucket respectively.
For example, the storage unit 306 may be specifically configured to obtain a set of objects to be stored, where the set of objects to be stored includes at least one object to be stored and an original object identifier of the object to be stored, screen, according to the original object identifier, an object storage slot corresponding to each object to be stored in the set of objects to be stored from object storage slots in a preset object storage bucket, obtain a basic object storage slot corresponding to the object to be stored, and add the object to be stored to the corresponding basic object storage slot based on the original object identifier.
In the implementation, each unit may be implemented as an independent entity, or may be implemented as the same entity or several entities in any combination, and the implementation of each unit may be referred to the foregoing method embodiment, which is not described herein again.
As can be seen from the foregoing, in this embodiment, after the receiving unit 301 receives the object reading request, the identifying unit 302 identifies the target object storage slots corresponding to the object storage slot identifiers carried in the object reading request in the preset object storage slots, then the obtaining unit 303 obtains the object storage information of the candidate objects stored in each target object storage slot, the screening unit 304 screens the target objects to be read from the candidate objects based on the object storage information to obtain the target object information of the target object storage slots, and then the sorting unit 305 sorts the target object information according to the object storage slot identifiers of the target object storage slots to obtain the object response information corresponding to the object reading request; in the scheme, a plurality of object storage tanks exist in the preset object storage tank, the object storage tanks are identified in the preset object storage tank through the object storage tank identification, and the object to be read is screened out from the object storage tanks and is ordered, so that independent object reading can be carried out on each object storage tank, the QPS of the object storage tank is greatly increased, and therefore, the processing efficiency of information processing in the process of object batch sequential reading can be improved.
The embodiment of the invention also provides an electronic device, as shown in fig. 10, which shows a schematic structural diagram of the electronic device according to the embodiment of the invention, specifically:
the electronic device may include one or more processing cores 'processors 401, one or more computer-readable storage media's memory 402, power supply 403, and input unit 404, among other components. It will be appreciated by those skilled in the art that the electronic device structure shown in fig. 10 is not limiting of the electronic device and may include more or fewer components than shown, or may combine certain components, or a different arrangement of components. Wherein:
the processor 401 is a control center of the electronic device, connects various parts of the entire electronic device using various interfaces and lines, and performs various functions of the electronic device and processes data by running or executing software programs and/or modules stored in the memory 402, and calling data stored in the memory 402, thereby performing overall monitoring of the electronic device. Optionally, processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, an application program, etc., and the modem processor mainly processes wireless communication. It will be appreciated that the modem processor described above may not be integrated into the processor 401.
The memory 402 may be used to store software programs and modules, and the processor 401 executes various functional applications and data processing by executing the software programs and modules stored in the memory 402. The memory 402 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program (such as a sound playing function, an image playing function, etc.) required for at least one function, and the like; the storage data area may store data created according to the use of the electronic device, etc. In addition, memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.
The electronic device further comprises a power supply 403 for supplying power to the various components, preferably the power supply 403 may be logically connected to the processor 401 by a power management system, so that functions of managing charging, discharging, and power consumption are performed by the power management system. The power supply 403 may also include one or more of any of a direct current or alternating current power supply, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and the like.
The electronic device may further comprise an input unit 404, which input unit 404 may be used for receiving input digital or character information and generating keyboard, mouse, joystick, optical or trackball signal inputs in connection with user settings and function control.
Although not shown, the electronic device may further include a display unit or the like, which is not described herein. In particular, in this embodiment, the processor 401 in the electronic device loads executable files corresponding to the processes of one or more application programs into the memory 402 according to the following instructions, and the processor 401 executes the application programs stored in the memory 402, so as to implement various functions as follows:
receiving an object reading request, wherein the object reading request carries at least one object storage tank identifier, identifying target object storage tanks corresponding to the object storage tank identifiers in a preset object storage tank according to the object reading request, respectively acquiring object storage information of candidate objects stored in each target object storage tank, screening target objects to be read from the candidate objects based on the object storage information to obtain target object information of the target object storage tanks, and sequencing the target object information according to the object storage tank identifiers of the target object storage tanks to obtain object response information corresponding to the object reading request.
For example, the electronic device may receive an object reading request generated by triggering the object storage tank identifier input by the user through the terminal, or when the number of the object storage tank identifiers is greater, may also receive an object reading request sent by the terminal or the third party server, where the object reading request carries a storage address of the object storage tank identifier, and acquire the object storage tank identifier based on the storage address. And creating a query thread group aiming at the object storage tank identification according to the object reading request, and querying the object storage identification in the candidate object storage tank in the preset object storage tank based on the parallel query threads in the query thread group to obtain query response information of the candidate object storage tank. And identifying a target object storage tank corresponding to the object storage tank identification in the candidate object storage tanks according to the query response information. And screening query response information corresponding to the target object storage tank from the query response information to obtain target query response information, and identifying object storage information of candidate objects stored in the target object storage tank from the target query response information. Extracting object identification of a candidate object from object storage information, identifying end object identification in the object identification, and screening target objects to be read from the candidate object based on the end object identification to obtain target object information of a target object storage tank. And acquiring target object storage tank identifiers of each version contained in the target object storage tank, sorting the target object information according to the number of the target object storage tank identifiers, and fusing the target object information based on the sorting information to obtain object response information corresponding to the object reading request.
The specific implementation of each operation may be referred to the previous embodiments, and will not be described herein.
As can be seen from the above, after receiving the object reading request, the embodiment of the present invention identifies the object storage slots corresponding to the object storage slot identifiers carried in the object reading request in the preset object storage slots, then respectively obtains the object storage information of the candidate objects stored in each object storage slot, screens out the target objects to be read from the candidate objects based on the object storage information, obtains the target object information of the target object storage slots, and then orders the target object information according to the object storage slot identifiers of the target object storage slots, so as to obtain the object response information corresponding to the object reading request; in the scheme, a plurality of object storage tanks exist in the preset object storage tank, the object storage tanks are identified in the preset object storage tank through the object storage tank identification, and the object to be read is screened out from the object storage tanks and is ordered, so that independent object reading can be carried out on each object storage tank, the QPS of the object storage tank is greatly increased, and therefore, the processing efficiency of information processing in the process of object batch sequential reading can be improved.
Those of ordinary skill in the art will appreciate that all or a portion of the steps of the various methods of the above embodiments may be performed by instructions, or by instructions controlling associated hardware, which may be stored in a computer-readable storage medium and loaded and executed by a processor.
To this end, an embodiment of the present invention provides a computer readable storage medium having stored therein a plurality of instructions capable of being loaded by a processor to perform the steps of any one of the information processing methods provided by the embodiment of the present invention. For example, the instructions may perform the steps of:
receiving an object reading request, wherein the object reading request carries at least one object storage tank identifier, identifying target object storage tanks corresponding to the object storage tank identifiers in a preset object storage tank according to the object reading request, respectively acquiring object storage information of candidate objects stored in each target object storage tank, screening target objects to be read from the candidate objects based on the object storage information to obtain target object information of the target object storage tanks, and sequencing the target object information according to the object storage tank identifiers of the target object storage tanks to obtain object response information corresponding to the object reading request.
For example, an object reading request generated by triggering the object storage tank identifier input by the user through the terminal is received, or when the number of the object storage tank identifiers is large, an object reading request sent by the terminal or the third party server may also be received, where the object reading request carries a storage address of the object storage tank identifier, and the object storage tank identifier is obtained based on the storage address. And creating a query thread group aiming at the object storage tank identification according to the object reading request, and querying the object storage identification in the candidate object storage tank in the preset object storage tank based on the parallel query threads in the query thread group to obtain query response information of the candidate object storage tank. And identifying a target object storage tank corresponding to the object storage tank identification in the candidate object storage tanks according to the query response information. And screening query response information corresponding to the target object storage tank from the query response information to obtain target query response information, and identifying object storage information of candidate objects stored in the target object storage tank from the target query response information. Extracting object identification of a candidate object from object storage information, identifying end object identification in the object identification, and screening target objects to be read from the candidate object based on the end object identification to obtain target object information of a target object storage tank. And acquiring target object storage tank identifiers of each version contained in the target object storage tank, sorting the target object information according to the number of the target object storage tank identifiers, and fusing the target object information based on the sorting information to obtain object response information corresponding to the object reading request.
The specific implementation of each operation above may be referred to the previous embodiments, and will not be described herein.
Wherein the computer-readable storage medium may comprise: read Only Memory (ROM), random access Memory (RAM, random Access Memory), magnetic or optical disk, and the like.
Because the instructions stored in the computer readable storage medium may execute the steps in any information processing method provided by the embodiments of the present invention, the beneficial effects that any information processing method provided by the embodiments of the present invention can be achieved, which are detailed in the previous embodiments and are not described herein.
Among other things, according to one aspect of the present application, a computer program product or computer program is provided that includes computer instructions stored in a computer readable storage medium. The computer instructions are read from a computer-readable storage medium by a processor of a computer device, and executed by the processor, cause the computer device to perform the methods provided in various alternative implementations of the information processing aspects or the object batch sequential reading aspects described above.
The foregoing has described in detail the methods, apparatuses, electronic devices and computer readable storage medium for processing information provided by the embodiments of the present invention, and specific examples have been applied to illustrate the principles and embodiments of the present invention, and the above description of the embodiments is only for aiding in understanding the methods and core ideas of the present invention; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in light of the ideas of the present invention, the present description should not be construed as limiting the present invention.

Claims (18)

1. An information processing method, characterized by comprising:
receiving an object reading request, wherein the object reading request carries at least one object storage tank identifier;
identifying a target object storage tank corresponding to the object storage tank identifier in a preset object storage bucket according to the object reading request;
object storage information of candidate objects stored in each target object storage tank is respectively obtained;
screening target objects to be read from the candidate objects based on the object storage information to obtain target object information of the target object storage tank;
And sequencing the target object information according to the object storage tank identification of the target object storage tank to obtain the object response information corresponding to the object reading request.
2. The information processing method according to claim 1, wherein the selecting, based on the object storage information, a target object to be read from the candidate objects to obtain target object information of the target object storage tank includes:
extracting object identifiers of the candidate objects from the object storage information;
identifying an ending object identifier in the object identifiers, wherein the ending object identifier is used for restricting the object identifier range of the target object;
and screening target objects to be read from the candidate objects based on the end object identification to obtain target object information of the target object storage tank.
3. The information processing method according to claim 2, wherein the identification of the end object identifier among the object identifiers includes:
screening out the largest object identifier from the object identifiers to obtain a target object identifier corresponding to each target object storage tank;
And screening out the smallest target object identifier from the target object identifiers to obtain an ending object identifier.
4. The information processing method according to claim 2, wherein the step of screening the target object to be read from the candidate objects based on the end object identification to obtain target object information of the target object storage tank includes:
comparing the object identification of the candidate object stored in each target object storage tank with the end object identification;
screening candidate objects of which the object identifications do not exceed the end object identifications from the candidate objects to obtain target objects to be read;
and fusing the object identifiers of the target objects to obtain target object information of the target object storage tank.
5. The information processing method according to any one of claims 1 to 4, wherein the sorting the target object information according to the object storage slot identification of the target object storage slot to obtain the object response information corresponding to the object read request includes:
acquiring a target object storage tank identifier of each version contained in the target object storage tank;
Sorting the target object information according to the number of the target object storage tank identifiers;
and fusing the target object information based on the ordering information to obtain object response information corresponding to the object reading request.
6. The information processing method according to claim 5, wherein the sorting the target object information according to the number of target object storage slot identifications comprises:
when the number of the target object storage tank identifiers is one, sorting the target object information according to the identifier size of the target object storage tank identifiers;
and when the number of the target object storage tank identifiers is a plurality of, sorting the target object information based on the versions of the target storage tank identifiers.
7. The information processing method according to claim 6, wherein the sorting the target object information based on the version of the target storage slot identification includes:
identifying object information corresponding to each version in the target object information according to the version of the target storage tank identifier, and obtaining current object information of the target storage tank identifier;
Sorting the current object information according to the identification size of the target storage tank identification;
the merging the target object information based on the ordering information to obtain object response information corresponding to the object reading request comprises the following steps: and fusing the current object information based on the ordering information to obtain object response information corresponding to the object request.
8. The information processing method according to any one of claims 1 to 4, wherein the identifying, according to the object read request, the target object storage slot corresponding to the object storage slot identification in a preset object storage bucket further includes:
acquiring an object set to be stored, wherein the object set to be stored comprises at least one object to be stored and an original object identifier of the object to be stored;
according to the original object identification, an object storage tank corresponding to each object to be stored in the object set to be stored is screened out from object storage tanks in the preset object storage bucket, and a basic object storage tank corresponding to the object to be stored is obtained;
and adding the object to be stored to a corresponding basic object storage tank based on the original object identification.
9. The method of claim 8, wherein the step of screening object storage slots corresponding to each object to be stored in the set of objects to be stored from object storage slots in the preset object storage bucket according to the original object identifier to obtain a basic object storage slot corresponding to the object to be stored includes:
carrying out hash calculation on the original object identifier to obtain a hash value of the original object identifier;
obtaining the number of object storage tanks in the preset object storage barrel, and taking a model of the number of object storage tanks based on the hash value to obtain an object storage tank identification set corresponding to the object set to be stored;
and determining a basic object storage tank corresponding to each object to be stored in the object storage tanks in the preset object storage barrel according to the object storage tank identification set.
10. The information processing method according to claim 9, wherein the adding the object to be stored to the corresponding base object storage slot based on the base object identification includes:
screening out a basic object storage tank identifier of the basic object storage tank from the basic object storage tank identifier set;
Adding the basic object storage tank identifier to the original object identifier of the corresponding object to be stored to obtain an updated object to be stored;
and storing the updated object to be stored into a corresponding basic object storage tank.
11. The method according to any one of claims 1 to 4, wherein the identifying, in a preset object storage bucket, a target object storage slot corresponding to the object storage slot identification according to the object read request includes:
creating a query thread group aiming at the object storage tank identification according to an object reading request, wherein the query thread group comprises parallel query threads aiming at each candidate object storage tank in the preset object storage tank;
inquiring the object storage identification in the candidate object storage tank based on the parallel inquiry thread to obtain inquiry response information of the candidate object storage tank;
and identifying a target object storage tank corresponding to the object storage tank identification in the candidate object storage tanks according to the query response information.
12. The information processing method according to claim 11, wherein the querying the object storage slot for the object storage identifier based on the parallel query thread, to obtain query response information of the candidate object storage slot, includes:
Generating an object storage identification query request based on the parallel query thread, and sending the object storage identification query request to the candidate object storage tank;
when receiving the return information of the candidate storage tank, taking the return information as the query response information of the candidate storage tank;
and when the return information of the candidate object storage tank is not received within the preset time or the query error information is not received, returning to execute the step of sending the object storage identification query request to the candidate object storage tank until the return information of the candidate object storage tank is received.
13. The information processing method according to claim 11, wherein the identifying, from the candidate object storage slots, the object storage slot identification corresponding to the target object storage slot based on the query response information, includes:
identifying a query result of the candidate storage tank in the query response information;
and screening candidate object storage tanks with the object storage tank identification from the candidate object storage tanks based on the query result to obtain a target object storage tank.
14. The information processing method according to claim 12, wherein the acquiring object storage information of the candidate objects stored in each of the target object storage slots, respectively, comprises:
screening out the query response information corresponding to the target object storage tank from the query response information to obtain target query response information;
identifying object storage information of candidate objects stored in the target object storage tank in the target query response information;
the step of screening the target object to be read from the candidate objects based on the object storage information to obtain target object information in the target object storage tank, includes: when at least one object storage information corresponding to the target object storage tank is extracted, screening target objects to be read from the candidate objects based on the object storage information to obtain target object information in the target object storage tank.
15. An information processing apparatus, characterized by comprising:
the receiving unit is used for receiving an object reading request, wherein the object reading request carries at least one object storage tank identifier;
the identification unit is used for identifying a target object storage tank corresponding to the object storage tank identifier in a preset object storage tank according to the object reading request;
The acquisition unit is used for respectively acquiring object storage information of the candidate objects stored in each target object storage groove;
the screening unit is used for screening target objects to be read from the candidate objects based on the object storage information to obtain target object information of the target object storage tank;
and the ordering unit is used for ordering the target object information according to the object storage tank identification of the target object storage tank so as to obtain the object response information corresponding to the object reading request.
16. An electronic device comprising a processor and a memory, the memory storing an application, the processor being configured to run the application in the memory to perform the steps in the information processing method of any one of claims 1 to 14.
17. A computer program product comprising computer programs/instructions which, when executed by a processor, implement the steps of the information processing method of any of claims 1 to 14.
18. A computer readable storage medium storing a plurality of instructions adapted to be loaded by a processor to perform the steps in the information processing method of any one of claims 1 to 14.
CN202111383230.7A 2021-11-22 2021-11-22 Information processing method, apparatus, electronic device, and computer-readable storage medium Pending CN116150431A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111383230.7A CN116150431A (en) 2021-11-22 2021-11-22 Information processing method, apparatus, electronic device, and computer-readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111383230.7A CN116150431A (en) 2021-11-22 2021-11-22 Information processing method, apparatus, electronic device, and computer-readable storage medium

Publications (1)

Publication Number Publication Date
CN116150431A true CN116150431A (en) 2023-05-23

Family

ID=86339411

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111383230.7A Pending CN116150431A (en) 2021-11-22 2021-11-22 Information processing method, apparatus, electronic device, and computer-readable storage medium

Country Status (1)

Country Link
CN (1) CN116150431A (en)

Similar Documents

Publication Publication Date Title
CN109614402B (en) Multidimensional data query method and device
US10187255B2 (en) Centralized configuration data in a distributed file system
CN106716335B (en) Asynchronous processing of mapping information
CN106951179B (en) Data migration method and device
CN107480205B (en) Method and device for partitioning data
CN110347651A (en) Method of data synchronization, device, equipment and storage medium based on cloud storage
CN105871983A (en) Method and device for selecting servers
US20190362016A1 (en) Frequent pattern analysis for distributed systems
CN111163072A (en) Method and device for determining characteristic value in machine learning model and electronic equipment
CN106331160A (en) Data migration method and system
CN110765082B (en) Hadoop file processing method and device, storage medium and server
CN112102063A (en) Data request method, device, equipment, platform and computer storage medium
CN110347656B (en) Method and device for managing requests in file storage system
CN116150431A (en) Information processing method, apparatus, electronic device, and computer-readable storage medium
CN103366014B (en) System for cloud computing data handling system and method based on cluster
CN112395337A (en) Data export method and device
CN110555158A (en) mutually exclusive data processing method and system, and computer readable storage medium
CN110688223B (en) Data processing method and related product
CN112230936A (en) Container group deployment method and device and server
US20150215264A1 (en) Common message sending method, electronic device, and storage medium
CN112437125A (en) Information concurrent processing method and device, electronic equipment and storage medium
CN112148461A (en) Application scheduling method and device
CN110717826A (en) Asset filtering method and device
CN104092735A (en) Cloud computing data access method and system based on binary tree
CN110738507A (en) exchange code acquisition method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 40086162

Country of ref document: HK