CN114416723B

CN114416723B - Data processing method, device, equipment and storage medium

Info

Publication number: CN114416723B
Application number: CN202111539748.5A
Authority: CN
Inventors: 张爱民; 李艳国
Original assignee: Beijing Dajia Internet Information Technology Co Ltd
Current assignee: Beijing Dajia Internet Information Technology Co Ltd
Priority date: 2021-12-15
Filing date: 2021-12-15
Publication date: 2023-01-20
Anticipated expiration: 2041-12-15
Also published as: CN114416723A

Abstract

The disclosure relates to a data processing method, device, equipment and storage medium, which can reduce occupied processing resources and improve data processing performance. The specific scheme comprises the following steps: and acquiring a target instruction, wherein the target instruction is used for indicating to process the field to be processed, the target instruction comprises a target index, the target index comprises the field to be processed, and the target index corresponds to the target object. Target metadata is obtained from the database according to the target index, the target metadata including first information, the first information including at least one field. And determining that the field to be processed does not exist in at least one field according to the first information and the field to be processed.

Description

Data processing method, device, equipment and storage medium

Technical Field

The present disclosure relates to the field of network technologies, and in particular, to a data processing method, apparatus, device, and storage medium.

Background

With the development of network technology, redis has become a widely used in-memory database. For many business scenarios, the access density is low and the cost of the in-memory database is high due to the large amount of data. There is therefore a need to store data on a lower cost storage medium. For example, data is stored on a Solid State Disk (SSD). At the same time, there is a need to provide the ability to access data using the Redis protocol for service usage.

In the current technical solution, when data is operated, it is necessary to determine whether the data exists and whether the data is expired. Specifically, in the process of operating the data, the corresponding metadata needs to be read through a key to determine whether the data is expired. And, reading the corresponding field value through the key and the field name to judge whether the data exists. However, if a plurality of data needs to be operated, the above process of determining whether the data exists occupies a large amount of processing resources, which affects the performance of processing the data.

Disclosure of Invention

The present disclosure provides a data processing method, apparatus, device and storage medium, which can reduce occupied processing resources and improve data processing performance. The technical scheme of the disclosure is as follows:

according to a first aspect of the present disclosure, there is provided a data processing method applied to a server, where the server is deployed with a database, and data of a target object is stored in the database, where the data of the target object includes: target metadata and at least one field value. The method comprises the following steps:

and acquiring a target instruction, wherein the target instruction is used for indicating to process the field to be processed, the target instruction comprises a target index, the target index comprises the field to be processed, and the target index corresponds to the target object. Target metadata is obtained from the database according to the target index, the target metadata including first information, the first information including at least one field. And determining that the field to be processed does not exist in at least one field according to the first information and the field to be processed.

Optionally, the data format of the at least one field is a preset data format, and the data amount of the at least one field represented according to the preset data format is smaller than a preset data amount threshold.

Optionally, the target metadata further includes second information, where the second information is a first identifier or a second identifier, the first identifier is used to indicate that the first information is valid data, and the second identifier is used to indicate that the first information is invalid data. The data processing method further comprises the following steps: and if the second information is the first identifier, determining that the field to be processed does not exist in at least one field according to the first information and the field to be processed.

Optionally, the target instruction is a rewrite-type instruction or a read-type instruction. The data processing method further comprises the following steps: and if the target instruction is a rewriting instruction, processing the field to be processed according to the target instruction. And if the target instruction is a reading instruction, not executing the target instruction.

Optionally, the data processing method further includes: and acquiring the target number, wherein the target number is the number of at least one field. And if the target number is greater than the preset number threshold, updating the second information into a second identifier.

Optionally, the data processing method further includes: and if the second information is the second identifier, reading the value of the field to be processed. And processing the field to be processed according to the target instruction and the value of the field to be processed.

Optionally, the value of the at least one field is divided into at least one field group, and each field group of the at least one field group includes: values of a portion of the at least one field; the target metadata further includes third information for indicating that the at least one field group is valid data; the target index also includes an identification of a set of fields to be processed, including fields to be processed. The data processing method further comprises the following steps: and if the second information is the second identifier and the third information exists in the target metadata, reading the values of all the fields in the field group to be processed according to the identifier of the field group to be processed. And processing all fields in the field group to be processed according to the target instruction and the values of all fields in the field group to be processed.

Optionally, the data processing method further includes: and storing the target metadata into a cache.

Optionally, the data processing method further includes: and if the time for caching the target metadata is greater than a preset time threshold, updating the target metadata.

According to a second aspect of the present disclosure, there is provided a data processing apparatus, applied to a server, where the server is deployed with a database, and data of a target object is stored in the database, where the data of the target object includes: target metadata and values of at least one field. The data processing device comprises: an acquisition unit and a processing unit.

The acquisition unit is configured to execute an acquisition target instruction, the target instruction is used for indicating to process the field to be processed, the target instruction comprises a target index, the target index comprises the field to be processed, and the target index corresponds to the target object. A processing unit configured to perform obtaining target metadata from the database according to the target index, the target metadata including first information, the first information including at least one field. And the processing unit is further configured to determine that the field to be processed does not exist in the at least one field according to the first information and the field to be processed.

Optionally, the target metadata further includes second information, where the second information is a first identifier or a second identifier, the first identifier is used to indicate that the first information is valid data, and the second identifier is used to indicate that the first information is invalid data. And the processing unit is specifically configured to determine that the field to be processed does not exist in at least one field according to the first information and the field to be processed if the second information is the first identifier.

Optionally, the target instruction is a rewrite-type instruction or a read-type instruction. And the processing unit is also configured to execute processing of the field to be processed according to the target instruction if the target instruction is a rewriting instruction. And the processing unit is also configured to execute the target instruction if the target instruction is the reading instruction.

Optionally, the obtaining unit is further configured to perform obtaining a target number, where the target number is the number of the at least one field. And the processing unit is further configured to update the second information to the second identifier if the target number is greater than a preset number threshold.

Optionally, the processing unit is further configured to read a value of the field to be processed if the second information is the second identifier. And the processing unit is also configured to process the field to be processed according to the target instruction and the value of the field to be processed.

Optionally, the value of the at least one field is divided into at least one field group, and each field group in the at least one field group includes: values of a portion of the at least one field; the target metadata further includes third information for indicating at least one field group as valid data; the target index also includes an identification of a set of fields to be processed, including fields to be processed. And the processing unit is also configured to read the values of all the fields in the field group to be processed according to the identifier of the field group to be processed if the second information is the second identifier and the third information exists in the target metadata. And the processing unit is also configured to execute processing of all fields in the field group to be processed according to the target instruction and the values of all fields in the field group to be processed.

Optionally, the data processing apparatus further includes a storage unit configured to store the target metadata into the cache.

Optionally, the processing unit is further configured to update the target metadata if the time for caching the target metadata is greater than a preset time threshold.

According to a third aspect of the present disclosure, there is provided a server comprising:

a processor. A memory for storing processor-executable instructions. Wherein the processor is configured to execute instructions to implement any of the above-described first aspect optional data processing methods.

According to a fourth aspect of the present disclosure, there is provided a computer readable storage medium having stored thereon instructions, which, when executed by a processor of a server, enable the server to perform any one of the above-mentioned first aspect optional data processing methods.

According to a fifth aspect of the present disclosure there is provided a computer program product containing instructions which, when run on a computer, cause the computer to perform a method of processing data as optionally described in any of the first aspects.

According to a sixth aspect of the present disclosure, there is provided a chip comprising a processor and a communication interface, the communication interface being coupled to the processor, the processor being configured to execute a computer program or instructions to implement a method of processing data as described in the first aspect and any one of the possible implementations of the first aspect.

The technical scheme provided by the disclosure at least brings the following beneficial effects: and acquiring a target instruction, wherein the target instruction is used for indicating to process the field to be processed, the target instruction comprises a target index, the target index comprises the field to be processed, the target index corresponds to a target object, and the data of the target object comprises target metadata and the value of at least one field. And then, acquiring target metadata from the database according to the target index, wherein the target metadata comprises first information. Since the first information includes at least one field of the target object, it may be determined that a field to be processed does not exist in the at least one field according to the first information and the field to be processed. Therefore, compared with the prior art, whether the field to be processed exists in the field of the target object is determined without reading the value of the field, so that the occupation of processing resources can be reduced, and the performance of processing data is improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure and are not to be construed as limiting the disclosure.

FIG. 1 is a schematic diagram illustrating the structure of a server in accordance with an exemplary embodiment;

FIG. 2 is a flow diagram illustrating a method of processing data in accordance with an exemplary embodiment;

FIG. 3 is a flow chart illustrating another method of processing data in accordance with an exemplary embodiment;

FIG. 4 is a block diagram of a data processing apparatus according to an exemplary embodiment;

fig. 5 is a schematic diagram illustrating another data processing apparatus according to an exemplary embodiment.

Detailed Description

In order to make the technical solutions of the present disclosure better understood, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the disclosure described herein are capable of operation in other sequences than those illustrated or described herein. The implementations described in the exemplary embodiments below do not represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

It should be noted that the user information (including but not limited to user device information, user personal information, etc.) referred to in the present disclosure is information authorized by the user or sufficiently authorized by each party.

First, an application scenario of the embodiment of the present disclosure is described.

The data processing method is applied to processing the data in the database. In the related art, when data is operated, whether the data exists or not and whether the data is out of date or not need to be determined. Specifically, in the process of operating data, corresponding metadata needs to be read through a key to judge whether the data is expired. And, reading the corresponding field value through the key and the field name to judge whether the data is stored in the database. If the field exists in the database, the field value can be returned; if the field does not exist in the database, the field value cannot be returned. However, if a plurality of data needs to be operated, the above process of determining whether the data exists occupies a large amount of processing resources, which affects the performance of processing the data.

In order to solve the above problem, an embodiment of the present disclosure provides a data processing method, where metadata includes a field name. Then, when the metadata is read through the index, not only can whether the data is expired be judged, but also the field name can be read, and whether the field to be processed exists can be judged. Therefore, when data is processed, whether the data is out of date or not and whether the data exists or not can be judged only by reading the metadata, field values do not need to be read, occupation of processing resources is reduced, and performance of processing the data can be improved.

Fig. 1 is a schematic structural diagram of a server to which the method provided by the present disclosure is applied according to an embodiment of the present disclosure. The server 10 includes a processor 101 and a memory 102.

The processor 101 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor 101 may include an Application Processor (AP), a modem processor, a Graphics Processor (GPU), an Image Signal Processor (ISP), a controller, a memory, a video codec, a Digital Signal Processor (DSP), a baseband processor, and/or a neural-Network Processing Unit (NPU), among others. Wherein, the different processing units may be independent devices or may be integrated in one or more processors.

Memory 102 may include one or more computer-readable storage media, which may be non-transitory. Memory 102 may also include high speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in the memory 102 is used to store at least one instruction for execution by the processor 101 to implement the group communication method provided by the disclosed method embodiments.

In some embodiments, the server 10 may further include: a peripheral interface 103 and at least one peripheral. The processor 101, memory 102 and peripheral interface 103 may be connected by bus or signal lines. Each peripheral may be connected to peripheral interface 103 via a bus, signal line, or circuit board. Specifically, the peripheral device includes: at least one of radio frequency circuitry 104, display screen 105, camera assembly 106, audio circuitry 107, positioning assembly 108, and power supply 109.

The peripheral interface 103 may be used to connect at least one peripheral related to I/O (Input/Output) to the processor 101 and the memory 102. In some embodiments, processor 101, memory 102, and peripheral interface 103 are integrated on the same chip or circuit board; in some other embodiments, any one or both of the processor 101, the memory 102, and the peripheral interface 103 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

The Radio Frequency circuit 104 is used for receiving and transmitting RF (Radio Frequency) signals, also called electromagnetic signals. The radio frequency circuitry 104 communicates with communication networks and other communication devices via electromagnetic signals. The rf circuit 104 converts an electrical signal into an electromagnetic signal for transmission, or converts a received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 104 comprises: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a subscriber identity module card, and so forth. The radio frequency circuitry 104 may communicate with other servers via at least one wireless communication protocol. The wireless communication protocols include, but are not limited to: metropolitan area networks, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and/or Wi-Fi (Wireless Fidelity) networks. In some embodiments, the rf circuit 104 may further include NFC (Near Field Communication) related circuits, which are not limited by this disclosure.

The display screen 105 is used to display a UI (User Interface). The UI may include graphics, text, icons, video, and any combination thereof. When the display screen 105 is a touch display screen, the display screen 105 also has the ability to capture touch signals on or over the surface of the display screen 105. The touch signal may be input to the processor 101 as a control signal for processing. At this point, the display screen 105 may also be used to provide virtual buttons and/or a virtual keyboard, also referred to as soft buttons and/or a soft keyboard. In some embodiments, the display screen 105 may be one, providing the front panel of the server 10; the Display 105 may be made of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), and other materials.

The camera assembly 106 is used to capture images or video. Optionally, camera assembly 106 includes a front camera and a rear camera. Generally, the front camera is disposed on the front panel of the server, and the rear camera is disposed on the back of the server. Audio circuitry 107 may include a microphone and a speaker. The microphone is used for collecting sound waves of a user and the environment, converting the sound waves into electric signals, and inputting the electric signals into the processor 101 for processing or inputting the electric signals into the radio frequency circuit 104 to realize voice communication. For stereo capture or noise reduction purposes, the microphones may be multiple and located at different locations on the server 10. The microphone may also be an array microphone or an omni-directional pick-up microphone. The speaker is used to convert electrical signals from the processor 101 or the radio frequency circuit 104 into sound waves. The loudspeaker can be a traditional film loudspeaker and can also be a piezoelectric ceramic loudspeaker. When the speaker is a piezoelectric ceramic speaker, the speaker can be used for purposes such as converting an electric signal into a sound wave audible to a human being, or converting an electric signal into a sound wave inaudible to a human being to measure a distance. In some embodiments, audio circuitry 107 may also include a headphone jack.

The positioning component 108 is used to locate the current geographic Location of the server 10 to implement navigation or LBS (Location Based Service). The Positioning component 108 can be a Positioning component based on the Global Positioning System (GPS) in the united states, the beidou System in china, the grignard System in russia, or the galileo System in the european union.

The power supply 109 is used to supply power to the various components in the server 10. The power source 109 may be alternating current, direct current, disposable or rechargeable. When power source 109 comprises a rechargeable battery, the rechargeable battery may support wired or wireless charging. The rechargeable battery may also be used to support fast charge technology.

In some embodiments, the server 10 also includes one or more sensors 1010. The one or more sensors 1010 include, but are not limited to: acceleration sensors, gyroscope sensors, pressure sensors, fingerprint sensors, optical sensors, and proximity sensors.

The acceleration sensor may detect the magnitude of acceleration on three coordinate axes of the coordinate system established with the server 10. The gyroscope sensor can detect the body direction and the rotation angle of the server 10, and the gyroscope sensor can cooperate with the acceleration sensor to acquire the 3D action of the user on the server 10. The pressure sensors may be located on the side frame of the server 10 and/or underneath the display screen 105. When the pressure sensor is provided on the side frame of the server 10, a holding signal of the server 10 by the user can be detected. The fingerprint sensor is used for collecting fingerprints of users. The optical sensor is used for collecting the intensity of ambient light. Proximity sensors, also known as distance sensors, are typically provided on the front panel of the server 10. The proximity sensor is used to collect the distance between the user and the front of the server 10.

An executing body of the data processing method provided by the present disclosure may be a data processing device, and the executing device may be a server shown in fig. 1. Meanwhile, the execution device may also be a Central Processing Unit (CPU) of the server, or a control module for Processing data in the server. In the embodiment of the present application, a method for processing data performed by a server is taken as an example, and a method for processing data provided in the embodiment of the present application is described.

In one implementable manner, the server is used to provide voice and/or data connectivity services to the user. Servers may be referred to by different names, such as UE side, terminal unit, terminal station, mobile station, remote terminal, mobile device, wireless communication device, vehicular user equipment, terminal agent, or terminal device, etc.

Optionally, the server may be various handheld devices, vehicle-mounted devices, wearable devices, and computers with communication functions, which is not limited in this respect in this disclosure. For example, the handheld device may be a smartphone. The in-vehicle device may be an in-vehicle navigation system. The wearable device may be a smart band. The computer may be a Personal Digital Assistant (PDA) computer, a tablet computer, and a laptop computer.

In the embodiment of the present disclosure, a database is deployed in a server, data of a target object is stored in the database, and the data of the target object includes: target metadata and at least one field value. Wherein the target metadata is used for information reflecting data of the target object.

Illustratively, the data of the target object includes: name-Zhang three, age-25, sex-male, etc. Wherein "name", "sex", "age" is a field (name), "Zhang III", "25" and "male" are values of the field. The target metadata may reflect whether the data is expired.

Optionally, the target metadata is encoded by a Protocol Buffers method.

Alternatively, in the case where the target metadata does not exist (or not), the target metadata may be created. For example, where the target instruction is a set instruction, target metadata may be created. Alternatively, the target metadata (version number) may be updated after execution of the set instruction.

After the application scenario and the server of the embodiment of the present disclosure are introduced, the server is taken as an example to describe in detail the data processing method provided by the embodiment of the present disclosure.

FIG. 2 is a flow chart illustrating a method of processing data according to an exemplary embodiment. As shown in fig. 2, the method may include steps 201-203.

201. And acquiring a target instruction.

The target instruction is used for indicating the processing of the field to be processed.

It should be noted that, in the embodiment of the present disclosure, the target instruction is not limited. For example, the target instruction may be a set instruction for adding or updating a pending field. As another example, the target instruction may be a del instruction for deleting pending fields. As another example, the target instruction may be a get instruction for reading the pending field. As another example, the target instruction may be a Redis instruction (e.g., a HMSET instruction, a HSET instruction, a HDEL instruction, etc.).

In an embodiment of the present disclosure, the target instruction includes a target index, the target index includes a field to be processed, and the target index corresponds to the target object.

It should be noted that the field to be processed may include one or more fields, which is not limited in this disclosure.

Optionally, the target index further includes a preset index, and the preset index corresponds to the target object. For example, the target index may be "format _ version" H "user _ key _ size" _ user _ key "_ version" _ F "user _ field". The format _ version "H" user _ key _ size "_ user _ key" _ version "_ F" is a preset index corresponding to a target object (e.g., a first object). The field to be processed is a user _ field, which for example comprises a field a and a field b.

It should be noted that the format _ version may be 1 byte in length, and is used to indicate the storage format version number. The length of the user _ key _ size is not limited, and may be represented by an ASCII character string of decimal numbers. The user _ Key is a Redis Key, and the length of the user _ Key is not limited.

Optionally, when the set instruction is executed, if the preset index is invalid data (the preset index is deleted or expired), the target metadata is created again. And the target metadata of different versions corresponding to the same preset index is different.

202. And acquiring target metadata from the database according to the target index.

Wherein the target metadata includes first information, the first information including at least one field.

It should be noted that, in this embodiment of the present disclosure, that the first information includes at least one field means that the first information includes names of all fields of the target object. The first information may include names of one or more fields.

Optionally, the target metadata further includes a version number, an expiration time, and the like.

Illustratively, the target metadata of the first object includes: version-0001, version _ _at-0, user _fields- "field a, field b, field c, field d" (i.e., first information). That is, the target metadata of the first object has a version number of 0001 and an expiration time of 0 (i.e., unexpired), and all fields in the first object include: field a, field b, field c, field d.

The length of version may be 8 bytes, which is the configuration version number of the data of the target object. The length of expire _ at may be 8 bytes, with millisecond precision. When expire _ at is 0, it indicates that the expiration time of the data of the target object is not set, that is, the data of the target object is not expired.

It should be noted that, when the number of all (i.e. at least one) fields in the target object is too large (for example, the number of fields is greater than 1000), reading the first information also occupies more resources, which affects the performance of processing data.

It should be noted that, the preset data amount threshold is not limited in the embodiment of the present disclosure. For example, the preset data amount threshold may be 10 Kilobytes (KB). For another example, the preset data amount threshold may be 11KB. For another example, the preset data amount threshold may be 15KB. And under the condition that the data format of at least one field is a preset data format, storing the corresponding relation between the at least one field and the at least one field expressed according to the preset data format in the server.

Illustratively, the predetermined data format may be an integer type data format or a bitmap data format. Taking the shaped data format as an example, if at least one field includes: name, age, sex, then at least one field expressed according to the shaping data format includes: 1. 2 and 3. Wherein 1 is used to represent name,2 is used to represent age, and 3 is used to represent sex.

It is understood that the data format of the at least one field is a preset data format, and the data amount of the at least one field expressed according to the preset data format is smaller than the preset data amount threshold. Therefore, the data processing amount can be reduced, the processing resources occupied when the first information is acquired can be further reduced, and the data processing performance can be improved.

Optionally, after the target metadata is obtained, it is determined whether the field to be processed is expired according to a version number and/or an expiration time in the target metadata. If the pending field has not expired, go to step 203. If the pending field has expired, the version number is updated and step 203 is executed. Or, if the field to be processed has expired, the target instruction is not executed.

203. And determining that the field to be processed does not exist in at least one field according to the first information and the field to be processed.

Illustratively, the first information includes: field a, field b, field c, field d. The fields to be processed include: a field a and a field b. It is determined that there is a field to be processed in the at least one field. For another example, the first information includes: field a, field b, field c, field d. The fields to be processed include: and e and d, determining that the field to be processed does not exist in at least one field.

In one possible implementation, if there is no pending field in at least one field, the value of the pending field is not read.

It is understood that, in step 203, it may be determined whether there is a field to be processed in the at least one field through the first information. Compared with the prior art, whether the field to be processed exists in the at least one field is determined without reading the value of the field to be processed. Therefore, the occupation of processing resources can be reduced, and the performance of processing data is improved.

Illustratively, suppose the number of fields to be processed is 100. With conventional solutions (i.e. determining whether a field is present by reading the value of the field) 100 data (or 101 data) have to be read. By adopting the scheme of the embodiment of the disclosure, target metadata data only needs to be read once.

The technical scheme provided by the embodiment at least has the following beneficial effects: and acquiring a target instruction, wherein the target instruction is used for indicating to process the field to be processed, the target instruction comprises a target index, the target index comprises the field to be processed, the target index corresponds to a target object, and the data of the target object comprises target metadata and the value of at least one field. And then, acquiring target metadata from the database according to the target index, wherein the target metadata comprises first information. Since the first information includes at least one field of the target object, it may be determined that a field to be processed does not exist in the at least one field according to the first information and the field to be processed. Therefore, compared with the prior art, whether the field to be processed exists in the field of the target object is determined without reading the value of the field, so that the occupation of processing resources can be reduced, and the performance of processing data is improved.

In one implementable approach, the target instruction is a rewrite-type instruction or a read-type instruction. Wherein the rewrite-like instruction is to rewrite the field. The read class instruction is used to read a field. For example, rewrite-like instructions may include set instructions and del instructions. The fetch class instruction includes a get instruction.

After determining that the field to be processed does not exist in at least one field according to the first information and the field to be processed (step 203), determining whether the target instruction is a rewrite-type instruction. And if the target instruction is a rewriting instruction, processing the field to be processed according to the target instruction. And if the target instruction is a read instruction, not executing the target instruction.

Illustratively, if the target instruction is a set instruction, the set instruction is used to indicate that "name: li four'. After that, the "name: lee four "was added to the database.

It will be appreciated that in the conventional art, in the case of a set instruction, the metadata and the value of the name field need to be read to determine if the name field is present. Thereafter, in a case where it is determined that the name field does not exist, add "name: li four'. In the embodiment of the present disclosure, when the metadata is read to determine that the name field does not exist in at least one field, the "name: li four'.

Illustratively, if the target instruction is a get instruction (e.g., for reading the value of the name field), it is determined that the name field is not present because of the read metadata. Therefore, the value of the name field need not be read, i.e., the get instruction is not executed.

The technical scheme provided by the embodiment at least has the following beneficial effects: the target instruction is a rewrite-type instruction or a read-type instruction. And if the target instruction is a rewriting instruction, processing the field to be processed according to the target instruction. And if the target instruction is a read instruction, the target instruction is not executed. In this way, the type of instruction may be combined to determine whether to execute the target instruction. Therefore, the occupied processing resources can be reasonably reduced, and the data processing capacity is further improved.

In a practical manner, as shown in fig. 3, before step 203, the data processing method may further include step 301.

301. And determining whether the first information is valid data according to the second information.

In an embodiment of the present disclosure, the target metadata further includes second information, and the second information is the first identifier or the second identifier. The first identifier is used for indicating that the first information is valid data, and the second identifier is used for indicating that the first information is invalid data.

It should be noted that the first identifier and the second identifier are not limited in the embodiments of the present disclosure. For example, the first identifier and the second identifier may be integer data (i.e., numerical values). As another example, the first and second identifiers may be letters (e.g., "a", "b"). The following describes embodiments of the present disclosure by taking the first identifier and the second identifier as numerical values.

Illustratively, the first flag is 0, which may indicate that the first information is valid data. The second flag is 1, which may indicate that the first information is invalid data.

Optionally, the second identifier may be used to indicate the number of fields in the first information.

Illustratively, suppose the first information includes: field a, field b, field c, and field d. The second identification is 4.

As a possible implementation manner, if the second information is the first identifier, it is determined that the first information is valid data. And if the second information is the second identifier, determining that the first information is invalid data.

In one possible implementation, if the first information is valid data, step 203 is executed. That is, if the second information is the first identifier, it is determined that the field to be processed does not exist in at least one field according to the first information and the field to be processed.

It is understood that the target metadata further includes second information, the second information is a first identifier or a second identifier, the first identifier is used for indicating that the first information is valid data, and the second identifier is used for indicating that the first information is invalid data. And if the second information is the first identifier, indicating that the first information is invalid data. In this way, it may be determined that there is no pending field in the at least one field that can be determined by the first information.

In one possible implementation, a target number is obtained, where the target number is the number of at least one field. And if the target number is greater than the preset number threshold, updating the second information into a second identifier.

Illustratively, if the target number is 2000 and the preset number threshold is 1000, the second information is updated to the second identifier.

Optionally, in a case that the second information is the first identifier, the target number is obtained, where the target number is the number of the at least one field. And if the target number is greater than the preset number threshold, updating the second information into a second identifier.

Optionally, when the second information is the second identifier, the operations of acquiring the target number and updating the second information are not performed.

The technical scheme provided by the embodiment at least has the following beneficial effects: and acquiring the target number, wherein the target number is the number of at least one field. And if the target number is greater than the preset number threshold, updating the second information into a second identifier. That is, in the case where the number of at least one field is large, the first information may be updated to invalid data. Therefore, different schemes can be flexibly adopted to execute the target instruction, and the performance of processing data can be improved.

In one possible implementation manner, if the second information is valid data, step 203 is executed. If the second message is invalid data, go to step 302.

As shown in fig. 3, in case the first information is invalid data, after step 301, the method further comprises steps 302-303.

302. And reading the value of the field to be processed.

As a possible implementation manner, if the second information is the second identifier, the value of the field to be processed is read according to the field to be processed.

Illustratively, suppose the fields to be processed include: name, age and sex, at least one field comprises: and if the name is Zhang III and the age is 13, reading the values of the name, the age and the sex respectively, and obtaining that the name is Zhang III and the age is 13 and the value of the sex cannot be read.

303. And processing the field to be processed according to the target instruction and the value of the field to be processed.

As a possible implementation, whether the field to be processed exists is determined according to the value of the field to be processed. And then processing the field to be processed according to the target instruction.

It should be noted that, for the introduction of processing the field to be processed according to the target instruction and the value of the field to be processed, reference may be made to a method for processing the field according to the key and the field name in the conventional technology, which is not described herein again.

It should be noted that, in the case that the second information is the second identifier (e.g., the number of at least one field), the number of fields may be too large. Thus, the scheme of steps 201-203 may increase the occupied processing resources. Therefore, under the condition that the second information is the second identifier, the occupied processing resource can be reduced through the conventional technical scheme.

The technical scheme provided by the embodiment at least has the following beneficial effects: if the second information is the second identifier, the first information is invalid data, and whether the field to be processed exists in the at least one field cannot be determined through the first information. Then, the value of the field to be processed can be read, and the field to be processed is processed according to the target instruction and the value of the field to be processed. Therefore, the fields to be processed can be processed by adopting different schemes according to the second information, and the flexibility of the schemes is improved.

It should be noted that, if the number of the fields to be processed is large, the execution of the target instruction may occupy more processing resources, which affects the performance of processing data. For example, if the target instruction is a set instruction, the set instruction is used to write 100 fields. The fields need to be written to the database through 100 preset indexes and fields.

In one possible implementation, the value of the at least one field is divided into at least one field group, each of the at least one field group comprising: values of a portion of the at least one field.

Illustratively, it is assumed that the value of at least one field includes: a first field-a, a second field-b, a third field-c, a fourth field-d, a fifth field-e, a sixth field-f. At least one field group includes: a first field group, a second field group, and a third field group. Wherein the first field group includes: a first field-a, a second field-b, a second field group comprising: a third field-c, a fourth field-d, a fifth field-e, and a sixth field-f.

In an embodiment of the present disclosure, the target metadata further includes third information indicating that the at least one field group is valid data.

The third information is expressed in a form that is not limited in the embodiment of the present disclosure. For example, the third information may be integer data (i.e., a numerical value). For another example, the third information may be letters (e.g., "a", "b"). The third information is taken as an example to describe the embodiment of the disclosure.

Optionally, the third information is further used to indicate the number of field groups.

Illustratively, if the third information is any one of values 1-255, it indicates that the field group is valid data. Optionally, if the third information is 0, the field group is represented as invalid data.

It should be noted that the third information may be represented by slot _ num. The slot _ num may have a length of 1 byte, and the third information corresponding to different preset indexes may be different.

In the embodiment of the present disclosure, the target index further includes an identifier of a field group to be processed, and the field group to be processed includes a field to be processed.

Illustratively, the pending field includes: the field group A to be processed comprises: a first field, a second field, and a third field. The identification of the pending field group a is "slot1". That is, slot1 corresponds to pending field set A (i.e., the first field, the second field, and the third field).

As a possible implementation manner, if the second information is the second identifier and the third information exists in the target metadata, values of all fields in the field group to be processed are read. And processing all fields in the field group to be processed according to the target instruction and the values of all fields in the field group to be processed.

Illustratively, suppose that at least one field set of the target object includes: a first field set (identified as slot _ 1), a second field set (identified as slot _ 2), and a third field set (identified as slot _ 3). If the first field set includes: a first field-a, a second field-b, and a third field-c. The identification of the field group to be processed is slo t _1. Thereafter, the values of all fields in the first field group (i.e., the first field-a, the second field-b, and the third field-c) may be read according to the preset index and slo t _1. Thereafter, if the target instruction is a set instruction (update the first field-a to the first field-d, the second field-b to the second field-e, and the third field-c to the first field-f), the fields in the first field set are updated.

Optionally, when at least one field group is valid data, the second information is a second identifier, that is, the first information is invalid data.

The technical scheme provided by the embodiment at least has the following beneficial effects: the value of the at least one field is divided into at least one field group, each of the at least one field group comprising: values of a portion of the at least one field. The target metadata further includes third information for indicating that at least one field group is valid data. The target index also includes an identification of a pending field group, which includes pending fields. And under the condition that the second information is the second identifier and the third information exists in the target metadata, reading the values of all the fields in the field group to be processed according to the identifier of the field group to be processed. That is to say, the values of all the fields can be read at a time according to the identifier of the field group to be processed, thereby reducing the occupied processing resources. And finally, processing all fields in the field group to be processed according to the target instruction and the values of all fields in the field group to be processed. In summary, by using the field group and the field group identifier, the processing resources occupied by reading a plurality of fields can be reduced, and the performance of processing data can be improved.

It should be noted that, when the target instruction is executed, the compact thread may be called to read the target metadata to determine whether the data is expired. And reading the value of the field and processing the field. However, if the target instruction is executed multiple times in a short time, the target metadata needs to be called multiple times. Thus, the occupied processing resources may be large, and the processing performance may be affected.

In one implementation, the target metadata is cached after being retrieved from the database according to the target index.

For example, if the target instruction is a get instruction, then the target metadata of the target object may be obtained according to the index and stored in the cache.

It should be noted that the preset index (e.g., key, format _ version "H" user _ key _ size "_" user _ key) of the same object is the same. This makes the reading of the metadata very local. The cache hit rate depends on the number of fields, the higher the number, the higher the hit rate. For example, one field hit rate is 0, two field hit rates are 1/2, and three field hit rates are 2/3.

The technical scheme provided by the embodiment at least has the following beneficial effects: and storing the target metadata into a cache. When the target instruction is executed again, the target metadata does not need to be acquired from the database, and can be directly acquired from the cache. Therefore, the occupied processing resources can be reduced, and the processing performance is improved.

In one implementation, the target metadata is updated if the time for caching the target metadata is greater than a predetermined time threshold.

Illustratively, suppose that the Time To Live (TTL) of the target metadata a proposed in the cache is 1 second. The target metadata a may be newly acquired from the database and the newly acquired target metadata a may be stored in the cache when the target metadata a is stored in the cache for more than 1 second.

The technical scheme provided by the embodiment at least has the following beneficial effects: and if the caching time of the target metadata is greater than a preset time threshold, updating the target metadata. Therefore, the target metadata in the cache can be guaranteed to be unexpired data (namely effective data), and the accuracy of processing the data can be further improved.

It will be appreciated that the above method may be implemented by a processing device of data. The data processing device includes hardware structures and/or software modules for performing the respective functions in order to realize the functions. Those of skill in the art will readily appreciate that the various illustrative modules and algorithm steps described in connection with the embodiments disclosed herein may be implemented as hardware or combinations of hardware and computer software. Whether a function is performed as hardware or computer software drives hardware depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the disclosed embodiments.

The embodiment of the present disclosure may perform division of functional modules on the processing apparatus of the data according to the method example, for example, each functional module may be divided corresponding to each function, or two or more functions may be integrated into one processing module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. It should be noted that, the division of the modules in the embodiments of the present disclosure is illustrative, and is only one division of logic functions, and there may be another division in actual implementation.

Fig. 4 is a block diagram illustrating a data processing apparatus according to an exemplary embodiment. Referring to fig. 4, the data processing apparatus 40 includes an acquisition unit 41, a processing unit 42, and a storage unit 43.

And the obtaining unit 41 is configured to execute a obtain target instruction, where the target instruction is used to instruct to process the to-be-processed field, and includes a target index, where the target index includes the to-be-processed field, and the target index corresponds to the target object. A processing unit 42 configured to perform obtaining target metadata from the database according to the target index, the target metadata comprising first information, the first information comprising at least one field. The processing unit 42 is further configured to determine that there is no field to be processed in the at least one field according to the first information and the field to be processed.

Optionally, the target metadata further includes second information, where the second information is a first identifier or a second identifier, the first identifier is used to indicate that the first information is valid data, and the second identifier is used to indicate that the first information is invalid data. The processing unit 42 is specifically configured to determine that no field to be processed exists in at least one field according to the first information and the field to be processed if the second information is the first identifier.

Optionally, the target instruction is a rewrite-type instruction or a read-type instruction. The processing unit 42 is further configured to execute, if the target instruction is a rewrite instruction, processing the field to be processed according to the target instruction. The processing unit 42 is further configured to execute the target instruction if the target instruction is a read-type instruction.

Optionally, the obtaining unit 41 is further configured to perform obtaining a target number, where the target number is the number of at least one field. The processing unit 42 is further configured to update the second information to the second identifier if the target number is greater than the preset number threshold.

Optionally, the processing unit 42 is further configured to perform, if the second information is the second identifier, reading a value of the field to be processed. The processing unit 42 is further configured to execute processing of the field to be processed according to the target instruction and the value of the field to be processed.

Optionally, the value of the at least one field is divided into at least one field group, and each field group in the at least one field group includes: values of some of the at least one field; the target metadata further includes third information for indicating that the at least one field group is valid data; the target index also includes an identification of a pending field group, which includes pending fields. The processing unit 42 is further configured to read values of all fields in the field group to be processed according to the identifier of the field group to be processed if the second information is the second identifier and the third information exists in the target metadata. The processing unit 42 is further configured to execute processing all fields in the field group to be processed according to the target instruction and the values of all fields in the field group to be processed.

Optionally, the data processing apparatus further includes a storage unit 43, and the storage unit 43 is configured to perform storing the target metadata into the cache.

Optionally, the processing unit 42 is further configured to update the target metadata if the time for caching the target metadata is greater than a preset time threshold.

With regard to the data processing apparatus in the above embodiments, the specific manner in which each module performs the operation has been described in detail in the embodiments related to the method, and will not be described in detail here.

Fig. 5 is a schematic structural diagram of a data processing apparatus 50 provided in the present disclosure. As shown in fig. 5, the data processing apparatus 50 may include at least one processor 501 and a memory 503 for storing instructions executable by the processor 501. Wherein the processor 501 is configured to execute instructions in the memory 503 to implement the processing method of data in the above-described embodiments.

In addition, the processing device 50 for data may also include a communication bus 502 and at least one communication interface 504.

The processor 501 may be a GPU, a micro-processing unit, an ASIC, or one or more integrated circuits for controlling the execution of programs in accordance with the disclosed aspects.

The communication bus 502 may include a path that conveys information between the aforementioned components.

The communication interface 504 may be any device, such as a transceiver, for communicating with other devices or communication networks, such as an ethernet, a Radio Access Network (RAN), a Wireless Local Area Network (WLAN), etc.

The memory 503 may be, but is not limited to, a read-only memory (ROM) or other type of static storage device that may store static information and instructions, a Random Access Memory (RAM) or other type of dynamic storage device that may store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disk read-only memory (CD-ROM) or other optical disk storage, optical disk storage (including compact disk, laser disk, optical disk, digital versatile disk, blu-ray disk, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. The memory may be self-contained and connected to the processing unit by a bus. The memory may also be integrated with the processing unit as a volatile storage medium in the GPU.

The memory 503 is used for storing instructions for executing the disclosed solution, and is controlled by the processor 501. The processor 501 is configured to execute instructions stored in the memory 503 to implement the functions of the disclosed method.

In one implementation, processor 501 may include one or more GPUs, such as GPU0 and GPU1 in fig. 5, as an example.

In a specific implementation, the data processing apparatus 50 may include a plurality of processors, such as the processor 501 and the processor 507 in fig. 5, as an example. Each of these processors may be a single-Core (CPU) processor or a multi-core (multi-GPU) processor. A processor herein may refer to one or more devices, circuits, and/or processing cores that process data (e.g., computer program instructions).

In a specific implementation, the data processing apparatus 50 may further include an output device 505 and an input device 506, as an embodiment. An output device 505, which is in communication with the processor 501, may display information in a variety of ways. For example, the output device 505 may be a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display device, a Cathode Ray Tube (CRT) display device, a projector (projector), or the like. The input device 506 is in communication with the processor 501 and can accept user input in a variety of ways. For example, the input device 506 may be a mouse, keyboard, touch screen device, or sensing device, among others.

Those skilled in the art will appreciate that the configuration shown in fig. 5 does not constitute a limitation of the processing means 50 for data, and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components may be employed.

The present disclosure also provides a computer-readable storage medium having instructions stored thereon, which, when executed by a processor of a server, enable the server to perform the group communication method provided by the embodiments of the present disclosure.

The embodiment of the present disclosure further provides a computer program product containing instructions, which when run on a server, causes the server to execute the method for processing data provided by the embodiment of the present disclosure.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice in the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. The data processing method is applied to a server, wherein the server is deployed with a database, and data of a target object is stored in the database, and the data of the target object includes: a value of the target metadata and the at least one field; the method comprises the following steps:

acquiring a target instruction, wherein the target instruction is used for indicating to process a field to be processed, the target instruction comprises a target index, the target index comprises the field to be processed, and the target index corresponds to the target object;

according to the target index, obtaining the target metadata from the database, wherein the target metadata comprises: the device comprises first information and second information, wherein the first information comprises at least one field, the second information is a first identifier or a second identifier, the first identifier is used for indicating that the first information is valid data, and the second identifier is used for indicating that the first information is invalid data;

if the second information is the first identifier, determining that the field to be processed does not exist in the at least one field according to the first information and the field to be processed.

2. The method according to claim 1, wherein the data format of the at least one field is a preset data format, and the data amount of the at least one field represented according to the preset data format is smaller than a preset data amount threshold.

3. The method of claim 1 or 2, wherein the target instruction is a rewrite-type instruction or a read-type instruction; after the determining, according to the first information and the field to be processed, that the field to be processed does not exist in the at least one field, the method further includes:

if the target instruction is the rewriting instruction, processing the field to be processed according to the target instruction;

and if the target instruction is the reading instruction, not executing the target instruction.

4. The method of claim 3, further comprising:

acquiring a target number, wherein the target number is the number of the at least one field;

and if the target number is greater than a preset number threshold, updating the second information into the second identifier.

5. The method according to claim 1 or 2, characterized in that the method further comprises:

if the second information is the second identifier, reading the value of the field to be processed;

and processing the field to be processed according to the target instruction and the value of the field to be processed.

6. The method of claim 1 or 2, wherein the value of the at least one field is divided into at least one field group, and wherein each field group of the at least one field group comprises: values of a portion of the at least one field; the target metadata further includes third information indicating that the at least one field group is valid data; the target index also comprises an identifier of a field group to be processed, and the field group to be processed comprises the field to be processed;

the method further comprises the following steps:

if the second information is the second identifier and the third information exists in the target metadata, reading values of all fields in the field group to be processed according to the identifier of the field group to be processed;

and processing all fields in the field group to be processed according to the target instruction and the values of all fields in the field group to be processed.

7. The method of claim 1, wherein after the obtaining the target metadata from the database according to the target index, the method further comprises:

and storing the target metadata into a cache.

8. The method of claim 7, further comprising:

and if the time for caching the target metadata is greater than a preset time threshold, updating the target metadata.

9. The data processing device is applied to a server, and the server is deployed with a database, data of a target object is stored in the database, and the data of the target object includes: target metadata and a value of at least one field; the method comprises the following steps:

an obtaining unit configured to execute a obtain target instruction, where the target instruction is used to instruct to process a field to be processed, and the target instruction includes a target index, where the target index includes the field to be processed, and the target index corresponds to the target object;

a processing unit configured to execute obtaining the target metadata from the database according to the target index, where the target metadata includes first information and second information, the first information includes the at least one field, the second information is a first identifier or a second identifier, the first identifier is used to indicate that the first information is valid data, and the second identifier is used to indicate that the first information is invalid data;

the processing unit is further configured to determine that the field to be processed does not exist in the at least one field according to the first information and the field to be processed if the second information is the first identifier.

10. The apparatus for processing data according to claim 9, wherein the data format of the at least one field is a preset data format, and the data amount of the at least one field represented according to the preset data format is smaller than a preset data amount threshold.

11. The apparatus according to claim 9 or 10, wherein the target instruction is a rewrite-type instruction or a read-type instruction;

the processing unit is further configured to execute, if the target instruction is the rewrite instruction, processing the field to be processed according to the target instruction;

the processing unit is further configured to execute the target instruction if the target instruction is the read-type instruction.

12. The apparatus for processing data according to claim 11,

the obtaining unit is further configured to perform obtaining a target number, where the target number is the number of the at least one field;

the processing unit is further configured to update the second information to the second identifier if the target number is greater than a preset number threshold.

13. The apparatus for processing data according to claim 9 or 10,

the processing unit is further configured to execute reading the value of the field to be processed if the second information is the second identifier;

the processing unit is further configured to execute processing of the field to be processed according to the target instruction and the value of the field to be processed.

14. The apparatus for processing data according to claim 9 or 10, wherein the value of the at least one field is divided into at least one field group, each field group of the at least one field group comprising: values of a portion of the at least one field; the target metadata further comprises third information for indicating that the at least one field group is valid data; the target index also comprises an identifier of a field group to be processed, and the field group to be processed comprises the field to be processed;

the processing unit is further configured to perform, if the second information is the second identifier and the third information exists in the target metadata, reading values of all fields in the field group to be processed according to the identifier of the field group to be processed;

the processing unit is further configured to execute processing of all fields in the field group to be processed according to the target instruction and values of all fields in the field group to be processed.

15. The apparatus for processing data according to claim 9, wherein said apparatus further comprises:

a storage unit configured to perform storing the target metadata into a cache.

16. The apparatus for processing data according to claim 15,

the processing unit is further configured to update the target metadata if the time for caching the target metadata is greater than a preset time threshold.

17. A server, characterized in that the server comprises:

a processor;

a memory for storing the processor-executable instructions;

wherein the processor is configured to execute the instructions to implement a method of processing data as claimed in any one of claims 1-8.

18. A computer-readable storage medium having instructions stored thereon, wherein the instructions in the computer-readable storage medium, when executed by a processor of a server, enable the server to perform a method of processing data according to any one of claims 1-8.