CN117251584A

CN117251584A - Data processing method, device, product, equipment and medium of block chain network

Info

Publication number: CN117251584A
Application number: CN202311194731.XA
Authority: CN
Inventors: 卢光宏
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2023-09-14
Filing date: 2023-09-14
Publication date: 2023-12-19

Abstract

The application discloses a data processing method, device, product, equipment and medium of a blockchain network, wherein the method comprises the following steps: executing the uplink processing aiming at the first multimedia data, wherein the uplink processing comprises the consensus processing corresponding to the first multimedia data; in the uplink processing, if the consensus is successful, acquiring index information of the first multimedia data and the first multimedia embedding feature based on the consensus data; storing the first multimedia data in association with the multimedia database based on the index information of the first multimedia data, and storing the first multimedia embedded feature in association with the feature database based on the index information of the first multimedia data; the result of the corresponding consensus process in the blockchain network is multimedia data with successful consensus, supporting the associated query based on the multimedia database and the feature database. By adopting the method and the device, the inquiry mode of the multimedia data (such as the first multimedia data) which are subjected to relevant uplink processing in the block chain network can be enriched.

Description

Data processing method, device, product, equipment and medium of block chain network

Technical Field

The present disclosure relates to the field of blockchain technologies, and in particular, to a data processing method, apparatus, product, device, and medium for a blockchain network.

Background

Blockchains are novel application modes of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanisms, encryption algorithms, and the like. The Blockchain (Blockchain), which is essentially a decentralised database, is a string of data blocks that are generated by cryptographic means in association, each data block containing a batch of information of network transactions for verifying the validity of the information (anti-counterfeiting) and generating the next block.

In many situations, the query processing of the multimedia data related to the uplink processing in the blockchain network is involved, in the existing application, the query processing of the multimedia data is realized by querying the uplink transaction where the multimedia data is located, and the query mode of the multimedia data is very single, so how to perform the corresponding query processing on the multimedia data related to the uplink processing in the blockchain network becomes a hotspot problem.

Disclosure of Invention

The application provides a data processing method, a device, a product, equipment and a medium of a blockchain network, which can enrich the query mode of multimedia data (such as first multimedia data) for relevant uplink processing in the blockchain network.

In one aspect, the present application provides a method for processing data in a blockchain network, where the method includes:

performing a uplink process for the first multimedia data in the blockchain network, the uplink process including a consensus process corresponding to the first multimedia data;

in the process of executing the uplink processing aiming at the first multimedia data, if the consensus result of the consensus processing corresponding to the first multimedia data is that the consensus is successful, acquiring index information of the first multimedia data and first multimedia embedding characteristics of the first multimedia data based on the consensus data;

storing the first multimedia data in association with the multimedia database based on the index information of the first multimedia data, and storing the first multimedia embedded feature in association with the feature database based on the index information of the first multimedia data;

the corresponding consensus result of the consensus processing in the blockchain network is multimedia data with successful consensus, and associated inquiry based on a multimedia database and a characteristic database is supported.

In one aspect, the present application provides a data processing apparatus for a blockchain network, the apparatus comprising:

the system comprises a block chain module, a first multimedia data processing module and a second multimedia data processing module, wherein the block chain module is used for executing the uplink processing aiming at the first multimedia data in a block chain network, and the uplink processing comprises the consensus processing corresponding to the first multimedia data;

The acquisition module is used for acquiring index information of the first multimedia data and first multimedia embedding characteristics of the first multimedia data based on the consensus data if the consensus result of the consensus process corresponding to the first multimedia data is successful in the process of executing the uplink process for the first multimedia data;

the storage module is used for storing the first multimedia data in association with the multimedia database based on the index information of the first multimedia data and storing the first multimedia embedded feature in association with the feature database based on the index information of the first multimedia data;

Optionally, the manner in which the uplink module performs the uplink processing for the first multimedia data in the blockchain network includes:

acquiring a data uplink transaction; the data-uplink transaction comprises first multimedia data, the data-uplink transaction being a transaction for performing a uplink process for the first multimedia data;

data uplink transactions are processed in a blockchain network.

Optionally, the data-uplink transaction has a transaction signature; the method for the uplink module to perform uplink processing on the data uplink transaction in the blockchain network comprises the following steps:

adopting the transaction signature to verify the data uplink transaction, and if the data uplink transaction is successfully verified, calling the virtual machine to execute the data uplink transaction;

if the data uplink transaction is successfully executed, index information of first multimedia data in the data uplink transaction is generated;

performing replacement processing on the first multimedia data in the data uplink transaction by adopting the generated index information, and packaging the data uplink transaction after the replacement processing into a target block;

performing consensus processing on the target block in the block chain network, and if the consensus result of the target block is successful, uploading the target block to the block chain network;

the consensus processing corresponding to the first multimedia data refers to the consensus processing of the target block, and the consensus data refers to the target block.

Optionally, before the replacing processing is performed on the first multimedia data in the data uplink transaction by using the generated index information, the apparatus is further configured to:

constructing a mapping relation between the generated index information and the first multimedia data in the data uplink transaction;

And caching the mapping relation into a cache space.

Optionally, if the consensus result of the consensus process corresponding to the first multimedia data is that the consensus is successful, the obtaining module obtains the index information of the first multimedia data and the first multimedia embedding feature of the first multimedia data based on the consensus data, including:

if the consensus result of the target block is successful, extracting index information of the first multimedia data from the target block;

acquiring first multimedia data with a mapping relation with the extracted index information in a cache space;

and performing embedding processing on the first multimedia data acquired in the cache space to generate a first multimedia embedding feature.

Optionally, after storing the first multimedia data in the multimedia database and storing the first multimedia embedded feature in the feature database, the apparatus is further configured to:

and deleting the mapping relation between the cached first multimedia data and the index information of the first multimedia data in the cache space.

Optionally, the method for generating index information of the first multimedia data in the data uplink transaction by the uplink module includes:

carrying out hash calculation on the first multimedia data to generate a hash value of the first multimedia data;

Taking the hash value of the first multimedia data as index information of the first multimedia data;

wherein the first multimedia data refers to any one of the following: image data, video data, text data, audio data, teletext data.

Optionally, the acquiring module acquires a procedure of the first multimedia embedding feature, including:

acquiring a multimedia embedded network;

and calling a multimedia embedding network to embed the first multimedia data, and generating a first multimedia embedding feature.

Optionally, the device is further configured to:

acquiring a first query request sent by a client; the first query request comprises second multimedia data, and is used for querying multimedia data matched with the second multimedia data;

calling a multimedia embedding network to embed the second multimedia data, and generating a second multimedia embedding feature of the second multimedia data;

acquiring multimedia embedded features similar to the second multimedia embedded features from a feature database as first matching embedded features, and taking index information associated with the first matching embedded features as first matching index information;

acquiring multimedia data associated with the first matching index information from a multimedia database as a first query result of a first query request, and returning the first query result to the client;

Wherein the multimedia embedding network is used to generate similar multimedia embedding features for the matched multimedia data.

Optionally, the method for obtaining, in the feature database, a multimedia embedded feature similar to the second multimedia embedded feature as the first matching embedded feature includes:

acquiring feature differences between the second multimedia embedded features and the multimedia embedded features in the feature database respectively;

sequencing each multimedia embedded feature in the feature database according to the sequence from small to large of feature difference between each multimedia embedded feature in the feature database and the second multimedia embedded feature, so as to obtain the sequenced multimedia embedded feature;

taking the first L multimedia embedded features in the sequenced multimedia embedded features as first matching embedded features; l is a positive integer.

Optionally, after the first query result is returned to the client, the apparatus is further configured to:

if the first indication information sent by the client is obtained and is used for indicating that the returned first query result is inaccurate, network parameters of the multimedia embedded network are optimized based on the first query result and the second multimedia data, and the optimized multimedia embedded network is obtained;

Wherein optimizing network parameters of the multimedia embedded network based on the first query result and the second multimedia data comprises: network parameters of the multimedia embedding network are optimized, so that feature differences between the multimedia embedding features generated by the multimedia embedding network on the first query result and the multimedia embedding features generated on the second multimedia data are increased.

Optionally, the first query request is initiated by the target object in the client, and the first query request includes an object signature of the target object;

the method for generating the second multimedia embedding characteristic of the second multimedia data by calling the multimedia embedding network to embed the second multimedia data comprises the following steps:

performing verification processing on the first query request by adopting an object signature, and if the first query request is successfully verified, judging whether the target object has the query authority on the multimedia data in the multimedia database;

and if the target object has the query authority for the multimedia data in the multimedia database, calling the multimedia embedding network to embed the second multimedia data, and generating a second multimedia embedding feature.

Optionally, the device is further configured to:

Acquiring a second query request sent by a client; the second query request comprises target text description information and is used for querying multimedia data described by the target text description information;

calling a text embedding network to embed the target text description information, and generating text embedding characteristics of the target text description information;

acquiring multimedia embedded features similar to text embedded features of target text description information from a feature database, taking the multimedia embedded features as second matching embedded features, and taking index information associated with the second matching embedded features as second matching index information;

and acquiring multimedia data associated with the second matching index information from the multimedia database as a second query result of the second query request, and returning the second query result to the client.

Optionally, the multimedia embedding feature in the feature database is generated by calling a multimedia embedding network, and the text embedding network is obtained by training the multimedia embedding feature generated based on the multimedia embedding network;

the multimedia embedding network is used for generating similar multimedia embedding characteristics for the matched multimedia data, the text embedding network is used for generating text embedding characteristics similar to the multimedia embedding characteristics of the multimedia data described by the text describing information for the text describing information, and the multimedia embedding characteristics of the multimedia data described by the text describing information are generated by the multimedia embedding network.

Optionally, after returning the second query result to the client, the apparatus is further configured to:

if the client side is obtained to send second indication information, and the second indication information is used for indicating that the returned second query result is inaccurate, optimizing network parameters of the text embedded network based on the second query result and the target text description information, and obtaining an optimized text embedded network;

wherein optimizing network parameters of the text embedded network based on the second query result and the target text description information comprises: and optimizing network parameters of the text embedding network to increase feature differences between text embedding features generated by the text embedding network on the target text description information and multimedia embedding features of the second query result.

Optionally, the device acquires, as a second matching embedded feature, a multimedia embedded feature similar to a text embedded feature of the target text description information in a feature database, where the method includes:

acquiring characteristic differences between text embedded characteristics of the target text description information and each multimedia embedded characteristic in a characteristic database;

according to the sequence from small to large of feature differences between each multimedia embedded feature in the feature database and the text embedded feature of the target text description information, sequencing each multimedia embedded feature in the feature database to obtain sequenced multimedia embedded features;

Taking the first M multimedia embedded features in the ordered multimedia embedded features as second matching embedded features; m is a positive integer.

In one aspect, the present application provides a computer device including a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform a method in one aspect of the present application.

In one aspect, the present application provides a computer readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the method of one of the above aspects.

According to one aspect of the present application, a computer program product is provided, the computer program product comprising a computer program stored in a computer readable storage medium. The processor of the computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program to cause the computer device to execute the method provided in various optional manners of the above aspect and the like.

The method comprises the steps of executing uplink processing aiming at first multimedia data in a blockchain network, wherein the uplink processing comprises consensus processing corresponding to the first multimedia data; in the process of executing the uplink processing aiming at the first multimedia data, if the consensus result of the consensus processing corresponding to the first multimedia data is that the consensus is successful, acquiring index information of the first multimedia data and first multimedia embedding characteristics of the first multimedia data based on the consensus data; storing the first multimedia data in association with the multimedia database based on the index information of the first multimedia data, and storing the first multimedia embedded feature in association with the feature database based on the index information of the first multimedia data; the corresponding consensus result of the consensus processing in the blockchain network is multimedia data with successful consensus, and associated inquiry based on a multimedia database and a characteristic database is supported. Therefore, according to the method provided by the application, for the multimedia data (such as the first multimedia data) with successful consensus as the consensus result of the corresponding consensus processing in the blockchain network, the multimedia data can be stored into the multimedia database through the index information of the multimedia data, the multimedia embedded features (such as the first multimedia embedded features) of the multimedia data can be stored into the feature database through the index information of the multimedia data, and then, the related query can be carried out on the multimedia data in the multimedia database and the feature database through the index information of the multimedia data, so that the query mode of the multimedia data (such as the first multimedia data) which is related to the uplink processing in the blockchain network is enriched.

Drawings

In order to more clearly illustrate the technical solutions of the present application or the prior art, the following description will briefly introduce the drawings that are required to be used in the embodiments or the prior art descriptions, it is obvious that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic diagram of a data processing architecture according to an embodiment of the present application;

FIG. 2 is a flowchart of a data processing method of a blockchain network according to an embodiment of the present disclosure;

fig. 3 is a schematic view of a scenario for querying multimedia data according to an embodiment of the present application;

fig. 4 is a schematic flow chart of a data query method according to an embodiment of the present application;

FIG. 5 is a schematic view of another scenario for querying multimedia data according to an embodiment of the present application;

FIG. 6 is a flowchart illustrating another data query method according to an embodiment of the present disclosure;

FIG. 7 is a schematic view of yet another scenario for querying multimedia data according to an embodiment of the present application;

FIG. 8 is a schematic block chain node architecture according to one embodiment of the present disclosure;

Fig. 9 is a schematic flow chart of a node configuration method provided in an embodiment of the present application;

fig. 10 is a schematic flow chart of a data uplink method according to an embodiment of the present application;

fig. 11 is a flow chart of a method for directly querying pictures according to an embodiment of the present application;

fig. 12 is a flowchart of a picture contract query method provided in an embodiment of the present application;

FIG. 13 is a schematic diagram of a block chain network data processing apparatus according to an embodiment of the present application;

fig. 14 is a schematic structural diagram of a computer device according to an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, are intended to be within the scope of the present application.

The present application relates to blockchain related technology. Blockchains are novel application modes of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanisms, encryption algorithms, and the like. The Blockchain (Blockchain), which is essentially a decentralised database, is a string of data blocks that are generated by cryptographic means in association, each data block containing a batch of information of network transactions for verifying the validity of the information (anti-counterfeiting) and generating the next block. The blockchain may include a blockchain underlying platform, a platform product services layer, and an application services layer.

The blockchain underlying platform may include processing modules for user management, basic services, smart contracts, and operational management. The user management module is responsible for identity information management of all blockchain participants, including maintenance of public and private key generation (account management), key management, maintenance of corresponding relation between the real identity of the user and the blockchain address (authority management) and the like, and under the condition of authorization, supervision and audit of transaction conditions of certain real identities, and provision of rule configuration (wind control audit) of risk control; the basic service module is deployed on all block chain node devices, is used for verifying the validity of a service request, recording the service request on a storage after the effective request is identified, for a new service request, the basic service firstly analyzes interface adaptation and authenticates the interface adaptation, encrypts service information (identification management) through an identification algorithm, and transmits the encrypted service information to a shared account book (network communication) in a complete and consistent manner, and records and stores the service information; the intelligent contract module is responsible for registering and issuing contracts, triggering contracts and executing contracts, a developer can define contract logic through a certain programming language, issue the contract logic to a blockchain (contract registering), invoke keys or other event triggering execution according to the logic of contract clauses to complete the contract logic, and simultaneously provide a function of registering contract upgrading; the operation management module is mainly responsible for deployment in the product release process, modification of configuration, contract setting, cloud adaptation and visual output of real-time states in product operation, for example: alarms, managing network conditions, managing node device health status, etc.

The platform product service layer provides basic capabilities and implementation frameworks of typical applications, and developers can complete the blockchain implementation of business logic based on the basic capabilities and the characteristics of the superposition business. The application service layer provides the application service based on the block chain scheme to the business participants for use.

The present application may perform query processing on multimedia data related to uplink processing in a blockchain network in a richer manner, and particularly, reference may be made to the following description of the corresponding embodiment of fig. 3.

The application also relates to artificial intelligence related technology. Among these, artificial intelligence (Artificial Intelligence, AI) is the theory, method, technique and application system that uses a digital computer or a digital computer-controlled machine to simulate, extend and extend human intelligence, sense the environment, acquire knowledge and use knowledge to obtain optimal results. In other words, artificial intelligence is an integrated technology of computer science that attempts to understand the essence of intelligence and to produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence, i.e. research on design principles and implementation methods of various intelligent machines, enables the machines to have functions of sensing, reasoning and decision.

The artificial intelligence technology is a comprehensive subject, and relates to the technology with wide fields, namely the technology with a hardware level and the technology with a software level. Artificial intelligence infrastructure technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and other directions.

The present application relates generally to machine learning in artificial intelligence. Machine Learning (ML) is a multi-domain interdisciplinary, and relates to multiple disciplines such as probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, etc., and it is specially studied how a computer simulates or implements Learning behavior of a human being to obtain new knowledge or skill, and reorganizes the existing knowledge structure to continuously improve its own performance. Machine learning is the core of artificial intelligence, a fundamental approach to letting computers have intelligence, which is applied throughout various areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, confidence networks, reinforcement learning, transfer learning, induction learning, teaching learning, and the like.

The method and the device can perform embedding processing on the multimedia data through a multimedia embedding network obtained through machine learning to obtain the embedding characteristics of the multimedia data, and perform embedding processing on text description information of the multimedia data through a text embedding network obtained through machine learning to obtain the embedding characteristics of the text description information, and subsequently, can realize associated query on the multimedia data processed in a uplink mode based on the embedding characteristics of the multimedia data or the embedding characteristics of the text description information, and can be particularly referred to as description in an embodiment corresponding to the following fig. 3.

Firstly, it should be noted that all data (such as multimedia data, related data of embedded features, transaction signatures, etc.) collected in the present application are collected under the condition that the object (such as a user, an organization, or an enterprise) to which the data belongs agrees and authorizes, and the collection, use, and processing of the related data need to comply with related laws and regulations and standards of related countries and regions.

Here, a description will be given of a related art concept to which the present application relates:

intelligent contract: smart contract is a computer protocol that aims to propagate, verify, or execute contracts in an informative manner. Smart contracts allow trusted transactions to be made without third parties, which transactions are traceable and irreversible.

Milvus: the open-source vector similarity search engine can quickly and efficiently compare and search large-scale vector data, supports various similarity measurement modes and query sentences, and provides extensible plug-ins and API interfaces. The goal of Milvus is to be a vector computation and similarity search criteria solution in machine learning and artificial intelligence applications.

Vector: vectors generally refer to mathematical quantities having directions and magnitudes, generally represented by arrows, the lengths of which represent their magnitudes, and the directions of which represent their directions. In computer science and data science, a vector generally refers to a point or coordinate in a multidimensional space. For example, in two-dimensional space, a vector may be represented as an ordered triplet, and in three-dimensional space, a vector may be represented as an ordered triplet. Whereas in machine learning and deep learning, vectors are typically represented as a set of numbers that represent data such as pictures, audio, text, etc. The similarity between two data may be calculated using vectors, typically measured using cosine similarity, euclidean distance, etc.

And (3) hash: hash, also known as Hash, is a method of mapping arbitrary length data into a fixed length sequence of numbers. The hash function may convert the input data into a hash value of a fixed size that is difficult to derive in the reverse direction. Hash functions are widely used in the fields of cryptography, data integrity checking, hash storage, hash lookup, and the like.

Wherein the hash function has the following characteristics:

1. fixed length: the hash function maps the input data to a hash value of a fixed length, typically tens or hundreds of bits.

2. Uniqueness: the hash function must generate different hash values for different input data so that hash collisions can be avoided as much as possible.

3. Irreversibility: the hash function is unidirectional, i.e. it is easy to calculate the hash value, but it is not possible to derive the original data from the hash value.

4. Hashing: the hash value generated by the hash function should be very different for small changes in the input data, so that it can be ensured that the hash function can effectively allocate data locations and reduce hash collisions.

A transaction pool: transaction Pool, also known as memboost. Refers to the collection of transactions waiting to be written to a chunk that are not packed into the chunk. In a blockchain network, each node maintains a pool of transactions for storing new transactions generated by users. When a user initiates a transaction, the transaction is broadcast throughout the network. If the transaction meets the transaction rules and validation rules of the blockchain network, the transaction is added to the transaction pool of the node. Transactions in the transaction pool need to wait to be verified and validated before waiting to be packed into blocks.

Contract query: the blockchain contract query is a query way that a smart contract can be run on the blockchain that allows a user to query the smart contract for readable public data or perform some read-only operation without creating a new transaction. Smart contracts are a mechanism for implementing automation and programmable logic on a blockchain.

Among other benefits, using a contract query is that no changes are made to blockchain state because the contract query is performed with only the contract data read and no transactions are performed. Query results will be returned to the user, which can be used to execute specific business logic or applied as a data source to other applications.

Referring to fig. 1, fig. 1 is a schematic structural diagram of a data processing architecture according to an embodiment of the present application. As shown in fig. 1, the data processing architecture may include clients and a blockchain network, and may actually include a client cluster, and, since the principle of data processing by each client may be the same, a client is illustrated here as an example, where the client may be present in a terminal device of a user. A plurality of blockchain nodes may be included in the blockchain network, each of which may be comprised of a corresponding node device, which may be a server, for example. Each block link point in the block chain network can be connected with each other through a network so as to perform network communication.

The servers described above may be independent physical servers, or may be server clusters or distributed systems formed by a plurality of physical servers, or may be cloud servers that provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs (distribution networks), and basic cloud computing services such as big data and artificial intelligence platforms. The terminal device where the client is located may be: intelligent terminals such as smart phones, tablet computers, notebook computers, desktop computers, intelligent televisions and the like.

A user may initiate a data uplink transaction, which may be a transaction for initiating a uplink process for first multimedia data, to a blockchain network through a client. After the data uplink transaction is successfully agreed in the blockchain network, the index information of the first multimedia data can be stored in an uplink mode, the first multimedia data can be stored in an under-chain multimedia database, and the multimedia embedded features of the first multimedia data can be stored in an under-chain feature database. By storing the index information of the first multimedia data in a uplink manner, instead of directly storing the first multimedia data in a uplink manner, the on-link storage space of the blockchain network can be saved.

Subsequently, the multimedia data (such as the first multimedia data) subjected to the related uplink processing in the blockchain network can be subjected to the related query through the multimedia database and the feature database, for example, the multimedia data in the multimedia database can be subjected to the query through the multimedia embedding feature in the feature database, and the specific process can also be referred to as description in the following embodiments.

By adopting the method provided by the application, richer query processing of the multimedia data for relevant uplink processing can be realized through the constructed multimedia database and the feature database, for example, the multimedia data can be queried through the multimedia embedded features of the multimedia data, so that the query of the multimedia data for relevant uplink processing in the blockchain network is more flexible.

Referring to fig. 2, fig. 2 is a flowchart of a data processing method of a blockchain network according to an embodiment of the present application. The execution body in the embodiment of the present application may be a data processing device, where the data processing device may be one computer device or a cluster of computer devices formed by multiple computer devices, and the computer device may be a server, a terminal device, or other devices, which is not limited to this. As shown in fig. 2, the method may include:

Step S101, performing a uplink process for the first multimedia data in the blockchain network, wherein the uplink process includes a consensus process corresponding to the first multimedia data.

Alternatively, a plurality of blockchain nodes may be included in the blockchain network, and a blockchain node may be formed of one or more node devices, so it will be appreciated that the data processing device may be a node device of any blockchain node in the blockchain network, in other words, each blockchain node may perform operations performed by the data processing device described below.

The first multimedia data may be any multimedia data to be subjected to the relevant uplink processing, and the first multimedia data may be submitted by a client, where the client may have a connection with a blockchain network, and the client may be understood to be a front end of the blockchain network, and the blockchain network may be understood to be a background of the client.

Alternatively, the multimedia data (e.g., the first multimedia data) described in the present application may be any of the following: image data, video data, audio data, text data, teletext data, and the like. The multimedia data described in the present application is specifically what data may be determined according to the actual application scenario, and this is not limited.

For example, the data processing apparatus may perform a uplink process for the first multimedia data in the blockchain network, the process may include:

the data processing device may obtain a data-uplink transaction, which may be a transaction initiated by a client of the user, which may contain the first multimedia data, which may be a transaction for performing a uplink process for the first multimedia data. Alternatively, the user may be referred to as an object, a transaction initiation object, or a transaction initiator, and the user may be embodied in a corresponding user account (also referred to as an object account) in the client.

The data processing device may thus perform a related uplink process on the data uplink transaction in the blockchain network, wherein the uplink process on the data uplink transaction may be understood as the uplink process for the first multimedia data, as described below.

The data uplink transaction may have a transaction signature, where the transaction signature may be obtained by performing signature processing on the data uplink transaction using a private key of a user (which may be understood to be a private key of a client of the user), for example, the transaction signature may be obtained by encrypting a hash value of the data uplink transaction using a private key of the user initiating the transaction, and the transaction signature may be submitted to the data processing device by the client when the client submits the data uplink transaction to the data processing device.

Thus, the data processing device may perform a verification process on the data-on transaction by means of the transaction signature: the public key of the user may be public and each blockchain node may hold a public key of the client (i.e., the user's public key) that may be used to decrypt data encrypted by the user's private key (e.g., a transaction signature of a data uplink transaction).

The data processing device may decrypt the transaction signature of the data uplink transaction by using the public key of the user to obtain a real hash value, the data processing device may perform hash operation on the data uplink transaction to generate a hash value of the data processing device, and the generated hash value may be referred to as a hash value to be verified, so the data processing device may compare the hash value to be verified with the real hash value, and if the hash value to be verified is compared to be consistent with (i.e. identical to) the real hash value, it indicates that the data uplink transaction is a legal and real transaction, i.e. verification on the data uplink transaction is successful, the data uplink transaction may be executed (e.g. a virtual machine is invoked to execute the data uplink transaction). Otherwise, if the hash value to be verified is inconsistent (i.e. different) with the real hash value, the data uplink transaction is not legal and real transaction, i.e. the data uplink transaction fails to be verified, the data uplink transaction can be discarded and an error is reported to the client.

More, if the data uplink transaction is successfully executed, the data processing device may generate index information of the first multimedia data in the data uplink transaction, and optionally, the index information may be calculated in a manner that: the data processing device may perform hash computation on the first multimedia data to generate a hash value of the first multimedia data, and the data processing device may perform hash computation on the first multimedia data by using a set hash algorithm, so as to generate the hash value of the first multimedia data. Further, the data processing apparatus may use the hash value of the first multimedia data as index information that is the first multimedia data.

The data processing device may further perform replacement processing on the first multimedia data in the data uplink transaction by using the generated index information, that is, replace the first multimedia data in the data uplink transaction with the index information of the first multimedia data, and further package the data uplink transaction (including the index information of the first multimedia data without the first multimedia data) after the replacement processing into the target block.

The premise of the data processing device obtaining the target block through packing may be that the data processing device is a node device of a block outlet node in the blockchain network, the block outlet node may be a blockchain node used for packing the block and sharing the uplink in the blockchain network, the block outlet node may be a consensus node, and optionally, the block outlet node may be alternatively acted by each consensus node in the blockchain network.

The blockchain network may include a consensus network, which may include a plurality of consensus nodes (a plurality of consensus nodes). Furthermore, the data processing device may broadcast the target block obtained by the packaging to a consensus network (may include a plurality of consensus nodes) in the blockchain network, so as to perform a consensus process on the target block in the consensus network, and if a consensus result of the target block in the consensus network is a consensus success (that is, a consensus result of the target block in the blockchain network is a consensus success), the target block may be uplink to the blockchain network, so that each blockchain node (including each consensus node and each service node, etc., the service node may be a blockchain node other than the consensus node) in the blockchain network may store the target block in a respective ledger (blockchain ledger).

The target block includes index information of the first multimedia data, and it is understood that the consensus process corresponding to the first multimedia data may refer to a consensus process of the target block, and the consensus data may refer to the target block.

In the method, the target block containing the index information of the first multimedia data is subjected to consensus processing, and then the target block is subjected to uplink processing, so that the uplink processing of the first multimedia data can be realized through the index information of the first multimedia data, the record of the first multimedia data on the chain can be realized through the index information of the first multimedia data recorded on the chain, and the first multimedia data can not be directly uplink by the way, but the index information of the first multimedia data is used for replacing the first multimedia data to carry out uplink, thereby saving the occupation of the uplink data in the blockchain network to the storage space of the blockchain account book and improving the running performance of the blockchain network.

Step S102, in the process of executing the uplink processing for the first multimedia data, if the consensus result of the consensus processing corresponding to the first multimedia data is that the consensus is successful, the index information of the first multimedia data and the first multimedia embedding feature of the first multimedia data are obtained based on the consensus data.

Optionally, before the replacing processing is performed on the first multimedia data in the data uplink transaction by using the generated index information, the data processing device may further construct a mapping relationship between the generated index information and the first multimedia data in the data uplink transaction, and may add (i.e. cache) the mapping relationship into the cache space. Wherein the buffer space can be a space for short-term storage of relevant data of the business process.

Thus, in performing the uplink processing for the first multimedia data, if the consensus result of the consensus processing corresponding to the first multimedia data is that the consensus is successful (i.e. if the target block is successfully consensus in the blockchain network), the data processing device may acquire the index information of the first multimedia data and the first multimedia embedding feature of the first multimedia data based on the consensus data, and the process may include:

The data processing apparatus may extract index information of the first multimedia data in a target block (i.e., common data), and may acquire the first multimedia data having a mapping relationship with the extracted index information in a buffer space.

Furthermore, the data processing device may perform embedding processing on the first multimedia data acquired in the buffer space, that is, may generate an embedding feature of the first multimedia data, may refer to the embedding feature of the multimedia data as a multimedia embedding feature, further may refer to the multimedia embedding feature of the first multimedia data as a first multimedia embedding feature, and optionally, the first multimedia embedding feature may be a feature vector.

Optionally, the process of acquiring the first multimedia embedding feature may include: the data processing device may obtain a multimedia embedding network, which may be a pre-trained network that may be used to embed multimedia data to generate multimedia embedding features for the multimedia data.

Thus, the data processing device may invoke the multimedia embedding feature to perform an embedding process on the first multimedia data to generate a first multimedia embedding feature of the first multimedia data.

The multimedia embedding network may be trained to generate similar multimedia embedding features for mutually matched multimedia data, and the feature differences between the similar multimedia embedding features are smaller. Wherein the matched multimedia data may also refer to similar multimedia data. For example, if the multimedia data is image data, several images containing similar carts may be mutually matched images.

The multimedia embedding network may be obtained by performing multiple iterative training on the multimedia embedding network to be trained through a plurality of first positive sample pairs and a plurality of first negative sample pairs, where one first positive sample pair may include two matched multimedia data, and one first negative sample pair may include two unmatched multimedia data, so that in the training process of the multimedia embedding network to be trained, the multimedia embedding network to be trained may generate similar multimedia embedding features (i.e., multimedia embedding features with small feature differences) on the two multimedia data in the first positive sample pair, and the multimedia embedding network to be trained may generate dissimilar multimedia embedding features (i.e., multimedia embedding features with large feature differences) on the two multimedia data in the first negative sample pair, and network parameters of the multimedia embedding network to be trained are sequentially and continuously adjusted, thereby obtaining the multimedia embedding network, that is, the trained network.

Step S103, storing the first multimedia data in association with the multimedia database based on the index information of the first multimedia data, and storing the first multimedia embedded feature in association with the feature database based on the index information of the first multimedia data.

Alternatively, the data processing device may store the first multimedia data in association with the multimedia database through the obtained index information of the first multimedia data, that is, the multimedia data in the multimedia database may be stored in association with the index information of the multimedia data. Alternatively, the multimedia database may be a key-value database, the key may represent index information of multimedia data, and the value may represent multimedia data.

The data processing device may store the first multimedia embedded feature of the first multimedia data in association with the feature database through the obtained index information of the first multimedia data, that is, in the feature database, the index information of the multimedia data may be stored in association with the multimedia embedded feature of the multimedia data. Alternatively, the feature database may be a key-value database, where the key may represent index information of multimedia data, and the value may represent multimedia embedded features of the multimedia data.

After the first multimedia data is stored in the multimedia database and the first multimedia embedded feature is stored in the feature database, the data processing device may delete the mapping relationship between the cached first multimedia data and the index information of the first multimedia data in the cache space to release the cache space.

The storage flow of the multimedia data stored in the multimedia database may be the same as the flow of the first multimedia data stored in the multimedia database, that is, the multimedia data stored in the multimedia database may also be multimedia data relevant to the uplink processing in the blockchain network.

Similarly, the feature database may also store a plurality of multimedia embedding features, and the storage flow of the multimedia embedding features stored in the feature database may be the same as the flow of the first multimedia embedding features of the first multimedia data stored in the feature database, that is, the multimedia embedding features stored in the feature database may be embedding features of multimedia data in the multimedia database, and the multimedia embedding features stored in the feature database and the multimedia data stored in the multimedia database may be in one-to-one correspondence, and may be associated with each other by index information of the multimedia data.

Alternatively, the feature database may be a Milvus database. Both the feature database and the multimedia database may be stored off-chain in the blockchain node, i.e., the feature database and the multimedia database may not be stored on-chain. The feature database and the multimedia database may be stored in some of the blockchain nodes in the blockchain network, or may be stored in all of the blockchain nodes in the blockchain network, and may specifically be set according to the actual application scenario.

The feature database and the multimedia database stored therein may be referred to as a Milvus node, which may also be referred to as a data search node, and subsequently, through the feature database and the multimedia database stored in the Milvus node, an associated query for multimedia data (such as first multimedia data) having a successful consensus as a result of the consensus processing corresponding to the blockchain network may be implemented, where the associated query supports a query for multimedia data through relevant embedded features, and the process may be specifically described with reference to the related descriptions in the corresponding embodiments of fig. 4 and 6 below.

Referring to fig. 3, fig. 3 is a schematic view of a scenario for querying multimedia data according to an embodiment of the present application. As shown in fig. 3, the multimedia database may store multimedia data for which related uplink processing is performed and index information of the multimedia data in association, and the feature database may store multimedia embedded features of each multimedia data in the multimedia database and index information of the multimedia data in association. Subsequently, the related query for the multimedia data in the multimedia database can be realized through the multimedia embedded features in the feature database, and the specific process can be seen from the related description in the corresponding embodiment of fig. 4 and fig. 6.

Referring to fig. 4, fig. 4 is a flowchart of a data query method according to an embodiment of the present application. The process of searching (i.e., querying) matching multimedia data through multimedia data is described in a corresponding embodiment in fig. 4. As shown in fig. 4, the method may include:

step S201, a first query request sent by a client is obtained; the first query request contains second multimedia data, and the first query request is used for querying multimedia data matched with the second multimedia data.

Alternatively, the data processing device may obtain a first query request sent by the client of the user, where the first query request may include second multimedia data, where the second multimedia data may be any multimedia data, and the second multimedia data may or may not be multimedia data in the multimedia database. The first query request may be for querying multimedia data that matches the second multimedia data.

For example, the second multimedia data may be an image of a dolly, and thus the first query request may be a request for querying an image matching (e.g. similar to) the image of the dolly.

Step S202, calling a multimedia embedding network to embed the second multimedia data, and generating a second multimedia embedding feature of the second multimedia data.

Optionally, the data processing device may invoke the trained multimedia embedding network to perform an embedding process (which may be understood as a feature extraction process) on the second multimedia data in the first query request, so as to generate a multimedia embedding feature of the second multimedia data, where the multimedia embedding feature of the second multimedia data may be referred to as a second multimedia embedding feature.

In one possible implementation manner, the process of calling the multimedia embedding network to embed the second multimedia data and generating the second multimedia embedding feature of the second multimedia data may include:

the first query request may be initiated in the client by a target object, which may be any user or represent any user account. The first query request may include an object signature of the target object, where the object signature may be obtained by signing data in the first query request except for the object signature by using a private key of the target object in the client. If the first query request may be obtained by encrypting the hash value of the data except the object signature in the first query request by using the private key of the target object.

Thus, the data processing apparatus may perform a verification process on the validity of the first query request by the object signature, and the process may include: the data processing device may decrypt the object signature by using the public key of the target object to obtain a real hash value, and further, the data processing device may perform hash calculation on the data except the object signature in the first query request to obtain a hash value to be verified of the data except the object signature in the first query request.

Furthermore, the data processing device may compare the hash value to be verified with the actual hash value, and if the hash value to be verified is compared to be consistent with (i.e. the same as) the actual hash value, it indicates that the first query request is legal (i.e. indicates that the first query request is successfully verified), and may perform subsequent operations. If the hash value to be verified is inconsistent (i.e. different) with the real hash value, the first query request is not legal (i.e. the verification of the first query request fails), the first query request can be discarded, and prompt information of data query errors can be reported to the client of the target object.

If the first query request is verified to be legal, the data processing device may further determine (e.g. query) whether the target object has the query authority for the multimedia data in the multimedia database, and if the target object is queried to have the query authority for the multimedia data in the multimedia data, the data processing device may call the multimedia embedding network to perform embedding processing on the second multimedia data, so as to generate the second multimedia embedding feature.

For example, the query authority of the target object to the multimedia data in the multimedia database may be configured for the target object in advance based on a related mechanism, where the related mechanism may be determined according to an actual application scenario, and this is not limited.

Step S203, the multimedia embedded features similar to the second multimedia embedded features are obtained from the feature database as the first matching embedded features, and the index information associated with the first matching embedded features is used as the first matching index information.

Alternatively, the data processing device may acquire, in the feature database, a multimedia embedding feature similar to the second multimedia embedding feature as the first matching embedding feature, and may use index information associated with the first matching embedding feature as the first matching index information. Alternatively, there may be one or more first matching embedded features, one first matching embedded feature having a first matching index information.

Illustratively, the process of obtaining a multimedia embedding feature in the feature database that is similar to the second multimedia embedding feature may include:

the multimedia embedded feature of the multimedia data may be a feature vector. The data processing device may obtain feature differences between the second multimedia embedded feature and each multimedia embedded feature in the feature database, and the data processing device may obtain a vector distance between the second multimedia embedded feature and each multimedia embedded feature in the feature database, where the feature differences between the second multimedia embedded feature and each multimedia embedded feature in the feature database are reflected by the vector distance, for example, the larger the vector distance between the multimedia embedded features is, the larger the feature differences between the multimedia embedded features may be, otherwise, the smaller the vector distance between the multimedia embedded features is, the smaller the feature differences between the multimedia embedded features may be.

Furthermore, the data processing device may sort the respective multimedia embedded features in the feature database according to the order from small to large (e.g., the order from small to large of the vector distance) of the feature differences between the respective multimedia embedded features and the second multimedia embedded feature in the feature database, so as to obtain sorted multimedia embedded features, and the multimedia embedded features with smaller feature differences (e.g., smaller vector distance, i.e., more similar) between the feature database and the second multimedia embedded feature may be arranged in front.

Furthermore, the data processing device may use the first L multimedia embedding features in the ranked multimedia embedding features as first matching embedding features, where L is a positive integer, and the specific value of L may be determined according to the actual application scenario, that is, there may be L first matching embedding features. The L first matching embedded features may be L multimedia embedded features with the smallest feature difference between the feature database and the second multimedia embedded feature, that is, L multimedia embedded features with the smallest feature difference between the feature database and the second multimedia embedded feature may be used as multimedia embedded features similar to the second multimedia embedded feature in the feature database.

Alternatively, a first feature difference threshold (e.g., a first vector distance threshold) may be set, and a multimedia embedding feature in the feature database having a feature difference from a second multimedia embedding feature that is less than or equal to the first feature difference threshold (i.e., a vector distance that is less than or equal to the first vector distance threshold) may be used as the first matching embedding feature. The multimedia embedded feature with the feature difference between the second multimedia embedded feature and the feature database being less than or equal to the first feature difference threshold (i.e., the vector distance being less than or equal to the first vector distance threshold) can be used as the multimedia embedded feature similar to the second multimedia embedded feature.

Step S204, multimedia data associated with the first matching index information is obtained from the multimedia database and used as a first query result of the first query request, and the first query result is returned to the client.

Alternatively, the data processing device may obtain, in the multimedia database, multimedia data associated with the first matching index information (which may or may not include the first multimedia data), as a query result of the first query request, a query result of the first query request may be referred to as a first query result, where the first query result may include multimedia data associated with the first matching index information in the multimedia database, and the number of multimedia data included in the first query result is the same as the number of the first matching embedded features, and one first matching embedded feature is used to obtain one queried multimedia data.

The data processing device may return the first query result to the client, and the client may output the first query result at the client interface for viewing and browsing by the user.

Referring to fig. 5, fig. 5 is a schematic view of another scenario for querying multimedia data according to an embodiment of the present application. As shown in fig. 5, after the data processing device obtains the first query request, the multimedia embedding network may be invoked to perform embedding processing on the second multimedia data in the first query request, so as to generate a second multimedia embedding feature of the second multimedia data.

Furthermore, a multimedia embedding feature similar to the second multimedia embedding feature may be queried in the feature database as a first matching embedding feature. And the multimedia data associated with the index information associated with the first matching embedded feature can be queried in the multimedia database to be used as a first query result of the first query request, and the first query result can be returned to the client of the user.

More, after the user views the first query result in the client, the user can determine whether the first query result is accurate, if not, the user can be triggered to generate corresponding indication information, the indication information can be used for indicating that the first query result of the query is not accurate, and the user can report the indication information to the data processing device. Thus, after returning the first query result to the client, the present application may also perform the following operations:

after the first query result is returned to the client, if the data processing device obtains the first indication information sent by the client and the first indication information is used for indicating that the first query result returned to the client is inaccurate, the data processing device can optimize network parameters of the multimedia embedding network in real time through the first query result and the second multimedia data so as to obtain an optimized multimedia embedding feature, and then can generate the multimedia embedding feature of the multimedia data through the optimized multimedia embedding feature, as described below.

Wherein optimizing network parameters of the multimedia embedded network through the first query result and the second multimedia data may include: a new first negative sample pair can be constructed in real time through the first query result and the second multimedia data, a new first negative sample pair can be constructed between one multimedia data and the second multimedia data in the first query result, and further, network parameters of the multimedia embedding network can be optimized (can be understood as further training) through the new first negative sample pair, so that the multimedia embedding network can generate multimedia embedding features with larger feature differences for the multimedia data (including the multimedia data in the first query result and the second multimedia data) in the new first negative sample pair, that is, the network parameters of the multimedia embedding network can be optimized (i.e. revised), so that feature differences between the multimedia embedding features of the trained multimedia embedding network for each multimedia data in the first query result and the multimedia embedding features generated for the second multimedia data can be increased.

By adopting the method, the network parameters of the multimedia embedded network can be further optimized in real time to form a forward circulation in the actual use process of the trained multimedia embedded network (such as the process of inquiring the multimedia data through the trained multimedia embedded network), and the multimedia embedded network can be continuously optimized according to the actual use condition of a user along with the actual use of the multimedia embedded network, so that the optimized multimedia embedded network can be utilized to search (i.e. inquire) the multimedia data more in accordance with the user expectations.

Referring to fig. 6, fig. 6 is a flowchart of another data query method according to an embodiment of the present application. Fig. 6 illustrates a process of searching (i.e., querying) matched multimedia data through text description information of the multimedia data, i.e., searching corresponding multimedia data through natural language, in a corresponding embodiment. As shown in fig. 6, the method may include:

step S301, a second query request sent by a client is obtained; the second query request contains the target text description information, and the second query request is used for querying multimedia data described by the target text description information.

Alternatively, the data processing apparatus may obtain a second query request sent by the client, where the second query request may include target text description information, and the target text description information may be text information for describing multimedia data, and thus, it may be understood that the second query request may be a request for querying multimedia data described by the target text description information.

The target text description information may be understood as a search term for the multimedia data, and may have related features for describing the multimedia data therein, if the multimedia data is image data and the target text description information is "red sports car", the second query request may be a request for querying an image of the red sports car.

Step S302, a text embedding network is called to embed the target text description information, and the text embedding characteristics of the target text description information are generated.

Alternatively, the data processing device may acquire a text-embedded network, which may be pre-trained by the above-described trained multimedia-embedded network. It can be seen from the above that the multimedia embedding feature in the feature database may be generated by calling the multimedia embedding network, and the multimedia embedding network is used to generate a similar multimedia embedding feature for the matched multimedia data.

It will thus be appreciated that the text embedding network may be used to embed text description information (text information describing multimedia data) to generate embedded features of the text description information, which may be referred to as text embedded features, which may also be feature vectors. More specifically, the text embedding network may be a text embedding feature for generating text description information similar to a multimedia embedding feature of multimedia data described by the text description information, and the multimedia embedding feature of the multimedia data described by the text description information may be generated by the multimedia embedding network obtained by the training.

In other words, the text-embedding network may be trained on a text-embedding network to be trained, and the training of the text-embedding network to be trained may be several second positive sample pairs and several second negative sample pairs. A second positive sample pair may include a text description information and multimedia embedding characteristics of any multimedia data actually described by the text description information, which may be generated by the above-mentioned multimedia embedding network obtained by training. A second negative sample pair may include a text description and any multimedia embedding feature that is not actually the multimedia data described by the text description, the multimedia embedding feature also being generated by the training derived multimedia embedding network.

Therefore, in the training process of the text embedding network to be trained by adopting the plurality of second positive sample pairs and the plurality of second negative sample pairs, the text embedding network to be trained can be modified, so that the text embedding network to be trained generates text embedding characteristics similar to the multimedia embedding characteristics in the second positive sample pair for the multimedia data in the second positive sample pair (such as that the characteristic difference between the generated text embedding characteristics and the multimedia embedding characteristics in the second positive sample pair is small); and, the text-embedding network to be trained may be modified, so that the text-embedding network to be trained generates text-embedding features dissimilar to the multimedia-embedding features in the second negative-sample pair (e.g., so that the feature differences between the generated text-embedding features and the multimedia-embedding features in the second negative-sample pair are large), that is, the text-embedding network to be trained may perform corresponding learning and training on the text description information according to the above principle to generate corresponding text-embedding features.

Thus, the data processing device may invoke the trained text embedding network to embed the target text description information to generate text embedded features of the target text description information, which may be feature vectors.

Step S303, acquiring the multimedia embedded features similar to the text embedded features of the target text description information in the feature database as second matching embedded features, and taking index information associated with the second matching embedded features as second matching index information.

Alternatively, the data processing device may further acquire, in the feature database, a multimedia embedding feature similar to the text embedding feature of the target text description information as a second matching embedding feature, and may associate index information associated with the second matching embedding feature as second matching index information. The second matching embedded feature is a multimedia embedded feature similar to the text embedded feature of the target text description information. Alternatively, there may be one or more second matching embedded features, one second matching embedded feature having a second matching index information.

Illustratively, the process of obtaining the multimedia embedding feature similar to the text embedding feature of the target text description information in the feature database may include:

Similarly, the multimedia embedded feature of the multimedia data may be a feature vector. The data processing device may obtain feature differences between the text embedded feature of the target text description information and each multimedia embedded feature in the feature database, and the data processing device may obtain vector distances between the text embedded feature of the target text description information and each multimedia embedded feature in the feature database, where feature differences between the text embedded feature of the target text description information and each multimedia embedded feature in the feature database are reflected through the vector distances, for example, the larger the vector distances between the multimedia embedded feature and the text embedded feature, the larger the feature differences between the multimedia embedded feature and the text embedded feature may be, otherwise, the smaller the vector distances between the multimedia embedded feature and the text embedded feature, the smaller the feature differences between the multimedia embedded feature and the text embedded feature may be.

Furthermore, the data processing device may sort the multimedia embedded features in the feature database according to the order from small to large (such as the order from small to large of the vector distance) of the feature differences between the multimedia embedded features in the feature database and the text embedded features of the target text description information, so as to obtain the sorted multimedia embedded features, and the multimedia embedded features with smaller feature differences (such as smaller vector distance, i.e. more similar) between the feature database and the text embedded features of the target text description information may be arranged in front.

Furthermore, the data processing device may use the first M multimedia embedding features in the ordered multimedia embedding features as second matching embedding features, where M is a positive integer, and the specific value of M may be determined according to the actual application scenario, that is, there may be M second matching embedding features. The M second matching embedded features may be M multimedia embedded features with the smallest feature difference between the feature database and the text embedded feature of the target text description information, i.e. M multimedia embedded features with the smallest feature difference between the feature database and the text embedded feature of the target text description information may be used as multimedia embedded features similar to the text embedded feature of the target text description information in the feature database.

Alternatively, a second feature difference threshold (for example, a second vector distance threshold may be set), and a multimedia embedding feature whose feature difference from the text embedding feature of the target text description information in the feature database is less than or equal to the second feature difference threshold (that is, the vector distance is less than or equal to the second vector distance threshold) may be set as the second matching embedding feature. I.e. the multimedia embedding feature in the feature database having a feature difference from the text embedding feature of the target text description information that is less than or equal to the second feature difference threshold (i.e. a vector distance that is less than or equal to the second vector distance threshold) can be used as a multimedia embedding feature that is similar to the text embedding feature of the target text description information.

Step S304, the multimedia data associated with the second matching index information is obtained from the multimedia database and used as a second query result of the second query request, and the second query result is returned to the client.

Alternatively, the data processing device may obtain the multimedia data associated with the second matching index information from the multimedia database, and as the query result of the second query request, the query result of the second query request may be referred to as a second query result. The second query result may include multimedia data associated with second matching index information in the multimedia database, where the second query result includes the same number of multimedia data as the second matching embedded feature, and one second matching embedded feature is used to obtain one piece of multimedia data that is queried.

The data processing device may return the second query result to the client, and the client may output the second query result at the client interface for viewing and browsing by the user.

Referring to fig. 7, fig. 7 is a schematic view of another scenario for querying multimedia data according to an embodiment of the present application. As shown in fig. 7, after the data processing device obtains the second query request, the text embedding network may be invoked to embed the target text description information in the second query request, so as to generate the text embedding feature of the target text description information.

Furthermore, a multimedia embedding feature similar to the text embedding feature of the target text description information may be queried in the feature database as a second matching embedding feature. And the multimedia data associated with the index information associated with the second matching embedded feature can be queried in the multimedia database to be used as a second query result of the second query request, and the second query result can be returned to the client of the user.

More, after the user checks the second query result in the client, the user can determine whether the second query result is accurate, if not, the user can be triggered to generate corresponding indication information, the indication information can be used for indicating that the returned second query result is not accurate, and the user can report the indication information to the data processing device. Thus, after returning the second query result to the client, the present application may also perform the following operations:

after the second query result is returned to the client, if the data processing device obtains the second indication information sent by the client and the second indication information is used for indicating that the second query result returned to the client is inaccurate, the data processing device can optimize the network parameters of the text embedding network in real time through the second query result and the target text description information to obtain optimized text embedding characteristics, and then can generate the text embedding characteristics of the text description information through the optimized text embedding characteristics, as described below.

Wherein optimizing the network parameters of the text embedding network through the second query result and the target text description information may include: a new second negative sample pair can be constructed in real time through the second query result and the target text description information, a new second negative sample pair can be constructed between the multimedia embedding feature of one multimedia data in the second query result and the target text description information, and further, the network parameters of the text embedding network can be optimized (can be understood as further training) through the new second negative sample pair, so that the text embedding network can generate, for the target text description information in the new second negative sample pair, a text embedding feature with larger feature difference from the multimedia embedding feature in the second negative sample pair, that is, the network parameters of the text embedding network can be optimized (i.e. corrected), so that the feature difference between the text embedding feature generated by the training text embedding network for the target text description information and the multimedia embedding feature of the multimedia data in the second query result (i.e. the multimedia embedding feature of the second query result) can be increased.

By adopting the method, the network parameters of the text embedded network can be further optimized in real time to form a forward circulation in the actual use period of the trained text embedded network (such as the period of inquiring the multimedia data through the trained text embedded network), and the network parameters of the text embedded network can be further optimized in real time according to whether the multimedia data actually inquired through the text embedded network are accurate, so that the text embedded network can be continuously optimized according to the actual use condition of a user along with the actual use of the text embedded network, and the optimized text embedded network can be utilized to search (namely inquire) the multimedia data more in accordance with the user expectations.

Optionally, when the client in the present application queries the corresponding multimedia data through the multimedia data or the text description information, the client may query any blockchain node, for example, may send a query request (such as the first query request or the second query request) to any blockchain node to query the corresponding multimedia data. The client for initiating the data uplink transaction for the first multimedia data and the client for querying the multimedia data may be the same client or different clients, and may specifically be determined according to an actual application scenario. It will be appreciated that the client that makes the multimedia data query and the client that initiates the data uplink transaction for the multimedia data may be any client that has access to the blockchain network.

If the client requests to query the multimedia data, the block link point stores the multimedia database and the feature database, and the block link node can query the corresponding multimedia data based on the multimedia database and the feature database stored by the client. If the client requests to query the blockchain node of the multimedia data, the blockchain node may forward the obtained query request to other blockchain nodes storing the multimedia database and the feature database, so as to request the blockchain node storing the multimedia database and the feature database to query the corresponding multimedia data according to the query request, and may return the queried multimedia data to the blockchain node requested by the client, and further, the blockchain node requested by the client may return the queried multimedia data (such as the first query result or the second query result) to the client.

Because the blockchain network is natural and is a distributed network, the multimedia database and the feature database can be stored in a distributed manner through a plurality of blockchain nodes in the blockchain network, so that the storage safety and reliability of the multimedia database and the feature database are ensured, and even if a small number of blockchain nodes storing the multimedia database and the feature database are down or have related fault problems, the small number of blockchain nodes can restore the multimedia database and the feature database through other blockchain nodes which normally operate and store the multimedia database and the feature database, thereby preventing the situation that the multimedia database and the feature database are completely lost and preventing the problem that the multimedia database and the feature database cannot be accessed due to the fact that the small number of blockchain nodes storing the multimedia database and the feature database are down.

By adopting the method, vector storage (such as storage of multimedia embedded features of multimedia data) is carried out when a block (such as a target block) is generated, so that a subsequent storage and searching mechanism for multimedia data such as images, videos, graphics, audios and the like can be supported by directly searching the multimedia data (such as searching the graphics in a graphics mode) and searching the multimedia data (such as searching the graphics in a natural language mode) through the multimedia data, and the storage and searching mechanism of the blockchain network for the multimedia data such as images, videos, graphics, audios and the like is greatly serviced.

In order to describe in more detail the configuration of the blockchain node that can be used for performing the multimedia data query in the present application, please refer to fig. 8, fig. 8 is a schematic diagram of the configuration of a blockchain node according to an embodiment of the present application. As shown in fig. 8, the blockchain node may include a plurality of structural modules, which may include a network module, an authentication module, a query service module, a transaction pool module, a blockscheduling module, a virtual machine module, a model module, and a storage module, the functions and roles of each of which are described below.

And (3) a network module: the block link points are used for externally performing network interaction modules, such as can be used for performing network interaction with a client of a user. Such as a user sending a data uplink transaction or query request to a blockchain link node, may be sent to a network module of the blockchain node.

And an authentication module: may be used to verify the signature of the transaction (e.g., the transaction signature described above) or the signature of the query request (e.g., the object signature described above) and may be used to determine the rights of the requesting user, such as whether the user has rights to query multimedia data in the multimedia database, or whether the user has rights to initiate a data-upload transaction. If the user does not have the corresponding right, the corresponding initiated operation (such as the data uplink operation or the multimedia data query operation) cannot be performed on the user.

And (5) a query service module: and a module for responding to the query request of the network module (which may be sent by the user through the client), and executing the responding query service.

A transaction pool module: and a module for caching the transaction and packaging the transaction, wherein the packaged transaction can be prepared to be executed in the block scheduling module.

And a block scheduling module: and means for scheduling transactions in the block and generating a new block. The method comprises the steps of distributing the packaged transaction to a virtual machine module for execution, and generating a new block after the transaction execution is completed.

Virtual machine module: a module for executing transactions, having a Milvus client, which may be a client inside a virtual machine module (a client not belonging to a user, which may not be perceived by the user), for delivering a user-initiated contract-type query request to a blockchain node in the background for querying corresponding multimedia data. The virtual machine module may include a contract repository (repository for holding all intelligent contracts) and may be used to launch a corresponding contract process, such as a contract process that makes multimedia data queries.

The method adopts the contract to inquire the multimedia data, so that the inquiry mode of the multimedia data can be further enriched, such as batch inquiry of the multimedia data or inquiry of multiple conditions (such as combination of the multimedia data and text description information, inquiry of the multimedia data together) can be carried out through the contract, and the like.

And a consensus module: and means for conducting consensus votes.

Model module: a module storing a text-embedding network (which may be referred to as a natural language vectorization model) and a multimedia-embedding network (which may be referred to as a multimedia vectorization model). The multimedia data may be vectorized (e.g., embedded) over a multimedia embedding network to generate multimedia embedding of the multimedia data, and the text description information may be vectorized over a text embedding network to generate text embedding characteristics of the text description information. And can query multimedia data through the embedded features after vectorization processing.

And a storage module: and a module for storing the blockchain data. Such as Milvus vector databases (such as the feature database described above), multimedia databases, state databases (databases for preserving the overall state of the blockchain network), and blockbooks (for storing blocks on chains) may be stored.

Alternatively, each blockchain node in the blockchain network may have each structural module, and only the storage modules of each blockchain node may store all the multimedia database and the feature database, or may store part of the multimedia database and the feature database.

The associated initialization configuration flow for the blockchain node in this application is described further below. Referring to fig. 9, fig. 9 is a flowchart of a node configuration method according to an embodiment of the present application. As shown in fig. 9, the method may include:

in step S401, the blockchain network in the present application may be a federated chain network, so first, the number of blockchain nodes in the federated chain network (i.e. how many blockchain nodes need to be configured) and the communication manner and communication protocol between the blockchain nodes are determined.

Step S402 is described below taking multimedia data as an example of a picture (i.e., image data). A picture database service, which may be a service for storing a picture database (i.e., a multimedia database), may be configured on a blockchain node and may be started.

In step S403, a Milvus vector service may be configured on the blockchain node, and may be started, where the Milvus vector service may be a service for performing multimedia data query by means of vectorization (i.e., embedding processing, such as embedding processing of multimedia data or embedding processing of text description information).

In step S404, a picture vectorization model (e.g., a multimedia embedding network) may be configured on the blockchain node.

In step S405, a natural language vectorization model (e.g., a text embedding network) may be configured on the blockchain node.

In step S406, the working condition of the cluster (such as the cluster formed by the configured blockchain nodes) for the Milvus vector service is verified, for example, whether the Milvus vector service can operate normally in the cluster is verified, so as to successfully query the corresponding multimedia data.

Step S407, determining whether the Milvus vector service can operate normally, if so, executing step S409, and if not, executing step S408.

In step S408, an error of failure of the Milvus vector service deployment is returned to prompt the corresponding technician.

In step S409, the configuration of the block link points is set, including access IP (network address), port, certificate, block configuration, and the like of the block chain node.

In step S410, a Milvus vector service-related configuration of blockchain nodes is set, including accessing IP, ports, etc. That is, the IP and ports for subsequent access to the Milvus vector service (which may be understood as a multimedia data query service) may be configured for the block link point additionally, which may enable the query service for multimedia data to not interfere with the block link point's normal handling of other on-chain traffic.

In step S411, all blockchain nodes in the federated chain network are started, and the blockchain nodes in the subsequent federated chain network can perform the corresponding operations described in the embodiments of the present application.

Referring to fig. 10 again, fig. 10 is a flow chart of a data uplink method according to an embodiment of the present application. As shown in fig. 10, the following description will take multimedia data as an example of a picture (i.e., image data), and the method may include:

in step S501, a user may send a transaction through a client onto a blockchain node.

In step S502, the network module of the blockchain node may perform type identification on the transaction sent by the user, where it may be identified that the transaction type of the transaction is a type of uplink, e.g., the transaction is a data uplink transaction.

In step S503, the data uplink transaction has a transaction signature, the blockchain node may verify the transaction signature (i.e. verify the data uplink transaction through the transaction signature), if the verification is successful, step S505 may be executed, and if the verification is failed, step S504 may be executed.

Step S504, returning an error of failure of transaction signature verification, and ending the current process.

In step S505, the blockchain node may further verify the authority of the user, for example, verify that the user has the authority of data uplink, if the verification is passed (i.e. verify that the user has the authority of data uplink), step S507 may be performed, and if the verification is not passed (i.e. verify that the user does not have the authority of data uplink), step S506 may be performed.

Step S506, returning to the condition that the user has no authority to perform the current operation (such as the data uplink operation), and ending the current process.

In step S507, the blockchain node may add the data uplink transaction to the transaction pool, waiting to be packaged for execution.

In step S508, the block scheduling module of the blockchain master node (i.e., the out-block node in the blockchain network, the requested blockchain node may also be the out-block node) may start to out a new block, for example, a batch of transactions may be packed from the transaction pool as all transactions for the new block are ready to be executed.

In step S509, the block scheduling module may schedule the packaged transaction distribution to the virtual machine module for execution according to a certain scheduling algorithm, that is, the transaction may be executed in the virtual machine.

In step S510, the virtual machine module searches the contract repository for a contract corresponding to the transaction, that is, for a contract executing the corresponding transaction, and if so, executes step S512 described below, and if not, executes step S511 described below.

In step S511, the virtual machine module marks that the contract corresponding to the transaction does not exist, and may perform step S514 described below.

In step S512, the virtual machine module may initiate a corresponding contract process through the transaction counterpart contract to execute the current transaction, if the execution result of the transaction is correct, the following step S515 may be executed, and if the execution result of the transaction is incorrect, the following step S513 may be executed.

In step S513, the virtual machine module may identify that the current transaction has failed to execute, and may execute step S514.

In step S514, the block scheduling module may mark the write set of the transaction as empty, which is the content of the transaction that needs to modify the data in the blockchain network, that is, the data in the blockchain network does not need to be modified currently, and the current flow ends.

Step S515, determining whether the write set of the transaction includes a picture (i.e. multimedia data) to be stored, if so, executing step S516 described below, if not, indicating that the current transaction is not a transaction for uplink to the multimedia data in the present application, and packaging into a block according to a normal flow for consensus uplink.

In step S516, the block scheduling module may calculate a hash value (i.e. index information) of the picture to be stored, and may use the hash value to replace the picture to be written into a block (e.g. the target block).

In step S517, the block scheduling module may establish a mapping relationship between the hash value of the picture and the picture, and may add the mapping relationship to the buffer, that is, the buffer is cached in the buffer space.

In step S518, the block scheduling module may package the above-executed transaction (including the data uplink transaction) into a block (e.g., a target block), where the picture in the packaged data uplink transaction has been replaced with the hash value of the picture.

In step S519, the consensus module may send the packed block to all slave nodes (i.e., other consensus nodes) for consensus verification.

Step S520, determining whether the block is successfully consensus in the consensus network, if so, executing step S522 described below, and if not, executing step S521 described below.

In step S521, the currently packaged block is discarded, and an error of failed consensus is returned, and the current flow is ended.

In step S522, the blockchain node may traverse the hash values of all the pictures (i.e. the multimedia data) in the block, i.e. obtain the hash values of all the pictures in the block (e.g. the index information including the first multimedia data).

In step S523, the blockchain node may find a picture corresponding to the obtained hash value (i.e. a picture having a mapping relationship with the hash value) from the buffer space, and may store the picture in a picture database (i.e. a multimedia database) in association with the hash value of the picture.

In step S524, the blockchain node may invoke a picture vectorization model (i.e. a multimedia embedding network) to perform vectorization calculation (i.e. embedding process) on the picture, so as to obtain a picture vector (i.e. a multimedia embedding feature) of the picture.

In step S525, the blockchain node may store the picture vector in the Milvus vector database (i.e. the feature database) through the hash value association of the picture, and may delete the mapping relationship between the picture and the corresponding hash value in the cache space.

More, the application also supports direct query or query through contracts for the multimedia data by the user, that is, the application can perform multi-mode query for the multimedia data, and the related description is given by taking the multimedia data as an example.

Referring to fig. 11, fig. 11 is a flowchart of a method for directly querying a picture according to an embodiment of the present application. As shown in fig. 11, the method may include:

in step S601, a user may send a query request (e.g., a first query request or a second query request) to a blockchain node via a client based on a query interface.

In step S602, the network module of the blockchain node may identify the query request, i.e., whether the query request is a direct query or a contract query, where it may be identified as a direct query (i.e., not by a contract query, but rather directly initiate a request).

In step S603, the query request may have a signature (e.g., the object signature described above), the blockchain node may verify the signature (i.e., verify the query request with the signature), if the verification is passed, the following step S605 may be executed, and if the verification is not passed, the following step S604 may be executed.

Step S604, returns an error that the signature verification failed, and the current flow ends.

Step S605, verifying the authority of the user, that is, verifying whether the user has the query authority for the multimedia data, if the verification is passed (that is, verifying that the user has the query authority for the multimedia data), may be performed as follows S607, and if the verification is not passed (that is, verifying that the user does not have the query authority for the multimedia data), may be performed as follows S606.

In step S606, an error that the user has no authority to perform the current operation (such as the query operation of the multimedia data) is returned, and the current flow is ended.

Step S607, it is determined whether the queue (for storing the initiated query request) in the query service module is full, if so, the following step S608 is executed, and if not, the following step S609 is executed.

Step S608, the error of congestion is returned, and the current flow ends.

In step S609, the current query request is put into a queue to be sequentially executed, and load balancing is performed, for example, the query request in the queue may be distributed to a plurality of different Milvus nodes (may be a plurality of idle Milvus nodes), so that the distributed plurality of Milvus nodes query corresponding multimedia data in parallel, so as to improve the query efficiency of the multimedia data, and optionally, the plurality of Milvus nodes may include the currently requested blockchain node.

In step S610, the query service module may sequentially fetch the query requests from the queue to perform a subsequent query procedure of the multimedia data.

In step S611, the blockchain node determines whether the current timestamp minus the timestamp of the acquired query request exceeds a threshold (a time threshold may be set by itself), if so, performs step S612 described below, and if not, performs step S613 described below.

Step S612, an error of the query timeout is returned, and the current flow ends.

Step S613, determining whether the query request is a picture query (e.g. determining whether the query request is of the first query request type), if so, executing step S614 described below, and if not, executing step S615 described below.

In step S614, the block link point invokes a picture vectorization model (i.e. the multimedia embedded network) to convert the picture (e.g. the second multimedia data) in the query request into a picture vector, and performs step S618 described below.

In step S615, the blockchain node determines whether the query request is a natural language query picture (e.g., determines whether the query request is of the second query request type), if so, then the following step S617 is executed, and if not, then the following step S616 is executed.

Step S616 indicates that the query type of the current query request is not supported, and an error that is not supported by the query type may be reported, and the current flow ends.

In step S617, the block link point invokes a natural language vectorization model (i.e., a text embedding network) to convert text description information (e.g., target text description information) in the query request into a natural language vector (e.g., text embedding features).

In step S618, the blockchain node may access a stored Milvus vector database (i.e., a feature database), and may obtain a picture vector (e.g., a first matching embedded feature or a second matching embedded feature) similar to the vector (e.g., a natural language vector or a picture vector) obtained by the conversion from the Milvus vector database, and may obtain a hash value of the picture vector.

In step S619, the blockchain node may access a picture database (i.e. a multimedia database) of the storage module, and may obtain a corresponding (e.g. associated) picture according to the obtained hash value.

Step S620, the storage module returns the inquired picture to the inquiry service module as an inquiry result.

In step S621, the query service module may give the query result to the network module.

In step S622, the network module may return the query result to the user' S client.

Through the above process, the process of directly initiating the corresponding query request through the query interface to query the multimedia data is realized.

Referring to fig. 12, fig. 12 is a flowchart of a picture contract query method provided in an embodiment of the present application. As shown in fig. 12, the method may include:

in step S701, a user may send a query request (e.g., a first query request or a second query request) to a blockchain node based on a contract SDK (contract tool) through a client.

In step S702, the network module of the blockchain node may identify the query request, i.e., whether the query request is a direct query or a contract query, where it may be identified as a contract query (i.e., a query via a contract).

In step S703, the query request may have a signature (e.g., the object signature described above), the blockchain node may verify the signature (i.e., verify the query request with the signature), if the verification is passed, the following step S705 may be performed, and if the verification is not passed, the following step S704 may be performed.

Step S704, an error of failed signature verification is returned, and the current flow ends.

Step S705, verifying the authority of the user, i.e. verifying whether the user has the query authority for the multimedia data, if the verification is passed (i.e. verifying that the user has the query authority for the multimedia data), the following step S707 may be performed, and if the verification is not passed (i.e. verifying that the user does not have the query authority for the multimedia data), the following step S706 may be performed.

Step S706, returns the error that the user has no authority to perform the current operation (such as the query operation of the multimedia data), and the current flow ends.

Step S707, determining whether the queue (for storing the initiated query request) in the query service module is full, if so, executing step S708 described below, and if not, executing step S709 described below.

Step S708, the error of congestion inquiry is returned, and the current flow ends.

Step S709, the current query request is put into a queue to be sequentially executed, and load balancing is performed, for example, the query request in the queue may be distributed to a plurality of different Milvus nodes (may be a plurality of idle Milvus nodes), so that the distributed Milvus nodes query corresponding multimedia data in parallel, so as to improve the query efficiency of the multimedia data, and optionally, the plurality of Milvus nodes may include the currently requested blockchain node.

In step S710, the query service module may sequentially fetch the query requests from the queue to perform a subsequent query procedure of the multimedia data.

In step S711, the blockchain node determines whether the current timestamp minus the timestamp of the acquired query request exceeds a threshold (a time threshold may be set by itself), if so, performs step S712 described below, and if not, performs step S713 described below.

Step S712, an error of the query timeout is returned, and the current flow ends.

In step S713, the virtual machine module starts a corresponding contract process according to the contract information (which method for indicating which contract is called) in the query request to process the corresponding query request through the started contract process.

In step S714, the contract process invokes a corresponding query method (e.g., a method for querying multimedia data, which may be a query method described in the foregoing embodiments).

In step S715, the contract process may sequentially execute the contract statements to perform the query of the multimedia data.

In step S716, the contract process may launch the Milvus client such that the Milvus client packages the query request as an internal query request before it is given to the blockchain node.

In step S717, it is determined whether the internal query request initiated by the Milvus client is a picture query picture (i.e. whether multimedia data is queried through multimedia data), if so, the picture in the internal query request (such as the second multimedia data) may be queried according to the method described in the corresponding embodiment of fig. 4, and if not, the following step S718 may be executed.

Step S718, it is determined whether the internal query request initiated by the Milvus client is a natural language query picture (i.e. whether multimedia data is queried through text description information), if yes, the picture query may be performed through the text description information (e.g. target text description information) in the internal query request according to the method described in the corresponding embodiment of fig. 6, and if not, the following step S719 may be executed.

In step S719, an error that is not supported by the query type of the current query request may be reported, and the current flow ends.

In step S720, the contract process may return the query result (i.e., the queried picture) to the query service module, so that the query service module may return the query result to the network module.

In step S721, the network module may return the query result to the client of the user.

Through the above process, the process of directly initiating the corresponding query request through the contract to query the multimedia data is realized.

By the method, both intelligent contracts and external users of the blockchain can be ensured to inquire the multimedia data, meanwhile, the authenticity and the reliability of the inquired multimedia data are ensured through the blockchain network (if the inquired multimedia data can also confirm whether corresponding index information exists on the chain so as to ensure the reliability of the inquired multimedia data), and the iteration model (comprising a text embedded network and a multimedia embedded network) can be continuously updated in the practical application process. And the multi-node natural construction of the distributed database (the characteristic database and the multimedia database of distributed storage) by means of the block chain network ensures the reliability and stability of the data in the database, and can be widely reused in industry.

Referring to fig. 13, fig. 13 is a schematic structural diagram of a data processing apparatus of a blockchain network according to an embodiment of the present application. As shown in fig. 13, the data processing apparatus 1 of the blockchain network may include: a winding module 11, an acquisition module 12 and a storage module 13.

A uplink module 11, configured to perform uplink processing for the first multimedia data in the blockchain network, where the uplink processing includes consensus processing corresponding to the first multimedia data;

an obtaining module 12, configured to obtain, in a process of performing uplink processing for the first multimedia data, index information of the first multimedia data and a first multimedia embedding feature of the first multimedia data based on the consensus data if a consensus result of the consensus processing corresponding to the first multimedia data is that the consensus is successful;

a storage module 13, configured to store the first multimedia data in association with the multimedia database based on the index information of the first multimedia data, and store the first multimedia embedded feature in association with the feature database based on the index information of the first multimedia data;

Optionally, the manner in which the uplink module 11 performs uplink processing for the first multimedia data in the blockchain network includes:

data uplink transactions are processed in a blockchain network.

Optionally, the data-uplink transaction has a transaction signature; the manner in which the uplink module 11 performs uplink processing on data uplink transactions in the blockchain network includes:

Optionally, before the replacing processing is performed on the first multimedia data in the data uplink transaction by using the generated index information, the apparatus 1 is further configured to:

and caching the mapping relation into a cache space.

Optionally, if the consensus result of the consensus process corresponding to the first multimedia data is that the consensus is successful, the obtaining module 12 obtains, based on the consensus data, index information of the first multimedia data and a manner of the first multimedia embedding feature of the first multimedia data, including:

Optionally, after storing the first multimedia data in the multimedia database and storing the first multimedia embedded feature in the feature database, the apparatus 1 is further configured to:

Optionally, the acquiring module 12 acquires a procedure of the first multimedia embedding feature, including:

acquiring a multimedia embedded network;

Optionally, the above device 1 is further configured to:

Optionally, the method for obtaining, by the device 1, a multimedia embedding feature similar to the second multimedia embedding feature in the feature database as the first matching embedding feature includes:

Optionally, after returning the first query result to the client, the apparatus 1 is further configured to:

the method for generating the second multimedia embedding feature of the second multimedia data by calling the multimedia embedding network to embed the second multimedia data by the device 1 includes:

Optionally, the above device 1 is further configured to:

Optionally, after returning the second query result to the client, the apparatus 1 is further configured to:

Optionally, the foregoing apparatus 1 acquires, as the second matching embedded feature, a multimedia embedded feature similar to a text embedded feature of the target text description information in the feature database, where the method includes:

According to one embodiment of the present application, the steps involved in the data processing method of the blockchain network shown in fig. 2 may be performed by respective modules in the data processing apparatus 1 of the blockchain network shown in fig. 13. For example, step S101 shown in fig. 2 may be performed by the uplink module 11 in fig. 13, and step S102 shown in fig. 2 may be performed by the acquisition module 12 in fig. 13; step S103 shown in fig. 2 may be performed by the storage module 13 in fig. 13.

The method can execute uplink processing aiming at the first multimedia data in a blockchain network, wherein the uplink processing comprises consensus processing corresponding to the first multimedia data; in the process of executing the uplink processing aiming at the first multimedia data, if the consensus result of the consensus processing corresponding to the first multimedia data is that the consensus is successful, acquiring index information of the first multimedia data and first multimedia embedding characteristics of the first multimedia data based on the consensus data; storing the first multimedia data in association with the multimedia database based on the index information of the first multimedia data, and storing the first multimedia embedded feature in association with the feature database based on the index information of the first multimedia data; the corresponding consensus result of the consensus processing in the blockchain network is multimedia data with successful consensus, and associated inquiry based on a multimedia database and a characteristic database is supported. Therefore, the device provided by the application can store the multimedia data (such as the first multimedia data) with successful consensus as the consensus result of the corresponding consensus processing in the blockchain network into the multimedia database through the index information of the multimedia data, and store the multimedia embedded features (such as the first multimedia embedded features) of the multimedia data into the feature database through the index information of the multimedia data, and then, perform the associated query on the multimedia data in the multimedia database and the feature database through the index information of the multimedia data, so as to enrich the query mode of the multimedia data (such as the first multimedia data) which performs the related uplink processing in the blockchain network.

According to one embodiment of the present application, each module in the data processing apparatus 1 of the blockchain network shown in fig. 13 may be separately or completely combined into one or several units to form a structure, or some (some) of the units may be further split into a plurality of sub-units with smaller functions, so that the same operation may be implemented without affecting the implementation of the technical effects of the embodiments of the present application. The above modules are divided based on logic functions, and in practical applications, the functions of one module may be implemented by a plurality of units, or the functions of a plurality of modules may be implemented by one unit. In other embodiments of the present application, the data processing apparatus 1 of the blockchain network may also include other units, and in practical applications, these functions may also be implemented with assistance by other units, and may be implemented by cooperation of a plurality of units.

According to one embodiment of the present application, a computer program capable of executing the steps involved in the respective methods shown in the embodiments of the present application may be run on a general purpose computer device, which may contain a Central Processing Unit (CPU), a random access storage medium (RAM), a read only storage medium (ROM), etc., processing elements and storage elements, to construct the data processing apparatus 1 of a blockchain network as shown in fig. 13. The above-described computer program may be recorded on, for example, a computer-readable recording medium, and may be loaded into and executed in the above-described computer apparatus through the computer-readable recording medium.

Referring to fig. 14, fig. 14 is a schematic structural diagram of a computer device according to an embodiment of the present application. As shown in fig. 14, the computer device 1000 may include: processor 1001, network interface 1004, and memory 1005, and, in some embodiments, computer device 1000 may further comprise: a user interface 1003, and at least one communication bus 1002. Wherein the communication bus 1002 is used to enable connected communication between these components. The user interface 1003 may include a Display (Display), a Keyboard (Keyboard), and the optional user interface 1003 may further include a standard wired interface, a wireless interface, among others. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as at least one disk memory. The memory 1005 may also optionally be at least one storage device located remotely from the processor 1001. As shown in fig. 14, an operating system, a network communication module, a user interface module, and a device control application program may be included in the memory 1005, which is one type of computer storage medium.

In the computer device 1000 shown in fig. 14, the network interface 1004 may provide network communication functions; while user interface 1003 is primarily used as an interface for providing input to a user; and the processor 1001 may be used to invoke a device control application stored in the memory 1005 to implement:

It should be understood that the computer device 1000 described in the embodiments of the present application may perform the description of the data processing method of the blockchain network in the embodiments of the present application, and may also perform the description of the data processing apparatus 1 of the blockchain network in the embodiment corresponding to fig. 13, which is not described herein. In addition, the description of the beneficial effects of the same method is omitted.

Furthermore, it should be noted here that: the present application further provides a computer readable storage medium, and the computer readable storage medium stores a computer program, which when executed by a processor, can perform the description of the data processing method of the blockchain network in the embodiments of the present application, and therefore, a detailed description will not be given here. In addition, the description of the beneficial effects of the same method is omitted. For technical details not disclosed in the embodiments of the computer storage medium related to the present application, please refer to the description of the method embodiments of the present application.

As an example, the above-described computer program may be deployed to be executed on one computer device or on a plurality of computer devices that are located at one site, or alternatively, may be executed on a plurality of computer devices that are distributed across a plurality of sites and interconnected by a communication network, and the plurality of computer devices that are distributed across the plurality of sites and interconnected by the communication network may constitute a blockchain network.

The computer readable storage medium may be an internal storage unit of the computer device, such as a hard disk or a memory of the computer device. The computer readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash card (flash card) or the like, which are provided on the computer device. Further, the computer-readable storage medium may also include both internal storage units and external storage devices of the computer device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

The present application provides a computer program product comprising a computer program stored in a computer readable storage medium. The processor of the computer device reads the computer program from the computer readable storage medium, and the processor executes the computer program, so that the computer device performs the description of the data processing method of the blockchain network in the embodiments of the present application, and therefore, a detailed description will not be given here. In addition, the description of the beneficial effects of the same method is omitted. For technical details not disclosed in the embodiments of the computer-readable storage medium according to the present application, please refer to the description of the method embodiments of the present application.

The terms first, second and the like in the description and in the claims and drawings of the embodiments of the present application are used for distinguishing between different objects and not for describing a particular sequential order. Furthermore, the term "include" and any variations thereof is intended to cover a non-exclusive inclusion. For example, a process, method, apparatus, article, or device that comprises a list of steps or elements is not limited to the list of steps or modules but may, in the alternative, include other steps or modules not listed or inherent to such process, method, apparatus, article, or device.

Those of ordinary skill in the art will appreciate that the elements and algorithm steps described in connection with the embodiments disclosed herein may be embodied in electronic hardware, in computer software, or in a combination of the two, and that the elements and steps of the examples have been generally described in terms of function in the foregoing description to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

The foregoing disclosure is only illustrative of the preferred embodiments of the present application and is not intended to limit the scope of the claims herein, as the equivalent of the claims herein shall be construed to fall within the scope of the claims herein.

Claims

1. A method of data processing for a blockchain network, the method comprising:

performing a uplink process for first multimedia data in a blockchain network, the uplink process including a consensus process corresponding to the first multimedia data;

in the process of executing the uplink processing for the first multimedia data, if the consensus result of the consensus processing corresponding to the first multimedia data is that the consensus is successful, acquiring index information of the first multimedia data and first multimedia embedding characteristics of the first multimedia data based on the consensus data;

storing the first multimedia data in association with a multimedia database based on index information of the first multimedia data, and storing the first multimedia embedded feature in association with a feature database based on index information of the first multimedia data;

the corresponding consensus result of the consensus processing in the blockchain network is multimedia data with successful consensus, and the associated query based on the multimedia database and the characteristic database is supported.

2. The method of claim 1, wherein the performing uplink processing for the first multimedia data in the blockchain network comprises:

acquiring a data uplink transaction; the data-uplink transaction comprising the first multimedia data, the data-uplink transaction being a transaction for performing a uplink process for the first multimedia data;

and carrying out uplink processing on the data uplink transaction in the block chain network.

3. The method of claim 2, wherein the data uplink transaction has a transaction signature; the processing the data uplink transaction in the blockchain network comprises the following steps:

adopting the transaction signature to verify the data uplink transaction, and if the data uplink transaction is successfully verified, calling a virtual machine to execute the data uplink transaction;

if the data uplink transaction is successfully executed, index information of the first multimedia data in the data uplink transaction is generated;

Performing consensus processing on the target block in the block chain network, and if the consensus result of the target block is that the consensus is successful, uploading the target block to the block chain network;

4. The method of claim 3, wherein prior to replacing the first multimedia data in the data-up transaction with the generated index information, the method further comprises:

and caching the mapping relation into a cache space.

5. The method of claim 4, wherein if the consensus result of the consensus process corresponding to the first multimedia data is that the consensus is successful, obtaining the index information of the first multimedia data and the first multimedia embedding feature of the first multimedia data based on the consensus data comprises:

Acquiring the first multimedia data with a mapping relation with the extracted index information in the cache space;

and embedding the first multimedia data acquired in the cache space to generate the first multimedia embedded feature.

6. The method of claim 5, wherein after storing the first multimedia data to the multimedia database and storing the first multimedia embedded feature to the feature database, the method further comprises:

7. The method of claim 3, wherein the generating index information for the first multimedia data in the data-up transaction comprises:

performing hash calculation on the first multimedia data to generate a hash value of the first multimedia data;

8. The method of claim 1, wherein the process of obtaining the first multimedia embedding feature comprises:

acquiring a multimedia embedded network;

and calling the multimedia embedding network to embed the first multimedia data, and generating the first multimedia embedding feature.

9. The method of claim 8, wherein the method further comprises:

acquiring a first query request sent by a client; the first query request comprises second multimedia data, and the first query request is used for querying multimedia data matched with the second multimedia data;

calling the multimedia embedding network to embed the second multimedia data to generate a second multimedia embedding feature of the second multimedia data;

acquiring multimedia embedded features similar to the second multimedia embedded features from the feature database as first matching embedded features, and taking index information associated with the first matching embedded features as first matching index information;

acquiring multimedia data associated with the first matching index information from the multimedia database as a first query result of the first query request, and returning the first query result to the client;

Wherein the multimedia embedding network is used for generating similar multimedia embedding characteristics for the matched multimedia data.

10. The method of claim 9, wherein the obtaining multimedia embedding features in the feature database that are similar to the second multimedia embedding features as first matching embedding features comprises:

sequencing each multimedia embedded feature in the feature database according to the sequence from small to large of feature differences between each multimedia embedded feature in the feature database and the second multimedia embedded feature, so as to obtain the sequenced multimedia embedded feature;

taking the first L multimedia embedded features in the sequenced multimedia embedded features as the first matching embedded features; l is a positive integer.

11. The method of claim 9, wherein after returning the first query result to the client, the method further comprises:

if the first indication information is obtained and sent by the client and is used for indicating that the returned first query result is inaccurate, optimizing network parameters of the multimedia embedded network based on the first query result and the second multimedia data to obtain an optimized multimedia embedded network;

Wherein optimizing network parameters of the multimedia embedded network based on the first query result and the second multimedia data comprises: and optimizing network parameters of the multimedia embedding network to increase the feature difference between the multimedia embedding feature generated by the multimedia embedding network for the first query result and the multimedia embedding feature generated for the second multimedia data.

12. The method of claim 9, wherein the first query request is initiated in the client by a target object, the first query request containing an object signature of the target object;

the calling the multimedia embedding network to embed the second multimedia data, generating a second multimedia embedding feature of the second multimedia data, including:

performing verification processing on the first query request by adopting the object signature, and if the first query request is successfully verified, judging whether the target object has the query authority on the multimedia data in the multimedia database;

and if the target object has the query authority for the multimedia data in the multimedia database, calling the multimedia embedding network to embed the second multimedia data, and generating the second multimedia embedding feature.

13. The method of claim 1, wherein the method further comprises:

acquiring multimedia embedded features similar to text embedded features of the target text description information from the feature database, wherein the multimedia embedded features are used as second matching embedded features, and index information associated with the second matching embedded features is used as second matching index information;

14. The method of claim 13, wherein the multimedia embedding feature in the feature database is generated by invoking a multimedia embedding network, the text embedding network being trained based on the multimedia embedding feature generated by the multimedia embedding network;

The multimedia embedding network is used for generating similar multimedia embedding characteristics for matched multimedia data, the text embedding network is used for generating text embedding characteristics similar to the multimedia embedding characteristics of the multimedia data described by the text describing information for the text describing information, and the multimedia embedding characteristics of the multimedia data described by the text describing information are generated by the multimedia embedding network.

15. The method of claim 13, wherein after returning the second query result to the client, the method further comprises:

if the second indication information is obtained and sent by the client and is used for indicating that the returned second query result is inaccurate, optimizing network parameters of the text embedded network based on the second query result and the target text description information to obtain an optimized text embedded network;

wherein optimizing network parameters of the text-embedded network based on the second query result and the target text description information comprises: and optimizing network parameters of the text embedding network to increase feature differences between text embedding features generated by the text embedding network on the target text description information and multimedia embedding features of the second query result.

16. The method of claim 13, wherein the obtaining, in the feature database, multimedia embedding features similar to text embedding features of the target text description information as second matching embedding features comprises:

acquiring characteristic differences between text embedded characteristics of the target text description information and each multimedia embedded characteristic in the characteristic database;

sequencing each multimedia embedded feature in the feature database according to the sequence from small to large of feature differences between each multimedia embedded feature in the feature database and the text embedded feature of the target text description information, so as to obtain the sequenced multimedia embedded feature;

taking the first M multimedia embedded features in the ordered multimedia embedded features as the second matching embedded features; m is a positive integer.

17. A data processing apparatus of a blockchain network, the apparatus comprising:

the system comprises a block chain module, a first multimedia data processing module and a second multimedia data processing module, wherein the block chain module is used for executing a first multimedia data uplink process in a block chain network, and the first multimedia data corresponding consensus process is included in the uplink process;

the storage module is used for storing the first multimedia data in a multimedia database in an associated mode based on the index information of the first multimedia data, and storing the first multimedia embedded feature in a feature database in an associated mode based on the index information of the first multimedia data;

18. A computer program product comprising a computer program which, when executed by a processor, implements the steps of the method of any of claims 1-16.

19. A computer device comprising a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method of any of claims 1-16.

20. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program adapted to be loaded by a processor and to perform the method of any of claims 1-16.