WO2023045534A1

WO2023045534A1 - Blockchain-based infringement detection

Info

Publication number: WO2023045534A1
Application number: PCT/CN2022/107797
Authority: WO
Inventors: 潘覃; 张伟; 黄凯明; 钱烽; 张晓博
Original assignee: 蚂蚁区块链科技(上海)有限公司; 支付宝(杭州)信息技术有限公司
Priority date: 2021-09-23
Filing date: 2022-07-26
Publication date: 2023-03-30
Also published as: CN113763211A

Abstract

Embodiments of the present description provide a blockchain-based infringement detection method and apparatus and an electronic device. The method comprises: performing significance detection on an image to be detected to obtain at least one local sub-image; extracting a local feature from the at least one local sub-image, and constructing a local feature vector on the basis of the local feature; and performing infringement detection on the local feature vector and an original feature vector stored in a blockchain to determine an infringement detection result, wherein the original feature vector comprises a local feature vector constructed by a local feature extracted from a local sub-image of an original image.

Description

Blockchain-based infringement detection

technical field

The embodiments of this specification relate to the technical field of blockchain, and in particular to a blockchain-based infringement detection method, device, and electronic equipment.

Background technique

With the continuous enhancement of copyright awareness, how to more accurately detect infringement has gradually become a hot spot.

In infringement detection for images, it is usually detected based on the global features of the overall image. However, global infringement detection cannot identify local infringement images such as image stitching and picture-in-picture.

Therefore, it is necessary to provide a scheme that can identify partial image infringement.

Contents of the invention

The embodiments of this specification provide a blockchain-based infringement detection method, device, and electronic equipment.

According to the first aspect of the embodiments of this specification, there is provided a blockchain-based infringement detection method, the method comprising: performing saliency detection on an image to be detected to obtain at least one partial sub-image; from the at least one partial sub-image extracting local features, constructing local feature vectors based on the local features; performing infringement detection on the local feature vectors and the original feature vectors stored in the block chain to determine the infringement detection results; wherein, the original feature vectors include A local feature vector constructed from local features extracted from local sub-images of the original image.

According to the second aspect of the embodiment of this specification, there is provided a blockchain-based infringement detection method, the method comprising: receiving an image to be detected uploaded by a client; performing saliency detection on the image to be detected to obtain at least one partial sub-image ; Extracting local features from the at least one local sub-image, constructing a local feature vector based on the local features; performing infringement detection on the local feature vector and the original feature vector stored in the block chain; wherein, the original The feature vector includes a local feature vector constructed from local features extracted from local sub-images of the original image; when the infringement detection result is non-infringing, the image to be detected is stored in the block chain.

According to the third aspect of the embodiment of this specification, there is provided a block chain-based infringement detection device, the device includes: a salience detection unit, which performs salience detection on an image to be detected to obtain at least one partial sub-image; a feature extraction unit , extracting local features from the at least one local sub-image, and constructing a local feature vector based on the local features; the infringement detection unit performs infringement detection on the local feature vector and the original feature vector stored in the blockchain, to Determine the infringement detection result; wherein, the original feature vector includes a local feature vector constructed from local features extracted from local sub-images of the original image.

According to the fourth aspect of the embodiments of this specification, there is provided a block chain-based infringement detection device, the device includes: an image receiving unit, receiving an image to be detected uploaded by a client; property detection to obtain at least one local sub-image; a feature extraction unit extracts local features from the at least one local sub-image, and constructs a local feature vector based on the local features; an infringement detection unit combines the local feature vector with the block The original feature vector stored in the chain is used for infringement detection; wherein, the original feature vector includes a local feature vector constructed from local features extracted from local sub-images of the original image; the image storage unit, when the infringement detection result is When there is no infringement, the image to be detected is stored in the block chain.

According to a fifth aspect of the embodiments of this specification, there is provided an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein, the processor is configured to execute any of the above-mentioned blockchain-based infringement detection method.

The embodiment of this specification provides a blockchain-based infringement detection scheme. By refining the infringement detection to a local area, if the local feature vector of the image to be detected is the same as the local feature vector of the original image stored in the blockchain Similar, it means that there is an infringing area that infringes the original image in the image to be detected. In this way, partially infringing images such as picture-in-picture, image mosaic, etc. can be identified.

On the other hand, since the data stored on the blockchain is non-tamperable, the local feature vectors extracted from the local sub-images of the original image can be stored in the blockchain to build a credible local feature library; thus This makes the infringement detection result of the image to be detected based on the local feature library stored on the blockchain credible; and the uplink of the infringement detection result can avoid being tampered with, ensuring the security of the infringement detection result.

Description of drawings

FIG. 1 is a schematic diagram of a blockchain-related network environment provided by an embodiment of this specification;

FIG. 2 is a flow chart of a blockchain-based infringement detection method based on a traditional blockchain provided by an embodiment of this specification;

Fig. 3 is a flow chart of a blockchain-based infringement detection method based on a traditional blockchain provided by an embodiment of this specification;

Fig. 4 is a hardware structural diagram of a blockchain-based infringement detection device provided by an embodiment of this specification;

Fig. 5 is a block chain-based infringement detection device module provided by an embodiment of this specification;

Fig. 6 is a module of a blockchain-based infringement detection device provided by an embodiment of this specification.

Detailed ways

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with this specification. Rather, they are merely examples of apparatuses and methods consistent with aspects of the present specification as recited in the appended claims.

The terms used in this specification are for the purpose of describing particular embodiments only, and are not intended to limit the specification. As used in this specification and the appended claims, the singular forms "a", "the", and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It should also be understood that the term "and/or" as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

It should be understood that although the terms, second, third, etc. may be used in this specification to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, information may also be called second information without departing from the scope of this specification, and similarly, second information may also be called information. Depending on the context, the word "if" as used herein may be interpreted as "at" or "when" or "in response to a determination."

This manual aims to propose a blockchain-based infringement detection scheme. By refining the infringement detection to local areas, if the local feature vector of the image to be detected is similar to the local feature vector of the original image stored in the blockchain, It means that there is an infringing area that infringes the original image in the image to be detected. In this way, partially infringing images such as picture-in-picture, image mosaic, etc. can be identified.

The blockchain described in this specification may specifically include private chains, public chains, and consortium chains, etc., which are not specifically limited in this specification. Node devices in the blockchain can be added without limit, and each node device can synchronize a system time to ensure the timeliness of smart contract execution.

It should be noted that the transaction (Transaction) described in this specification refers to a piece of data that is created by the client of the blockchain and needs to be finally released to the data storage system of the blockchain.

Transactions in the blockchain usually have narrow-sense transactions and broad-sense transactions. A transaction in a narrow sense refers to a value transfer issued by a user to the blockchain; for example, in the traditional Bitcoin blockchain network, a transaction can be a transfer initiated by a user in the blockchain. In a broad sense, a transaction refers to a piece of business data with business intentions released by the user to the blockchain; for example, the operator can build a consortium chain based on actual business needs, relying on the consortium chain to deploy some other types of data that have nothing to do with value transfer online business (for example, can be broadly divided into query business, calling business, etc.), and in this type of consortium chain, a transaction can be a business message or business request with business intent published by a user in the consortium chain.

The above-mentioned client can include any type of upper-layer application that uses the underlying business data stored in the blockchain as data support to realize specific business functions.

Please refer to FIG. 1, which is a schematic diagram of a blockchain-related network environment shown in this specification.

In the network environment shown in Figure 1, it may include a client-side computing device 101, a server-side 102, and at least one blockchain system; for example, a blockchain system 103, a blockchain system 104, and a blockchain system 105.

In one embodiment, the client-side computing device 101 may include various types of client-side computing devices; for example, the client-side terminal device may include PC terminal devices, mobile terminal devices, Internet of Things devices, and Other forms of smart devices with some computing power, and so on.

In one embodiment, at least part of the computing devices in the client-side terminal device 101 may be coupled to the server-side 102 through various communication networks; for example, the device 3 shown in FIG. 1 is coupled to the server-side 102 catch.

It is not difficult to understand that some terminal devices in the client-side terminal device 101 may not be coupled with the server 102, but are directly coupled to the blockchain system as blockchain nodes through various communication networks; for example, The device 4 shown in FIG. 1 can be coupled to the blockchain system as a blockchain node.

Wherein, the above-mentioned communication network may include a wired and/or wireless communication network; for example, it may be a local area network (Local Area Network, LAN) implemented based on a wired access network or a wireless access network (such as a mobile cellular network) provided by an operator, Wide Area Network (Wide Area Network, WAN), the Internet, or a combination thereof.

In one embodiment, the client-side computing device 101 may further include one or more user-side servers; for example, the device 5 shown in FIG. 1 . At least part of the computing devices in the client-side terminal device 101 may be coupled to the user-side server, and the user-side server may be further coupled to the above-mentioned server 102; for example, the device 1 and the device shown in FIG. 1 2 is coupled to device 5, and device 5 is further coupled to server end 102.

In one embodiment, the above-mentioned user-side server may be implemented by a service entity that has established a user account system; the above-mentioned service entity may include an operating entity that provides various online and/or offline service carriers for users; wherein , the above-mentioned service carrier may include a service carrier in the form of software, and may also include a service carrier in the form of hardware.

In one embodiment, the above-mentioned service carrier may include various client softwares that provide online Internet services; for example, websites, webpages, APPs, and the like.

In one embodiment, the above-mentioned service carrier may also include various smart devices deployed offline and capable of providing offline services; for example, smart express cabinets deployed in residential areas, office areas, and public places.

Correspondingly, the above-mentioned operating entity may include the operator corresponding to the above-mentioned service carrier; for example, the above-mentioned operating entity may include individuals, organizations, companies, enterprises, etc. that operate and manage the above-mentioned service carrier.

In one embodiment, the server end 102 can also be coupled to one or more blockchain systems through various communication networks; for example, the server end 102 shown in FIG. 1 can be respectively coupled to the blockchain system 103 , block chain system 104 and block chain system 105, and so on.

In one embodiment, each blockchain system can maintain one or more blockchains (for example, public blockchains, private blockchains, consortium blockchains, etc.), and includes a or multiple blockchain nodes of multiple blockchains; for example, blockchain node 1, blockchain node 2, blockchain node 3, blockchain node 4, blockchain Node i, etc. can jointly host one or more blockchains. Cross-chain data access can also be performed between the blockchains contained in each blockchain system and between each blockchain system.

In one embodiment, blockchain nodes may include full nodes and light nodes. The full node can fully download the blockchain transactions contained in each block in the blockchain, and can perform consensus verification on the blockchain transactions contained in each blockchain according to the blockchain consensus algorithm carried .

The light node does not need to download the complete blockchain, but can only download the block header data of each block in the blockchain, and use the data contained in the block header as the verification root to verify the blockchain authenticity of the transaction. Light nodes can attach to full nodes to access more functions of the blockchain.

For example, each blockchain node in the blockchain system 103 shown in Figure 1 can be used as a full node; and the device 4 directly coupled to the blockchain system shown in Figure 1 can be used as a light node , attached to each full node in the blockchain system 103.

In one embodiment, the blockchain node can be a physical device, or a virtual device implemented in a server or a server cluster; for example, a blockchain node device can be a physical host in a server cluster, or it can be It is a virtual machine created after virtualizing the hardware resources carried by a server or server cluster based on virtualization technology. Each blockchain node can be coupled together through various types of communication methods (such as TCP/IP) to form a network to carry one or more blockchains.

In one embodiment, the server end 102 may include a BaaS platform (also referred to as a BaaS cloud) for providing Blockchain as a Service (BaaS, Blockchain as a Service). The BaaS platform can provide pre-written software for activities that occur on the blockchain (such as subscriptions and notifications, user verification, database management, and remote updates), oriented to client-side computing devices coupled with the BaaS platform, providing Easy to use, one-click deployment, fast verification, flexible and customizable blockchain services, which can accelerate the development, testing, and launch of blockchain business applications, and help the implementation of blockchain business application scenarios in various industries.

For example, in one example, the BaaS platform can provide software such as MQ (Message Queue, message queue) service; the client-side computing device coupled with the BaaS platform can subscribe to the blockchain system coupled with the BaaS platform A smart contract deployed on a certain blockchain in the blockchain generates contract events on the blockchain after the execution is triggered; while the BaaS platform can monitor the events generated by the smart contract on the blockchain after the execution is triggered, and then based on the MQ The service-related software adds the contract event to the message queue in the form of a notification message, so that the client-side computing device that subscribes to the message queue can get notifications related to the above contract event.

In one implementation, the BaaS platform can also provide enterprise-level platform services based on blockchain technology to help enterprise-level customers build a safe and stable blockchain environment and easily manage the deployment, operation, and maintenance of blockchain and development.

For example, in one example, the BaaS platform can implement rich security policies and a multi-tenant isolation environment based on cloud technology, provide advanced security protection based on chip encryption technology, and provide rapid expansion based on highly reliable data storage. End-to-end high-availability services that will not be interrupted; in another example, enhanced management capabilities can be provided to help customers build enterprise-level blockchain network environments; and local Support, support mainstream open source blockchain technologies such as Hyperledger Fabric and Enterprise Ethereum-Quorum to build an open and inclusive technology ecosystem.

After introducing the blockchain technology above, the blockchain-based infringement detection method provided in this specification is introduced below.

Referring to Fig. 2, Fig. 2 is a flow chart of a blockchain-based infringement detection method shown in an embodiment of this specification, which can be applied on the server side. The server end may be the server end 102 shown in the aforementioned FIG. 1 ; it may also be the client (such as the device 4 ) directly connected to the block chain shown in the aforementioned FIG. 1 .

Specifically, the method described in FIG. 2 may include the following steps.

Step 210: Perform saliency detection on the image to be detected to obtain at least one partial sub-image.

Wherein, the image to be detected may refer to an image work completed by the user; usually, the user may upload the original image work to an original platform for registration after completing the original image work, and the original platform may be the above-mentioned server.

During implementation, the server can detect the salient region in the image to be detected based on the saliency detection (Saliency Detection) technology, and cut out the salient region to obtain N local sub-images.

The saliency detection technology here can use common algorithms in the industry, such as machine learning models such as detection network and Mask RCNN network. These machine learning models usually require model training in advance, through a large number of sample images with labels, in which each region with salient features and the label information represented by the region are marked. For example, for a face image sample, it is possible to mark the facial features region of the face and the name label of the facial features represented by each region.

Usually, a large number of sample images can be used to train the aforementioned machine learning model, and various parameters in the model can be optimized through continuous calculation, so that the recognition accuracy of the model is getting higher and higher. When the model training meets the preset requirements (for example, the accuracy rate exceeds the threshold, and the number of iterations exceeds the preset number), the trained model can be used. At this point, the image to be detected is input into the model for calculation, and local sub-images with salient features can be output.

Step 220: Extract local features from the at least one local sub-image, and construct a local feature vector based on the local features.

After obtaining N local sub-images, the local feature vectors of N local sub-images can also be obtained from the local feature vectors extracted from each local sub-image.

Here feature extraction can use including but not limited to deep feature extraction network, such as VGG model, ResNet, MobileNet; or feature extraction methods such as SIFT, SURF, ORB.

Similar to the model training method adopted by the aforementioned local sub-images, the model used for feature extraction also needs to be trained in advance. No more details will be given at this time.

Step 230: Perform infringement detection on the local feature vector and the original feature vector stored in the blockchain to determine the infringement detection result; wherein, the original feature vector includes local features extracted from local sub-images of the original image Constructed local feature vectors.

There are original certified original images in the blockchain, as well as local feature vectors constructed from local features extracted from local sub-images of original images; these local feature vectors of original images can be used as original feature vectors for infringement detection Provide a credible local feature library.

Wherein, the local sub-images of the original image and the local feature vectors in the local sub-images are all obtained in the same manner as the image to be detected.

In practical applications, whether it is a public chain, a private chain or an alliance chain, it is possible to provide the function of a smart contract (Smart contract). Smart contracts on the blockchain are contracts that can be triggered by transactions on the blockchain. Smart contracts can be defined in the form of code.

Smart contracts can be independently executed on each node in the blockchain network in a prescribed manner, and all execution records and data are stored on the blockchain, so when such a transaction is executed, the blockchain will save Tampering, non-lost transaction credentials.

When implementing, the business logic of the smart contract can be published to the blockchain in the form of code, so that the blockchain can create a corresponding smart contract, and the smart contract can access the code after being called to realize the execution of the business logic.

In this specification, however, smart contracts containing codes for infringement detection logic can be published to the blockchain.

In one implementation, the server can call the infringement detection logic declared in the smart contract published in the blockchain, and perform infringement detection on the local feature vector and the original feature vector stored in the blockchain.

In this way, the server can act as a node of the blockchain and directly call the smart contract locally for infringement detection.

In another implementation, the server can publish the local feature vector as a blockchain transaction to the blockchain; so that the accounting node in the blockchain responds to the transaction and calls the The infringement detection logic declared in the smart contract performs infringement detection based on the local feature vector and the original feature vector stored in the blockchain.

In this way, the server can initiate a transaction for infringement detection, so that the accounting nodes in the blockchain call the smart contract for infringement detection.

In an embodiment, before the step 230, the method may further include: performing dimensionality reduction processing on the local feature vector; however, based on the reduced dimensionality of the local feature vector and the blockchain Infringement detection is performed on the original feature vector stored in the certificate; wherein, the original feature vector can also be an original feature vector after dimensionality reduction.

Wherein, the dimensionality reduction processing can adopt PCA (Principal Component Analysis, principal component analysis) algorithm, SVD (Singular Value Decomposition, singular value decomposition) and other dimensionality reduction algorithms.

In this embodiment, the data volume of local feature vectors can be reduced through dimensionality reduction, thereby reducing the amount of calculations consumed in infringement detection calculations, and thus the detection efficiency will be improved due to the reduced calculations.

In the foregoing step 230, the infringement detection is performed on the local feature vector and the original feature vector stored in the blockchain through steps A1 to A2.

Step A1: Calculate the similarity between the local feature vector and the original feature vector stored in the blockchain; when implementing, compare the N local feature vectors with the original feature vector, so that each local feature The vector can recall M original eigenvectors (M represents the number of original eigenvectors); then, screen the N*M eigenvector groups (1 local eigenvector and 1 original eigenvector) to determine that there are similar The original feature vector of .

In one implementation, feature aggregation can be performed on original feature vectors of the same original image to obtain an original feature aggregation vector; feature aggregation can be performed on local feature vectors to obtain a local feature aggregation vector; the local feature aggregation vector and each original feature aggregation vector can be calculated. Similarity of feature aggregation vectors; determine original feature aggregation vectors whose similarity is greater than a threshold.

In another implementation manner, original feature vectors similar to local feature vectors can be screened out by means of similar feature number threshold screening, score confidence interval screening, and the like.

Wherein, the threshold screening of the number of similar features may refer to calculating the number of similar features in the local feature vector and the original feature vector, and when the similar number exceeds a certain threshold, it can be determined that the original feature vector is similar to the local feature vector.

Step A2: When there is no similarity between the local feature vector and the original feature vector greater than the threshold, determine that the infringement detection result is non-infringing; publish the relevant information of the local sub-image to the blockchain for deposit.

If there is no original feature vector similar to the local feature vector, it means that the image to be detected is not similar to the original image stored in the blockchain, so it can be concluded that the image to be detected is an original image, and there is no infringement registered original image.

After it is determined that the image to be detected does not infringe, the relevant information of the image to be detected and partial sub-images can be stored in the blockchain.

Wherein, the relevant information of the local sub-image includes: the local feature vector (as new original feature information), the corresponding relationship between the local feature vector and the image to be detected, and the local sub-image corresponds to the image to be detected location information.

By storing the local feature vector of the image to be detected belonging to the original image to the blockchain, the content of the local feature library stored in the blockchain is improved, so as to provide credible local feature information of the original image for subsequent infringement detection.

Step A3: When the similarity between the local feature vector and the original feature vector is greater than the threshold, determine that the infringement detection result is infringement.

When the infringement detection result is an infringement, the infringement information is stored in the blockchain; wherein the infringement information includes: the image to be detected and the infringement area in the original image; wherein the infringement area includes similarity Local feature vectors greater than a threshold correspond to local subimages in the image to be detected, and original feature vectors whose similarity is greater than a threshold correspond to local subimages in the original image.

By storing the infringement information in the blockchain, the purpose of solidifying the certificate can be achieved. When an infringement dispute occurs, the infringement information stored in the blockchain can be used as the evidence of the original party's rights protection, thereby improving the success rate of rights protection.

By refining the infringement detection to the local area, if the local feature vector of the image to be detected is similar to the local feature vector of the original image stored in the blockchain, it means that there is an infringement area in the image to be detected that infringes the original image. In this way, partially infringing images such as picture-in-picture, image mosaic, etc. can be identified.

Referring to Fig. 3 again below, Fig. 3 is a flow chart of a blockchain-based infringement detection method shown in an embodiment of this specification, which can be applied to a server corresponding to a client; wherein, the client includes A decentralized client (such as the aforementioned device 3 shown in FIG. 1 ), and the server includes a blockchain-as-a-service platform (such as the aforementioned server 102 shown in FIG. 1 ). The method may include the following steps: Step 310: Receive the image to be detected uploaded by the client.

The image to be detected may refer to an image work completed by the user; usually, the user may upload the original image work to an original platform for registration after completing the original image work, and the original platform may be the above-mentioned server.

Step 320: Perform saliency detection on the image to be detected to obtain at least one partial sub-image; this step is the same as step 210 described in the embodiment of FIG.

Step 330: extracting local features from the at least one local sub-image, and constructing a local feature vector based on the local features; this step is the same as step 220 described in the embodiment of FIG. No more details are given here.

Step 340: performing infringement detection on the local feature vector and the original feature vector stored in the blockchain; wherein, the original feature vector includes a local feature vector constructed from local features extracted from local sub-images of the original image; This step is similar to the step 230 described in the aforementioned embodiment of FIG. 2 , and reference may be made to the content recorded in the aforementioned step 230 , which will not be repeated here.

Step 350: When the infringement detection result is non-infringement, store the image to be detected in the block chain.

When the infringement detection result is non-infringement, it means that the image to be detected is an original image, so the image to be detected can be stored in the blockchain as an original image.

This embodiment provides a block chain-based infringement detection scheme, by refining the infringement detection to a local area, using the local feature vector of the original image stored in the block chain to detect the local salient area in the image Perform local infringement detection. In this way, partially infringing images such as picture-in-picture, image mosaic, etc. can be identified.

Since the image to be detected does not partially infringe the original work, the image to be detected can be stored in the blockchain as an original work, and the original information of the original work can be recorded by using the non-tamperable characteristics of the blockchain (for example, the time on the chain can be regarded as The original time of the work); the original works stored in the blockchain can guarantee the original rights and interests. For example, the original information recorded can be used as evidence for rights protection.

An embodiment of the present specification below also provides a blockchain-based infringement detection method based on traditional blockchains. This method is written from the blockchain side, with the nodes in the blockchain as the execution subject. The method may include steps as described below.

Step B1: Receive a call transaction for infringement detection of the image to be detected; wherein, the call transaction includes a local feature vector constructed from local features extracted from a local subimage of the image to be detected, and the partial subimage includes An image region obtained by performing saliency detection on the image to be detected.

The saliency detection, local feature extraction, etc. in this step have been introduced in the foregoing embodiments, and will not be repeated here.

It is worth mentioning that in some embodiments, the call transaction may only include the image to be detected, and then the saliency detection, local feature extraction, and feature vector construction can be executed by calling the declaration infringement detection logic in the smart contract.

Step B2: In response to the call transaction, invoke the infringement detection logic declared in the smart contract published in the blockchain, and perform infringement detection based on the local feature vector and the original feature vector stored in the blockchain; where , the original feature vector includes a local feature vector extracted from a local sub-image of the original image.

For infringement detection in this step, reference may be made to the examples in the foregoing embodiments, and details are not repeated here.

Step B3: Store the infringement detection result in the blockchain.

In view of the presence of original feature vectors similar to the local feature vectors, the infringement detection result includes: the image to be detected and the infringement area in the original image; wherein the infringement area includes similar local feature vectors and original The feature vectors correspond to local sub-images in the image to be detected and the original image.

Corresponding to the aforementioned embodiment of the blockchain-based infringement detection method, this specification also provides an embodiment of a blockchain-based infringement detection device. The device embodiments can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by reading the corresponding computer business program instructions in the non-volatile memory into the memory for operation through the processor of the device where it is located. From the hardware level, as shown in Figure 4, it is a hardware structure diagram of the device where the blockchain-based infringement detection device in this manual is located, except for the processor, network interface, memory and non-volatile In addition to the memory, the device where the device in the embodiment is usually based on the actual function of the blockchain-based infringement detection may also include other hardware, which will not be repeated here.

Please refer to Figure 5, which is a block diagram of a block chain-based infringement detection device provided by an embodiment of this specification. The device corresponds to the embodiment shown in Figure 2. The device includes: a significant detection unit 510, to be detected The image is subjected to saliency detection to obtain at least one local sub-image; the feature extraction unit 520 extracts local features from the at least one local sub-image, and constructs a local feature vector based on the local features; the infringement detection unit 530 extracts the local The feature vector and the original feature vector stored in the block chain are used for infringement detection to determine the infringement detection result; wherein, the original feature vector includes a local feature vector constructed from local features extracted from local sub-images of the original image.

Optionally, in the infringement detection unit 530, the infringement detection of the local feature vector and the original feature vector stored in the block chain includes: calling the infringement statement declared in the smart contract published in the block chain The detection logic performs infringement detection on the local feature vector and the original feature vector stored in the blockchain.

Optionally, in the infringement detection unit 530, performing infringement detection on the local feature vector and the original feature vector stored in the blockchain, including: a calculation subunit, combining the local feature vector with the original feature vector in the blockchain Calculate the similarity of the stored original feature vector; determine the sub-unit, and when the similarity between the local feature vector and the original feature vector is greater than the threshold, determine the infringement detection result as infringement.

Optionally, the calculation subunit includes: performing feature aggregation on original feature vectors of the same original image to obtain an original feature aggregation vector; performing feature aggregation on local feature vectors to obtain a local feature aggregation vector; calculating the local feature aggregation The similarity between the vector and each original feature aggregation vector; determine the original feature aggregation vector whose similarity is greater than a threshold.

Optionally, the device further includes: a certificate storage subunit, which stores the infringement information in the block chain when the infringement detection result is infringement; wherein, the infringement information includes: the image to be detected and the original An infringing area in an image; wherein, the infringing area includes a local feature vector whose similarity is greater than a threshold corresponding to a local sub-image in the image to be detected, and an original feature vector whose similarity is greater than a threshold corresponds to a sub-image in the original image local subimage.

Optionally, the device further includes: a certificate storage subunit, when there is no similarity between the local feature vector and the original feature vector greater than the threshold, determine that the infringement detection result is not infringement; publish the relevant information of the partial sub-image To the blockchain for depositing certificates.

Optionally, the relevant information of the local sub-image includes: the local feature vector, the corresponding relationship between the local feature vector and the image to be detected, and the position information of the local sub-image corresponding to the image to be detected.

Please refer to FIG. 6 , which is a block diagram of a block chain-based infringement detection device provided by an embodiment of this specification. The device corresponds to the embodiment shown in FIG. 3 , and the device includes: an image receiving unit 610, a receiving client The uploaded image to be detected; the saliency detection unit 620, which performs saliency detection on the image to be detected to obtain at least one local sub-image; the feature extraction unit 630, extracts local features from the at least one local sub-image, based on the local features Construct a local feature vector; the infringement detection unit 640 performs infringement detection on the local feature vector and the original feature vector stored in the block chain; wherein, the original feature vector includes a partial sub-image extracted from the original image The local feature vector constructed by the feature; the image certificate storage unit 650, when the infringement detection result is non-infringement, deposit the image to be detected to the block chain.

Optionally, the image storage unit 650 includes: when the infringement detection result is non-infringement, the local feature vector of the image to be detected, the corresponding relationship between the local feature vector and the image to be detected , The local sub-image corresponding to the location information in the image to be detected is stored in the block chain as original information.

Optionally, the device is applied to a server corresponding to the client; wherein, the client includes a decentralized client, and the server includes a blockchain-as-a-service platform.

A block diagram of a block chain-based infringement detection device provided by an embodiment of this specification, the device includes: a receiving unit, receiving a calling transaction for performing infringement detection on an image to be detected; wherein, the calling transaction includes the pending Detect the image; the determining unit, in response to the calling transaction, invokes the infringement detection logic declared in the smart contract published in the block chain, and determines the partial sub-image in the image to be detected; wherein, the partial sub-image includes the to-be-determined Detecting an image region with distinctive features in an image; an extraction unit, extracting a local feature vector from the partial sub-image; a detection unit, based on the local feature vector and the original feature vector stored in the block chain Infringement detection; wherein, the original feature vector includes a local feature vector extracted from a local sub-image of the original image; a proof storage unit stores the result of the infringement detection into the block chain.

The systems, devices, modules, or units described in the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementing device is a computer, which may take the form of a personal computer, laptop computer, cellular phone, camera phone, smart phone, personal digital assistant, media player, navigation device, e-mail device, game control device, etc. desktops, tablets, wearables, or any combination of these.

For the implementation process of the functions and effects of each unit in the above device, please refer to the implementation process of the corresponding steps in the above method for details, and will not be repeated here.

As for the device embodiment, since it basically corresponds to the method embodiment, for related parts, please refer to the part description of the method embodiment. The device embodiments described above are only illustrative, and the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in One place, or it can be distributed to multiple network elements. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution in this specification. It can be understood and implemented by those skilled in the art without creative effort.

This specification also provides an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein, the processor is configured to execute any one of the above blockchain-based infringement detection methods.

In the embodiment of the above-mentioned electronic device, it should be understood that the processor may be a central processing unit (English: Central Processing Unit, referred to as: CPU), and may also be other general-purpose processors, digital signal processors (English: Digital Signal Processor , referred to as: DSP), application specific integrated circuit (English: Application Specific Integrated Circuit, referred to as: ASIC) and so on. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc., and the aforementioned memory can be a read-only memory (English: read-only memory, abbreviated: ROM), random access memory (English: : random access memory, referred to as: RAM), flash memory, hard disk or solid state disk. The steps of the methods disclosed in connection with the embodiments of the present invention may be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules in the processor.

Each embodiment in this specification is described in a progressive manner, the same and similar parts of each embodiment can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the electronic device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and for relevant parts, please refer to part of the description of the method embodiment.

Other embodiments of the specification will readily occur to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This description is intended to cover any modification, use or adaptation of this description, and these modifications, uses or adaptations follow the general principles of this description and include common knowledge or conventional technical means in the technical field not disclosed in this description . The specification and examples are to be considered exemplary only, with a true scope and spirit of the specification being indicated by the following claims.

It should be understood that this specification is not limited to the precise constructions which have been described above and shown in the accompanying drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the specification is limited only by the appended claims.

Claims

A blockchain-based infringement detection method, the method comprising:

performing saliency detection on the image to be detected to obtain at least one local sub-image;

extracting local features from the at least one local sub-image, and constructing a local feature vector based on the local features;

Perform infringement detection on the local feature vector and the original feature vector stored in the block chain to determine the infringement detection result; wherein the original feature vector includes a local feature constructed from local sub-images of the original image. Feature vector.
According to the method according to claim 1, the infringement detection of the local feature vector and the original feature vector stored in the block chain includes:

Invoke the infringement detection logic declared in the smart contract published in the blockchain, and perform infringement detection on the local feature vector and the original feature vector stored in the blockchain.
According to the method according to claim 1 or 2, said performing infringement detection on said local feature vector and the original feature vector stored in the block chain, comprising:

Computing the similarity between the local feature vector and the original feature vector stored in the blockchain;

When the similarity between the local feature vector and the original feature vector is greater than a threshold, it is determined that the infringement detection result is infringement.
The method of claim 3, further comprising:

Perform feature aggregation on the original feature vectors of the same original image to obtain the original feature aggregation vector;

Perform feature aggregation on local feature vectors to obtain local feature aggregation vectors;

Calculating the similarity between the local feature aggregation vector and each original feature aggregation vector;

Determine the original feature aggregation vectors whose similarity is greater than a threshold.
The method of claim 3, further comprising:

When the infringement detection result is an infringement, the infringement information is stored in the block chain; wherein, the infringement information includes:

The image to be detected and the infringing area in the original image; wherein, the infringing area includes a local feature vector whose similarity is greater than a threshold corresponding to a local sub-image in the image to be detected, and an original feature whose similarity is greater than a threshold The vectors correspond to local sub-images in the original image.
The method of claim 3, further comprising:

When there is no similarity between the local feature vector and the original feature vector greater than the threshold, it is determined that the infringement detection result is non-infringement;

Publish the relevant information of the partial sub-image to the block chain for certificate deposit.
According to the method according to claim 6, the relevant information of the partial sub-image comprises:

The local feature vector, the corresponding relationship between the local feature vector and the image to be detected, and the local sub-image corresponds to position information in the image to be detected.
A blockchain-based infringement detection method, the method comprising:

Receive the image to be detected uploaded by the client;

performing saliency detection on the image to be detected to obtain at least one local sub-image;

extracting local features from the at least one local sub-image, and constructing a local feature vector based on the local features;

performing infringement detection on the local feature vector and the original feature vector stored in the block chain; wherein, the original feature vector includes a local feature vector constructed from local features extracted from local sub-images of the original image;

When the infringement detection result is non-infringement, the image to be detected is stored in the block chain.
According to the method according to claim 8, when the infringement detection result is non-infringement, storing the image to be detected in the block chain includes:

When the infringement detection result is non-infringement, corresponding the local feature vector of the image to be detected, the corresponding relationship between the local feature vector and the image to be detected, and the local sub-image in the image to be detected The location information is stored in the blockchain as original information.
The method according to claim 8, which is applied on a server corresponding to the client; wherein the client includes a decentralized client, and the server includes a blockchain-as-a-service platform.
A blockchain-based infringement detection device, said device comprising:

A saliency detection unit is configured to perform saliency detection on the image to be detected to obtain at least one local sub-image;

a feature extraction unit, extracting local features from the at least one local sub-image, and constructing a local feature vector based on the local features;

The infringement detection unit performs infringement detection on the local feature vector and the original feature vector stored in the block chain to determine the infringement detection result; wherein, the original feature vector includes the partial sub-image extracted from the original image Local feature vectors for feature construction.
A blockchain-based infringement detection device, said device comprising:

The image receiving unit receives the image to be detected uploaded by the client;

A saliency detection unit is configured to perform saliency detection on the image to be detected to obtain at least one local sub-image;

a feature extraction unit, extracting local features from the at least one local sub-image, and constructing a local feature vector based on the local features;

The infringement detection unit performs infringement detection on the local feature vector and the original feature vector stored in the block chain; wherein, the original feature vector includes a local feature vector constructed from local features extracted from local sub-images of the original image ;

The image certificate storage unit stores the image to be detected in the block chain when the infringement detection result is non-infringement.
An electronic device comprising:

processor;

memory for storing processor-executable instructions;

Wherein, the processor is configured to execute the method described in any one of claims 1-10 above.