CN113568571B - Image de-duplication method based on residual neural network - Google Patents
Image de-duplication method based on residual neural network Download PDFInfo
- Publication number
- CN113568571B CN113568571B CN202110722552.3A CN202110722552A CN113568571B CN 113568571 B CN113568571 B CN 113568571B CN 202110722552 A CN202110722552 A CN 202110722552A CN 113568571 B CN113568571 B CN 113568571B
- Authority
- CN
- China
- Prior art keywords
- image
- processed
- client
- neural network
- database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 45
- 238000000034 method Methods 0.000 title claims abstract description 44
- 230000008447 perception Effects 0.000 claims abstract description 20
- 238000001514 detection method Methods 0.000 claims abstract description 13
- 238000012795 verification Methods 0.000 claims description 43
- 230000008569 process Effects 0.000 claims description 8
- 238000007781 pre-processing Methods 0.000 claims description 7
- 238000004891 communication Methods 0.000 claims description 4
- 230000007935 neutral effect Effects 0.000 claims description 3
- 238000013519 translation Methods 0.000 claims description 3
- 238000012545 processing Methods 0.000 claims description 2
- 230000000007 visual effect Effects 0.000 abstract description 5
- 230000000694 effects Effects 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 5
- QWXOJIDBSHLIFI-UHFFFAOYSA-N [3-(1-chloro-3'-methoxyspiro[adamantane-4,4'-dioxetane]-3'-yl)phenyl] dihydrogen phosphate Chemical compound O1OC2(C3CC4CC2CC(Cl)(C4)C3)C1(OC)C1=CC=CC(OP(O)(O)=O)=C1 QWXOJIDBSHLIFI-UHFFFAOYSA-N 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 238000002203 pretreatment Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Abstract
The invention discloses an image deduplication method based on a residual neural network, which is applied to a server and comprises the following steps: receiving an uploading request; according to the first accurate matching data and the first perception hash characteristic value, similarity detection is carried out on the image to be processed in the database; when a first image matched with the image to be processed is detected in the database, comparing the image quality scores of the image to be processed and the first image; and if the image quality score of the image to be processed is higher than that of the first image, sending an uploading instruction to the client and receiving the image to be processed uploaded by the client. The invention can judge the images with the same visual information as similar images by comparing the first accurate matching data with the first perception hash characteristic value, and has good duplicate removal precision. In addition, by comparing the image quality of the image to be processed and the image quality of the first image and deleting the redundant image with poor quality, a good duplicate removal effect can be obtained and user experience can be improved.
Description
Technical Field
The invention belongs to the field of image processing, and particularly relates to an image deduplication method based on a residual neural network.
Background
The images contain rich and visual information, and a large number of images are required to transmit information to users in the fields of social, shopping, travel and the like. As the number of images increases, the local storage overhead increases, so that more and more users upload images from the local client to the cloud server for storage.
In order to avoid unnecessary increase of storage capacity of a cloud server caused by re-uploading similar or same images, how to realize safe deduplication of images from a large amount of data is a research hotspot in the field. Currently, a Client-based multimedia data deduplication method (a Client-based Security Provable Deduplication of Multimedia Data, CSPD) is used in the related art to implement image deduplication, and the method includes three stages of similar image detection, ownership authentication and quality comparison. However, since the related art performs similarity discrimination from two angles of cryptographic hash and discrete cosine transform, only an original image and an image thereof subjected to one distortion change can be identified, and when the original image is subjected to more than two kinds of distortion superposition, the identification accuracy of the similar image is reduced, thereby reducing the image de-duplication accuracy.
Disclosure of Invention
In order to solve the problems in the prior art, the invention provides an image de-duplication method based on a residual neural network. The technical problems to be solved by the invention are realized by the following technical scheme:
In a first aspect, the present invention provides an image deduplication method based on a residual neural network, applied to a server, including:
receiving an uploading request from a client; the method comprises the steps that communication connection is pre-established between the client and the server, and the uploading request comprises first accurate matching data of an image to be processed and a first perception hash characteristic value;
According to the first accurate matching data and the first perception hash characteristic value, similarity detection is carried out on the image to be processed in a database; the database comprises a plurality of groups of pre-stored images, accurate matching data corresponding to the images and perceived hash characteristic values;
Comparing an image quality score of a to-be-processed image with a first image matched with the to-be-processed image when the first image is detected in the database; and if the image quality score of the image to be processed is higher than the image quality score of the first image, sending an uploading instruction to the client and receiving the image to be processed uploaded by the client.
In one embodiment of the present invention, the step of performing similarity detection on the image to be processed in the database according to the first exact match data and the first perceptual hash feature value includes:
Detecting whether second exact match data equal to the first exact match data exists in the exact match data stored in the database;
If so, determining the image corresponding to the second accurate matching data as a first image; otherwise, calculating a first average absolute error distance between the first perceptual hash feature value and each perceptual hash feature value stored in the database;
and comparing the first average absolute error distance with a first preset threshold value, and determining the image corresponding to the second hash characteristic value as a first image after determining the perceived hash characteristic value of which the first average absolute error distance is smaller than the preset threshold value as the second hash characteristic value.
In one embodiment of the present invention, when a first image matching an image to be processed is detected in the database, before the step of comparing the image quality of the image to be processed with that of the first image, the method further comprises:
and carrying out ownership verification on the client.
In one embodiment of the present invention, the step of performing ownership verification on the client includes:
After dividing a plurality of image blocks in the first image, selecting the first image block from the plurality of image blocks, and preprocessing the first image block;
Scaling the preprocessed first image block and the first image to the same size, and mixing to obtain a first target image;
inputting the first target image to a first residual neural network to obtain a perceived hash characteristic value of the first target image;
generating an ownership verification request, wherein the ownership verification request comprises network parameters of the first residual neural network;
Sending the ownership verification request to a client, so that the client divides a plurality of image blocks in an image to be processed, then selects a second image block from the image blocks, pre-processes the second image block, scales the pre-processed second image block and the image to be processed to the same size, mixes the pre-processed second image block and the image to be processed to obtain a second target image, sets network parameters of a second residual neural network according to network parameters of the first residual neural network, inputs the second target image to the second residual neural network, and sends the obtained perceived hash characteristic value of the second target image to the server;
and judging whether the client passes ownership verification according to the perceived hash characteristic value of the first target image and the perceived hash characteristic value of the second target image.
In one embodiment of the present invention, the step of determining whether the client passes ownership verification according to the perceptual hash feature value of the first target image and the perceptual hash feature value of the second target image includes:
calculating a second average absolute error distance between the perceived hash feature value of the first target image and the perceived hash feature value of the second target image;
comparing the second average absolute error distance with a second preset threshold; if the second average absolute error distance is smaller than a second preset threshold, the client passes ownership verification, otherwise, the client fails ownership verification;
And sending the verification result to the client.
In one embodiment of the invention, the pre-treatment comprises a rotation and/or translation operation.
In one embodiment of the present invention, when a first image matching an image to be processed is detected in the database, the step of comparing the image quality score of the image to be processed with the image quality score of the first image further includes:
and if the image quality score of the image to be processed is lower than that of the first image, transmitting a link of the first image to the client.
In one embodiment of the present invention, when there is no first image matching the image to be processed in the database, after the step of performing similarity detection on the image to be processed in the database according to the first exact matching data and the first perceptual hash feature value, the method further includes:
And after the uploading instruction is sent to the client, receiving the image to be processed uploaded by the client.
In one embodiment of the present invention, after the step of receiving the image to be processed uploaded by the client, the method further includes:
And storing the image to be processed, the first accurate matching data and the first perceptual hash characteristic value into a database.
In a second aspect, the present invention further provides an image deduplication method based on a residual neural network, applied to a client, including:
acquiring an image to be processed;
determining first exact match data of the image to be processed according to a Sha256 algorithm;
determining a first perception hash characteristic value of the image to be processed by using a pre-trained second residual neutral network model;
Sending an uploading request of the image to be processed to a server, so that the server carries out similarity detection on the image to be processed in a database according to the first accurate matching data and the first perception hash characteristic value, and compares the image quality scores of the image to be processed and the first image when the first image matched with the image to be processed is detected;
If the image quality score of the image to be processed is higher than the image quality score of the first image, receiving an uploading instruction and uploading the image to be processed to the server; otherwise, a link to the first image is received.
Compared with the prior art, the invention has the beneficial effects that:
In the image deduplication method based on the residual neural network, the server receives the uploading request from the client, and because the uploading request comprises the first accurate matching data of the image to be processed and the first perception hash characteristic value, the image can be judged to be similar by comparing the first accurate matching data with the accurate matching data of the image stored in advance in the database, and the image can be judged to be similar by comparing the first perception hash characteristic value with the perception hash characteristic value of the image stored in advance in the database, so that the image with the same visual information can be judged to be similar, and the image deduplication method has good deduplication precision. In addition, when the first image which is the same as or similar to the image to be uploaded exists in the database, the image quality of the first image and the image quality of the second image are compared, and redundant images with poor quality are deleted, so that the user experience is improved while a good duplicate removal effect is obtained.
The present invention will be described in further detail with reference to the accompanying drawings and examples.
Drawings
Fig. 1 is a schematic flow chart of an image deduplication method based on a residual neural network according to an embodiment of the present invention;
Fig. 2 is a schematic diagram of an image deduplication method based on a residual neural network according to an embodiment of the present invention;
FIG. 3 is a schematic diagram illustrating ownership verification performed by a server according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of interaction between a client and a server in the ownership verification process according to an embodiment of the present invention;
fig. 5 is another flow chart of an image deduplication method based on a residual neural network according to an embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to specific examples, but embodiments of the present invention are not limited thereto.
As shown in fig. 1, the image deduplication method based on the residual neural network provided by the embodiment of the invention is applied to a server and comprises the following steps:
S101, receiving an uploading request from a client; the method comprises the steps that communication connection is pre-established between a client and a server, and an uploading request comprises first accurate matching data of an image to be processed and a first perception hash characteristic value;
S102, performing similarity detection on an image to be processed in a database according to first accurate matching data and a first perception hash characteristic value; the database comprises a plurality of groups of pre-stored images, accurate matching data corresponding to the images and perceived hash characteristic values;
s103, when a first image matched with the image to be processed is detected in the database, comparing the image quality scores of the image to be processed and the first image; and if the image quality score of the image to be processed is higher than that of the first image, sending an uploading instruction to the client and receiving the image to be processed uploaded by the client.
In this embodiment, the server and the client establish communication connection in advance, and the database of the server includes a plurality of groups of pre-stored images, accurate matching data corresponding to the images, and perceptual hash feature values, where the pre-stored images may be uploaded to the server through the client before. Optionally, the upload request sent by the client to the server includes a feature sequence of the image to be processed, where the feature sequence includes the first exact match data and the first perceptual hash feature value. After receiving the uploading request, the server searches in the database according to the first accurate matching data and the first perceived hash feature, and detects whether an image similar to the image to be processed exists. It should be noted that, in this embodiment, the image similar to the image to be processed includes not only the image which is completely the same as the image to be processed in the encoding mode, but also the similar image which has the same semantic information after the image to be processed is subjected to operations such as rotation, filtering, compression, etc.
Optionally, the client determines first exact match data of the image to be processed by using a Sha256 algorithm, and determines a first perceptual hash feature value of the image to be processed by using a pre-trained second residual neural network model. In the embodiment, the residual neural network and the hash technology are introduced into the similarity phase discrimination of the images, so that the images with the same visual information can be judged to be similar, and the robustness and the distinguishing property of image de-duplication are improved.
Of course, in some other embodiments of the present invention, other algorithms may be used to calculate the first exact match data and the first perceptual hash feature value of the image to be processed, which is not limited by the present invention.
In the above step S103, when there is a first image similar to the image to be processed in the database, the image quality scores of the two images are further compared, wherein a higher image quality score indicates a better image quality. When the image quality of the image to be processed is better than that of the first image, the image to be processed which is requested to be uploaded by the client is better, so that the server sends an uploading instruction to the client and receives the image to be processed which is uploaded by the client.
In addition, in the above-described step S103, if the image quality score of the image to be processed is lower than that of the first image, the link of the first image is transmitted to the client so that the user uses the first image according to the link download.
Of course, in this embodiment, if the server does not retrieve the first image, it indicates that the same or similar image as the image to be processed is not uploaded, so that an upload instruction may be sent to the client and the image to be processed uploaded by the client may be received.
Optionally, in the step S102, the step of performing similarity detection on the image to be processed in the database according to the first exact match data and the first perceptual hash feature value includes:
detecting whether second exact match data equal to the first exact match data exists in the exact match data stored in the database;
If so, determining an image corresponding to the second accurate matching data as a first image; otherwise, calculating a first average absolute error distance between the first perceptual hash feature value and each perceptual hash feature value stored in the database;
And comparing the first average absolute error distance with a first preset threshold value, and determining an image corresponding to the second hash characteristic value as a first image after determining the perceived hash characteristic value of which the first average absolute error distance is smaller than the preset threshold value as the second hash characteristic value.
In this embodiment, after receiving an upload request, the server first detects whether there is second exact match data equal to the first exact match data in the exact match data stored in the database; if the image corresponding to the second accurate matching data exists, determining the image corresponding to the second accurate matching data as a first image, wherein the first image and the image to be processed are the same image; otherwise, the fact that the image which is the same as the image to be processed does not exist in the server is indicated, at this time, a first average absolute error distance between the first perceived hash characteristic value and each perceived hash characteristic value stored in the database is further calculated, after the perceived hash characteristic value with the first average absolute error distance smaller than a preset threshold is determined to be a second hashed characteristic value, the image corresponding to the second hashed characteristic value is determined to be a first image, and the first image and the image to be processed are similar images.
Optionally, when the first image matching the image to be processed is detected in the database, before the step of comparing the image quality of the image to be processed with the image quality of the first image, the method further comprises:
and carrying out ownership verification on the client.
In this embodiment, when the first image is stored in the server, ownership verification is performed on the client. It should be understood that when the deduplication is performed according to the feature sequence, if an attacker guesses or illegally obtains the feature sequence of the image to be processed, it may be used to obtain the usage right, and further obtain the first image, which leads to data leakage, so that security authentication is necessary for the client.
Fig. 3 is a schematic diagram of ownership verification performed by a server provided by an embodiment of the present invention, and fig. 4 is an interactive schematic diagram of a client and a server in the ownership verification process provided by the embodiment of the present invention.
Referring to fig. 3-4, the steps of ownership verification for the client include:
After dividing a plurality of image blocks in a first image, selecting the first image block from the plurality of image blocks, and preprocessing the first image block;
scaling the preprocessed first image block and the first image to the same size, and mixing to obtain a first target image;
inputting a first target image to a first residual neural network to obtain a perceived hash characteristic value of the first target image;
Generating an ownership verification request, wherein the ownership verification request comprises network parameters of the first residual neural network;
Sending an ownership verification request to a client, so that the client divides a plurality of image blocks in an image to be processed, then selects a second image block from the image blocks, pre-processes the second image block, scales the pre-processed second image block and the image to be processed to the same size, mixes the pre-processed second image block and the image to be processed to obtain a second target image, sets network parameters of a second residual neural network according to network parameters of a first residual neural network, inputs the second target image to the second residual neural network, and sends the obtained perceived hash characteristic value of the second target image to a server;
and judging whether the client passes ownership verification according to the perceived hash characteristic value of the first target image and the perceived hash characteristic value of the second target image.
Specifically, as shown in fig. 3, in the process of verifying ownership of a client, a server first divides a first image into a plurality of image blocks, and randomly selects the first image block from the image blocks; then, preprocessing the first image block, wherein the preprocessing comprises rotation and/or translation operation, and scaling the preprocessed first image block; finally, the unprocessed first image is scaled to the same size as the first image block, and then the unprocessed first image and the first image block are mixed to generate a first target image. Further, as shown in fig. 4, after the server generates the first target image, the first target image is input into the first residual neural network to obtain a perceived hash feature value of the first target image.
It can be understood that in this embodiment, ownership verification of the client is performed by comparing a first target image generated by the server side with a second target image generated by the client side, where the generating process of the second target image is the same as that of the first target image, that is, the client divides an image to be processed into a plurality of image blocks, randomly selects the second image block from the plurality of image blocks, performs the same preprocessing operation on the second image block as that of the first image block, scales and mixes the image to be processed and the processed second image block to the same size, and then inputs the mixed second target image into the second residual neural network to obtain the perceptual hash characteristic value of the second target image.
After the server side completes the calculation of the perceived hash characteristic value of the first target image, an ownership verification request is generated and sent, the ownership verification request contains network parameters of the first residual neural network, and the client side sets network parameters of the second residual neural network according to the network parameters of the first residual neural network.
Further, the client sends the perceived hash feature value of the second target image to the server, and the server can judge whether the client passes ownership verification according to the perceived hash feature value of the first target image and the perceived hash feature value of the second target image.
Optionally, please continue to refer to fig. 4, the step of determining whether the client passes the ownership verification according to the perceptual hash feature value of the first target image and the perceptual hash feature value of the second target image includes:
calculating a second average absolute error distance between the perceptual hash feature value of the first target image and the perceptual hash feature value of the second target image;
Comparing the second average absolute error distance with a second preset threshold; if the second average absolute error distance is smaller than a second preset threshold, the client passes ownership verification, otherwise, the client fails ownership verification;
And sending the verification result to the client.
Specifically, the server calculates a second average absolute error distance between the perceived hash feature value of the first target image and the perceived hash feature value of the second target image, and when the second average absolute error distance is smaller than a second preset threshold, the client passes ownership verification; and when the second average absolute error distance is greater than or equal to a second preset threshold, the client fails to pass the ownership verification, and the server refuses the uploading request.
Therefore, the invention does not need to introduce a special server for ownership verification, does not need to set an additional reference image library, and the perceived hash characteristic value for ownership verification cannot be reused, so that the image deduplication method is ensured to have excellent safety and reliability.
It should be noted that, whether the image to be processed is finally uploaded to the server or not, the image to be processed, the first accurate matching data and the first perceptual hash feature value can be stored in the database, and the updated database is favorable for being used as a basis for similarity discrimination in the process of de-duplication of other subsequent images.
As shown in fig. 5, an embodiment of the present invention further provides an image deduplication method based on a residual neural network, which is applied to a client, and includes:
S501, acquiring an image to be processed;
s502, determining first accurate matching data of an image to be processed according to a Sha256 algorithm;
S503, determining a first perception hash characteristic value of an image to be processed by using a pre-trained second residual neutral network model;
S504, sending an uploading request of the image to be processed to a server, so that the server carries out similarity detection on the image to be processed in a database according to the first accurate matching data and the first perception hash characteristic value, and compares the image quality scores of the image to be processed and the first image when the first image matched with the image to be processed is detected;
S505, if the image quality score of the image to be processed is higher than the image quality score of the first image, receiving an uploading instruction and uploading the image to be processed to a server;
s506, if the image quality score of the image to be processed is lower than the image quality score of the first image, receiving the link of the first image.
In the image deduplication method based on the residual neural network, the server receives the uploading request from the client, and because the uploading request comprises the first accurate matching data of the image to be processed and the first perception hash characteristic value, the image can be judged to be similar by comparing the first accurate matching data with the accurate matching data of the image stored in advance in the database, and the image can be judged to be similar by comparing the first perception hash characteristic value with the perception hash characteristic value of the image stored in advance in the database, so that the image with the same visual information can be judged to be similar, and the image deduplication method has good deduplication precision. In addition, when the first image which is the same as or similar to the image to be uploaded exists in the database, the image quality of the first image and the image quality of the second image are compared, and redundant images with poor quality are deleted, so that the user experience is improved while a good duplicate removal effect is obtained.
In the description of the present invention, the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature. In the description of the present invention, the meaning of "a plurality" is two or more, unless explicitly defined otherwise.
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Further, one skilled in the art can engage and combine the different embodiments or examples described in this specification.
Although the application is described herein in connection with various embodiments, other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed application, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.
The foregoing is a further detailed description of the invention in connection with the preferred embodiments, and it is not intended that the invention be limited to the specific embodiments described. It will be apparent to those skilled in the art that several simple deductions or substitutions may be made without departing from the spirit of the invention, and these should be considered to be within the scope of the invention.
Claims (10)
1. An image deduplication method based on a residual neural network is characterized by being applied to a server and comprising the following steps:
receiving an uploading request from a client; the method comprises the steps that communication connection is pre-established between the client and the server, and the uploading request comprises first accurate matching data of an image to be processed and a first perception hash characteristic value;
According to the first accurate matching data and the first perception hash characteristic value, similarity detection is carried out on the image to be processed in a database; the database comprises a plurality of groups of pre-stored images, accurate matching data corresponding to the images and perceived hash characteristic values; the first accurate matching data is obtained by carrying out Sha256 algorithm processing on the image to be processed;
Comparing an image quality score of a to-be-processed image with a first image matched with the to-be-processed image when the first image is detected in the database; and if the image quality score of the image to be processed is higher than the image quality score of the first image, sending an uploading instruction to the client and receiving the image to be processed uploaded by the client.
2. The image deduplication method based on the residual neural network according to claim 1, wherein the step of performing similarity detection on the image to be processed in a database according to the first exact match data and the first perceptual hash feature value comprises:
Detecting whether second exact match data equal to the first exact match data exists in the exact match data stored in the database;
If so, determining the image corresponding to the second accurate matching data as a first image; otherwise, calculating a first average absolute error distance between the first perceptual hash feature value and each perceptual hash feature value stored in the database;
and comparing the first average absolute error distance with a first preset threshold value, and determining the image corresponding to the second hash characteristic value as a first image after determining the perceived hash characteristic value of which the first average absolute error distance is smaller than the preset threshold value as the second hash characteristic value.
3. The residual neural network-based image deduplication method according to claim 2, wherein, when a first image matching an image to be processed is detected in the database, the step of comparing the image quality of the image to be processed with the first image is preceded by the step of:
and carrying out ownership verification on the client.
4. The image deduplication method based on the residual neural network according to claim 3, wherein the step of performing ownership verification on the client comprises:
After dividing a plurality of image blocks in the first image, selecting the first image block from the plurality of image blocks, and preprocessing the first image block;
Scaling the preprocessed first image block and the first image to the same size, and mixing to obtain a first target image;
inputting the first target image to a first residual neural network to obtain a perceived hash characteristic value of the first target image;
generating an ownership verification request, wherein the ownership verification request comprises network parameters of the first residual neural network;
Sending the ownership verification request to a client, so that the client divides a plurality of image blocks in an image to be processed, then selects a second image block from the image blocks, pre-processes the second image block, scales the pre-processed second image block and the image to be processed to the same size, mixes the pre-processed second image block and the image to be processed to obtain a second target image, sets network parameters of a second residual neural network according to network parameters of the first residual neural network, inputs the second target image to the second residual neural network, and sends the obtained perceived hash characteristic value of the second target image to the server;
and judging whether the client passes ownership verification according to the perceived hash characteristic value of the first target image and the perceived hash characteristic value of the second target image.
5. The image deduplication method based on the residual neural network according to claim 4, wherein the step of judging whether the client passes ownership verification according to the perceptual hash feature value of the first target image and the perceptual hash feature value of the second target image comprises:
calculating a second average absolute error distance between the perceived hash feature value of the first target image and the perceived hash feature value of the second target image;
comparing the second average absolute error distance with a second preset threshold; if the second average absolute error distance is smaller than a second preset threshold, the client passes ownership verification, otherwise, the client fails ownership verification;
And sending the verification result to the client.
6. The residual neural network-based image deduplication method of claim 4, wherein the preprocessing comprises rotation and/or translation operations.
7. The residual neural network-based image deduplication method according to claim 1, further comprising, after the step of comparing the image quality score of the image to be processed with the first image when the first image matching the image to be processed is detected in the database:
and if the image quality score of the image to be processed is lower than that of the first image, transmitting a link of the first image to the client.
8. The method for image deduplication based on the residual neural network according to claim 1, wherein when there is no first image matching an image to be processed in the database, the step of performing similarity detection on the image to be processed in the database according to the first exact match data and the first perceptual hash feature value further comprises:
And after the uploading instruction is sent to the client, receiving the image to be processed uploaded by the client.
9. The image deduplication method based on the residual neural network according to claim 1 or 8, further comprising, after the step of receiving the image to be processed uploaded by the client:
And storing the image to be processed, the first accurate matching data and the first perceptual hash characteristic value into a database.
10. An image deduplication method based on a residual neural network is characterized by being applied to a client and comprising the following steps:
acquiring an image to be processed;
determining first exact match data of the image to be processed according to a Sha256 algorithm;
determining a first perception hash characteristic value of the image to be processed by using a pre-trained second residual neutral network model;
Sending an uploading request of the image to be processed to a server, so that the server carries out similarity detection on the image to be processed in a database according to the first accurate matching data and the first perception hash characteristic value, and compares the image quality scores of the image to be processed and the first image when the first image matched with the image to be processed is detected;
If the image quality score of the image to be processed is higher than the image quality score of the first image, receiving an uploading instruction and uploading the image to be processed to the server; otherwise, a link to the first image is received.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110722552.3A CN113568571B (en) | 2021-06-28 | Image de-duplication method based on residual neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110722552.3A CN113568571B (en) | 2021-06-28 | Image de-duplication method based on residual neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113568571A CN113568571A (en) | 2021-10-29 |
| CN113568571B true CN113568571B (en) | 2024-06-04 |
Family
ID=
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016177259A1 (en) * | 2015-05-07 | 2016-11-10 | 阿里巴巴集团控股有限公司 | Similar image recognition method and device |
| CN108021457A (en) * | 2017-10-31 | 2018-05-11 | 阿里巴巴集团控股有限公司 | Data migration method and device |
| CN108055121A (en) * | 2017-10-23 | 2018-05-18 | 北京邮电大学 | The encryption method and decryption method of image |
| CN108182367A (en) * | 2017-12-15 | 2018-06-19 | 西安电子科技大学 | A kind of encrypted data chunk client De-weight method for supporting data update |
| CN109472690A (en) * | 2018-10-25 | 2019-03-15 | 深圳壹账通智能科技有限公司 | The loan measures and procedures for the examination and approval, device, storage medium and electronic equipment based on block chain |
| CN110321447A (en) * | 2019-07-08 | 2019-10-11 | 北京字节跳动网络技术有限公司 | Determination method, apparatus, electronic equipment and the storage medium of multiimage |
| CN111310129A (en) * | 2020-02-20 | 2020-06-19 | 北京海益同展信息科技有限公司 | Method, device and storage medium for maintaining right of image |
| CN111553259A (en) * | 2020-04-26 | 2020-08-18 | 北京宙心科技有限公司 | Image duplicate removal method and system |
| CN112507159A (en) * | 2020-11-20 | 2021-03-16 | 有米科技股份有限公司 | Hash network training method, advertisement image material retrieval method and related device |
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016177259A1 (en) * | 2015-05-07 | 2016-11-10 | 阿里巴巴集团控股有限公司 | Similar image recognition method and device |
| CN108055121A (en) * | 2017-10-23 | 2018-05-18 | 北京邮电大学 | The encryption method and decryption method of image |
| CN108021457A (en) * | 2017-10-31 | 2018-05-11 | 阿里巴巴集团控股有限公司 | Data migration method and device |
| CN108182367A (en) * | 2017-12-15 | 2018-06-19 | 西安电子科技大学 | A kind of encrypted data chunk client De-weight method for supporting data update |
| CN109472690A (en) * | 2018-10-25 | 2019-03-15 | 深圳壹账通智能科技有限公司 | The loan measures and procedures for the examination and approval, device, storage medium and electronic equipment based on block chain |
| CN110321447A (en) * | 2019-07-08 | 2019-10-11 | 北京字节跳动网络技术有限公司 | Determination method, apparatus, electronic equipment and the storage medium of multiimage |
| CN111310129A (en) * | 2020-02-20 | 2020-06-19 | 北京海益同展信息科技有限公司 | Method, device and storage medium for maintaining right of image |
| CN111553259A (en) * | 2020-04-26 | 2020-08-18 | 北京宙心科技有限公司 | Image duplicate removal method and system |
| CN112507159A (en) * | 2020-11-20 | 2021-03-16 | 有米科技股份有限公司 | Hash network training method, advertisement image material retrieval method and related device |
Non-Patent Citations (1)
| Title |
|---|
| 一种基于混沌系统的新型图像加密算法;王宏达;;光学技术;20170515(03);第260-266页 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8837769B2 (en) | Video signature based on image hashing and shot detection | |
| US10657243B2 (en) | Variation analysis-based public turing test to tell computers and humans apart | |
| US11003896B2 (en) | Entity recognition from an image | |
| CN112788033A (en) | Authentication method and authentication system | |
| CN109977684B (en) | Data transmission method and device and terminal equipment | |
| CN111428269B (en) | Data access method, device and terminal equipment | |
| US11985253B2 (en) | Non-fungible token authentication | |
| US11500968B2 (en) | Method of and system for providing access to access restricted content to a user | |
| CN104899499A (en) | Internet image search based Web verification code generation method | |
| CN111898484A (en) | Method and device for generating model, readable storage medium and electronic equipment | |
| CN111651408B (en) | Method, device, terminal and storage medium for acquiring data | |
| JP2009530906A (en) | Endpoint verification using call sign | |
| CN109495471B (en) | Method, device and equipment for judging WEB attack result and readable storage medium | |
| CN111159352A (en) | Encryption and decryption method supporting multi-keyword weighted retrieval and result sorting and capable of being verified | |
| US11328095B2 (en) | Peceptual video fingerprinting | |
| CN113568571B (en) | Image de-duplication method based on residual neural network | |
| CN112445873B (en) | List display processing method, related device, equipment and medium | |
| CN112860933A (en) | Ciphertext image retrieval method and device, terminal equipment and storage medium | |
| CN110618989B (en) | Information processing method, information processing device and related products | |
| CN113568571A (en) | Image de-duplication method based on residual error neural network | |
| CN113688762B (en) | Face recognition method, device, equipment and medium based on deep learning | |
| CN110020040B (en) | Method, device and system for querying data | |
| CN115361376A (en) | Government affair file uploading method and device, electronic equipment and storage medium | |
| CN114398678A (en) | Registration verification method and device for preventing electronic file from being tampered, electronic equipment and medium | |
| CN115225278B (en) | Data-holding proving method based on data-aware hash |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |