CN107545049B

CN107545049B - Picture processing method and related product

Info

Publication number: CN107545049B
Application number: CN201710713380.7A
Authority: CN
Inventors: 宋翔宇; 贺伟; 郭德安; 黄桂洲; 江启泉
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2017-08-18
Filing date: 2017-08-18
Publication date: 2022-10-14
Anticipated expiration: 2037-08-18
Also published as: CN107545049A

Abstract

The embodiment of the invention discloses a picture processing method and a related product, which mainly utilize a cognitive similarity comparison technology, can identify whether a picture has a copyright and what copyright possibly the picture has, and can recommend a similar non-copyright picture to a user, so that the user can replace and use the similar non-copyright picture conveniently, thereby helping the user avoid the use risk possibly brought by the copyright problem and improving the quality of copyright related services.

Description

Picture processing method and related product

Technical Field

The present invention relates to the field of internet technologies, and in particular, to the field of picture processing technologies, and in particular, to a picture processing method, a picture processing apparatus, a computer storage medium, and a service device.

Background

The image comparison technology is to extract the features (such as color, fingerprint, etc.) of the images through a feature extraction algorithm, and then calculate the feature similarity between the images so as to achieve the purpose of comparison. At present, most of the image processing schemes are implemented by using an image comparison technology, and in addition, some schemes are implemented by using model matching on the basis of the image comparison technology, for example: a BoW model (Bag-of-words model), a machine learning training model, and so forth. The above prior schemes all belong to strict similarity comparison, that is, two similar pictures obtained from the comparison result are considered to be strictly similar from the visual point of human or machine. For a picture with copyright, the matching picture searched by the strict similarity comparison is probably the picture itself, and the matching picture also has the copyright problem.

Disclosure of Invention

The embodiment of the invention provides a picture processing method and a related product, which can realize the identification service of picture copyright, recommend a replaceable associated non-copyright picture, help to avoid the risk possibly caused by intentional or unintentional use of the copyright picture and improve the practicability.

In one aspect, an embodiment of the present invention provides an image processing method, including:

acquiring feature information of a target picture to be processed, wherein the feature information comprises color features, main body region features and marking features;

calling a copyright library and a non-copyright library to identify the characteristic information of the target picture so as to confirm the type of the target picture, wherein the copyright library is associated with the non-copyright library;

if the identification fails, acquiring attribute information of the target picture, wherein the attribute information comprises a color attribute, an emotion attribute and a text attribute;

performing similarity matching in the copyright gallery and the non-copyright gallery according to the attribute information of the target picture to obtain matched recommended content;

and outputting the recommended content, wherein the recommended content comprises a copyright picture similar to the target picture and a non-copyright picture associated with the similar copyright picture or comprises a non-copyright picture similar to the target picture.

In the technical scheme, for a target picture to be processed, firstly analyzing characteristic information of the target picture, such as color characteristics, main body region characteristics and marking characteristics; then calling a gallery to identify whether the picture belongs to a copyright picture or a non-copyright picture, if the identification fails, analyzing attribute information of the target picture, such as color attribute, emotional attribute and text attribute, combining the gallery to execute similarity matching to search a copyright picture or a non-copyright picture which is similar to the target picture in a cognitive mode, and finally outputting recommended content; the whole process can identify whether the target picture has the copyright or not, can search the copyright picture or the non-copyright picture similar to the target picture, and can recommend the non-copyright picture which can be used for replacement, so that legal risks caused by intentional or unintentional use of the copyright picture are avoided, the practicability is high, and the quality of picture copyright related services is improved.

As a possible implementation manner, the copyright library includes at least one copyright picture, characteristic information of each copyright picture, attribute information of each copyright picture, and a non-copyright picture associated with each copyright picture;

the copyright-free picture library comprises at least one copyright-free picture, characteristic information of each copyright-free picture and attribute information of each copyright-free picture;

the fact that one copyright picture is associated with one non-copyright picture means that the copyright picture and the non-copyright picture belong to pictures similar in cognition.

In the embodiment, the copyright gallery and the non-copyright gallery are constructed in advance, and the association relationship between the copyright pictures and the non-copyright pictures is established, so that the comparison, matching and query can be performed by utilizing the pre-constructed gallery when the picture copyright related service is provided, and the picture processing efficiency is improved.

As a possible implementation, the color features include a global hue vector and a dominant hue vector;

the subject region features comprise at least one feature descriptor;

the labeling feature comprises an effective label set, and if the effective label set is not empty, the effective label set is indicated to contain at least one effective label for describing the meaning of the picture; and if the effective label set is empty, indicating that the effective label set does not contain effective labels for describing the meanings of the pictures.

In the above embodiment, since the feature information of the picture has a great variety, such as color, texture, shape, and the like, the color feature, the main region feature, and the labeling feature of the picture are selected by repeated experiments and analysis in combination with the visual perception of human vision on the picture, so that the copyright identification of the picture based on the feature information is conveniently realized, and the reliability of the identification result is ensured.

As another possible implementation manner, the acquiring feature information of the target picture to be processed includes:

traversing the color value of each pixel point of the target picture;

constructing a color histogram of the target picture by adopting a color partitioning method according to the color value of each pixel point of the target picture, wherein the color partitioning method defines a plurality of color partitions;

counting the number of pixel points in each color partition, and sequentially combining the counting results to obtain a global hue vector of the target picture;

extracting dominant hue pixel points from the color histogram of the target picture;

and counting the number of dominant hue pixel points in each color partition, and sequentially combining the counting results to obtain a dominant hue vector of the target picture.

In the embodiment, through the construction and analysis of the color histogram, the global hue vector and the dominant hue vector of the picture can be obtained, the similarity comparison between every two pictures is converted into the similarity comparison between vectors, and the picture processing efficiency is improved.

As another possible implementation manner, the acquiring feature information of the target picture to be processed further includes:

extracting at least one main body region characteristic pixel point from the target picture by adopting a characteristic extraction algorithm;

and generating a feature descriptor of each main body region feature pixel point by adopting a feature description algorithm.

In the embodiment, the main region features of the pictures are extracted and described through the algorithm, that is, the main parts of the pictures are extracted and described, the similarity comparison between every two pictures is converted into the similarity comparison between the main parts, and the picture processing efficiency is improved.

creating an effective label set for the target picture;

judging whether a label for describing the meaning expressed by the target picture is acquired;

if not, setting the value of the effective label set to be null;

and if the effective label set is acquired, setting the value of the effective label set to be non-null, screening at least one effective label from the acquired labels by adopting a probability statistical algorithm, and adding the at least one effective label to the effective label set.

In the above embodiment, the meaning expressed by the picture is described by one or more effective tags, and the similarity comparison between two pictures is converted into comparison between two sets describing the meaning of the picture by the effective tag set, so that the picture processing efficiency is improved.

As another possible implementation, the invoking the copyright library and the non-copyright library to identify the feature information of the target picture to confirm the type of the target picture includes:

respectively calculating a first matching degree between the characteristic information of the target picture and the characteristic information of each copyright picture in the copyright picture library;

judging whether the copyright picture matched with the target picture is contained in the copyright picture library or not according to the first matching degree, if so, successfully identifying, and confirming that the target picture is the copyright picture;

if not, respectively calculating a second matching degree between the characteristic information of the target picture and the characteristic information of each non-copyright picture in the non-copyright picture library;

judging whether a non-copyright picture matched with the target picture exists in the non-copyright picture library according to the second matching degree, if so, successfully identifying, and confirming that the target picture is the non-copyright picture;

if not, the authentication fails.

In the embodiment, the matching degree between the picture in the gallery and the target picture is called to identify the target picture, and the threshold value of the matching degree can be set according to the actual situation, so that the picture identification result can be accurately obtained according to the actual requirement.

As another possible implementation, the calculating of the first matching degree or the second matching degree includes:

calculating a matching result A1 between color features of the two pictures, a matching result B1 between main body region features and a matching result C1 of an effective label set;

weighting the A1, the B1 and the C1 according to a preset weighting rule;

and calculating a total score S1 of the A1, the B1 and the C1 after weighting, wherein the total score is used for representing the matching degree between the two pictures.

In the above embodiment, considering that any feature (color feature, main region feature or labeled feature) cannot be well processed in comparison with all picture types, a weighting score calculation mechanism is adopted, weights are set for matching results of each feature according to actual needs, the weighted total score is calculated, and the matching degree between pictures is expressed through the total score, so that the matching results are more comprehensive and accurate.

As another possible implementation manner, the picture processing method further includes:

if the authentication is successful and the target picture is confirmed to be the copyright picture, outputting a first authentication result, wherein the first authentication result at least comprises the copyright picture matched with the target picture and a non-copyright picture associated with the matched copyright picture;

and if the authentication is successful and the target picture is confirmed to be a non-copyright picture, outputting a second authentication result, wherein the second authentication result at least comprises the non-copyright picture matched with the target picture.

In the above embodiment, successful authentication outputs an authentication result that includes a matching copyright picture or a matching non-copyright picture, and also recommends a non-copyright picture that is associated with the matching copyright picture and is available for replacement use, so as to help the user avoid copyright risk.

As yet another possible implementation, the color attributes include a global hue vector;

the emotion attribute comprises an emotion phrase set, and if the emotion phrase set is not empty, the emotion phrase set is represented to contain at least one key phrase for describing picture emotion; if the emotion word group set is empty, indicating that the emotion word group set does not contain key words for describing picture emotion;

the text attribute comprises a text label set, and if the text label set is not empty, the text label set is represented to contain at least one text label phrase for describing the meaning of the picture; and if the text label set is empty, the text label set does not contain the text label phrase for describing the picture meaning.

In the embodiment, as the types of the attribute information of the picture are very many, such as colors, description contents and the like, the color attribute, the emotional attribute and the text of the picture are selected by repeated experiments and analysis and combining the visual cognition of human vision on the picture, so that the similar matching of the picture based on the attribute information is conveniently realized, and the reliability of the matching result is ensured.

As another possible implementation manner, the acquiring attribute information of the target picture includes:

creating an emotion phrase set and a text label set for the target picture;

judging whether a target article to which the target image belongs is acquired;

if not, setting the values of the emotion phrase set and the text label set to be null;

if the target article is acquired, extracting a full text abstract of the target article and upper and lower paragraph abstracts of the corresponding position of the target image in the target article;

performing word segmentation processing on the full text abstract and the upper and lower paragraph abstract to obtain a plurality of first alternative phrases for describing emotion and a plurality of second alternative phrases for describing meaning;

screening at least one key phrase from the first candidate phrases by adopting a probability statistical algorithm, and screening at least one text labeling phrase from the second candidate phrases;

and adding the at least one key phrase to an emotion phrase set of the target picture, and adding the at least one text labeling phrase to a text labeling set of the target picture.

In the embodiment, the emotion phrase set and the text label set of the picture can be obtained by analyzing the abstract and the context paragraph of the article in which the picture is located, the similarity comparison between every two pictures is converted into the similarity comparison between the sets, and the picture processing efficiency is improved.

As another possible implementation manner, the performing similarity matching in the copyright gallery and the non-copyright gallery according to the attribute information of the target picture to obtain a matching recommended content includes:

respectively calculating first similarity between the attribute information of the target picture and the attribute information of each copyright picture in the copyright picture library;

judging whether the copyright gallery contains copyright pictures similar to the target picture or not according to the first similarity, and if so, acquiring the similar copyright pictures and non-copyright pictures associated with the similar copyright pictures to generate recommended content;

if not, respectively calculating a second similarity between the attribute information of the target picture and the attribute information of each non-copyright picture in the non-copyright picture library;

and judging whether a non-copyright picture similar to the target picture exists in the non-copyright gallery according to the second similarity, and if so, acquiring the similar non-copyright picture to generate recommended content.

In the embodiment, the similarity between the picture in the gallery and the target picture is called to identify the target picture, and the similarity threshold value can be set according to the actual situation, so that the picture similarity matching result and the recommended content can be accurately obtained according to the actual requirement.

As still another possible implementation, the calculating of the first similarity or the second similarity includes:

calculating a similar result A2 between color attributes, a similar result B2 between emotion attributes and a similar result C2 between text attributes of the two pictures;

weighting the A2, the B2 and the C2 according to a preset weighting rule;

and calculating a total score S2 of the A2, the B2 and the C2 after weighting, wherein the total score S2 is used for representing the similarity between the two pictures.

In the above embodiment, considering that any attribute similarity matching cannot well process all picture types, a weighting score calculation mechanism is adopted, weights are set for the similarity results of each attribute according to actual needs, the total score after weighting is calculated, and the similarity between pictures is expressed through the total score, so that the results are more comprehensive and accurate.

On the other hand, an embodiment of the present invention further provides an image processing apparatus, which may include:

the characteristic acquisition unit is used for acquiring characteristic information of a target picture to be processed, wherein the characteristic information comprises color characteristics, main body area characteristics and labeling characteristics;

the authentication unit is used for calling a copyright library and a non-copyright library to authenticate the characteristic information of the target picture so as to confirm the type of the target picture, and the copyright library is associated with the non-copyright library;

the attribute acquisition unit is used for acquiring attribute information of the target picture if the identification fails, wherein the attribute information comprises a color attribute, an emotion attribute and a text attribute;

the matching unit is used for executing similar matching in the copyright gallery and the non-copyright gallery according to the attribute information of the target picture to obtain matched recommended content;

and the recommending unit is used for outputting the recommended content, and the recommended content comprises a copyright picture similar to the target picture and a non-copyright picture associated with the similar copyright picture or comprises a non-copyright picture similar to the target picture.

In the technical scheme, aiming at a target picture to be processed, firstly analyzing characteristic information of the target picture, such as color characteristics, main body region characteristics and labeling characteristics; then calling a gallery to identify whether the picture belongs to a copyright picture or a non-copyright picture, if the identification fails, analyzing attribute information of the target picture, such as color attribute, emotional attribute and text attribute, combining the gallery to execute similarity matching to search a copyright picture or a non-copyright picture which is similar to the target picture in a cognitive mode, and finally outputting recommended content; the whole process can identify whether the target picture has the copyright or not, can search the copyright picture or the non-copyright picture similar to the target picture, and can recommend the non-copyright picture which can be used for replacement, so that legal risks caused by intentional or unintentional use of the copyright picture are avoided, the practicability is high, and the quality of picture copyright related services is improved.

the fact that one copyright picture is associated with one non-copyright picture means that the two pictures are similar in cognition.

In the embodiment, the copyright gallery and the non-copyright gallery are constructed in advance, and the association relationship between the copyright picture and the non-copyright picture is established, so that the comparison, matching and query can be performed by utilizing the pre-constructed gallery when the picture copyright related service is provided, and the picture processing efficiency is improved.

As another possible implementation, the color features include a global hue vector and a dominant hue vector;

the subject region features comprise at least one feature descriptor;

In the embodiment, as the types of the feature information of the picture are very many, such as color, texture, shape and the like, the color feature, the main region feature and the labeling feature of the picture are selected by repeated experiments and analysis and combining visual cognition of human vision on the picture, so that the copyright identification of the picture based on the feature information is conveniently realized, and the reliability of the identification result is ensured.

As another possible implementation manner, the feature obtaining unit is specifically configured to:

traversing the color value of each pixel point of the target picture;

counting the number of pixel points in each color partition, and sequentially combining the counting results to obtain a global tone vector of the target picture;

extracting dominant hue pixel points from the color histogram of the target picture; and the number of the first and second groups,

In the embodiment, the global hue vector and the dominant hue vector of the picture can be obtained through the construction and analysis of the color histogram, the similarity comparison between every two pictures is converted into the similarity comparison between vectors, and the picture processing efficiency is improved.

As another possible implementation, the feature obtaining unit is further configured to:

extracting at least one main body region characteristic pixel point from the target picture by adopting a characteristic extraction algorithm; and (c) a second step of,

creating an active tag set for the target picture;

judging whether a label for describing the meaning expressed by the target picture is acquired or not;

if not, setting the value of the effective label set to be null; and (c) a second step of,

As a further possible embodiment, the identification unit is specifically configured to:

judging whether the copyright picture matched with the target picture is contained in the copyright gallery according to the first matching degree, if so, successfully identifying, and confirming that the target picture is the copyright picture;

judging whether a non-copyright picture matched with the target picture exists in the non-copyright picture library according to the second matching degree, if so, successfully identifying, and confirming that the target picture is the non-copyright picture; and the number of the first and second groups,

if not, the authentication fails.

In the embodiment, the matching degree between the picture in the image library and the target picture is called to identify the target picture, and the matching degree threshold value can be set according to the actual situation, so that the picture identification result can be accurately obtained according to the actual requirement.

weighting the A1, the B1 and the C1 according to a preset weighting rule;

As still another possible embodiment, the identification unit is further configured to:

As still another possible embodiment, the color attributes include a global hue vector;

the emotion attribute comprises an emotion phrase set, and if the emotion phrase set is not empty, the emotion phrase set is represented to contain at least one key phrase for describing picture emotion; if the emotion phrase set is empty, indicating that the emotion phrase set does not include key phrases for describing picture emotions;

As another possible implementation manner, the attribute obtaining unit is specifically configured to:

creating an emotion phrase set and a text label set for the target picture;

judging whether a target article to which the target image belongs is acquired;

if not, setting values of the emotion phrase set and the text label set to be null;

if the target article is acquired, extracting the full-text abstract of the target article and the upper and lower paragraph abstracts of the corresponding position of the target picture in the target article;

In the embodiment, the sentiment phrase set and the text label set of the picture can be obtained by analyzing the abstract and the context paragraph of the article in which the picture is positioned, the similarity comparison between every two pictures is converted into the similarity comparison between the sets, and the picture processing efficiency is improved.

As another possible implementation manner, the matching unit is specifically configured to:

judging whether the copyright gallery contains copyright pictures similar to the target picture according to the first similarity, and if so, acquiring the similar copyright pictures and non-copyright pictures associated with the similar copyright pictures to generate recommended content;

In the embodiment, the similarity between the picture in the image library and the target picture is called to identify the target picture, and the similarity threshold value can be set according to the actual situation, so that the picture similarity matching result and the recommended content can be accurately obtained according to the actual requirement.

weighting the A2, the B2 and the C2 according to a preset weighting rule;

In yet another aspect, an embodiment of the present invention further provides a computer storage medium, where one or more instructions are stored, and the one or more instructions are adapted to be loaded by a processor and execute the following steps:

calling a copyright gallery and a non-copyright gallery to identify the characteristic information of the target picture so as to confirm the type of the target picture, wherein the copyright gallery is associated with the non-copyright gallery;

the copyright-free picture library comprises at least one non-copyright picture, characteristic information of each non-copyright picture and attribute information of each copyright-free picture;

the subject region features comprise at least one feature descriptor;

the labeling feature comprises an effective label set, and if the effective label set is not empty, the effective label set is represented to contain at least one effective label for describing the meaning of the picture; and if the effective label set is empty, indicating that the effective label set does not contain effective labels for describing the meanings of the pictures.

As another possible implementation manner, in the process that the one or more instructions are suitable for being loaded by the processor and executing the step of obtaining the feature information of the target picture to be processed, the following steps are specifically executed:

traversing the color value of each pixel point of the target picture;

As still another possible implementation manner, during the step of obtaining the feature information of the target picture to be processed, the one or more instructions are adapted to be loaded by the processor and executed, and further perform the following steps:

creating an effective label set for the target picture;

if not, setting the value of the effective label set to be null;

As another possible implementation manner, in the process of loading and executing the step of calling the copyright library and the non-copyright library to identify the feature information of the target picture to confirm the type of the target picture, the one or more instructions are adapted to specifically execute the following steps:

if not, the authentication fails.

weighting the A1, the B1 and the C1 according to a preset weighting rule;

In the above embodiment, considering that any feature (color feature, main region feature or labeling feature) cannot be well compared with all picture types, a weighting score calculation mechanism is adopted, weights are set for matching results of each feature according to actual needs, then a total score after weighting is calculated, and the matching degree between pictures is expressed through the total score, so that the matching result is more comprehensive and accurate.

As still another possible implementation, the one or more instructions are adapted to be loaded by a processor and to perform the steps of:

if the authentication is successful and the target picture is confirmed to be the copyright picture, outputting a first authentication result, wherein the first authentication result at least comprises a copyright picture matched with the target picture and a non-copyright picture associated with the matched copyright picture;

In the above embodiment, successful authentication outputs an authentication result including a matching copyrighted picture or a matching non-copyrighted picture, and also recommends a non-copyrighted picture that can be used for replacement, in association with the matching copyrighted picture, to help the user avoid copyright risk.

In the embodiment, due to the fact that the types of the attribute information of the pictures are very many, such as colors, description contents and the like, the color attributes, the emotion attributes and the texts of the pictures are selected through repeated experiments and analysis and combined with visual cognition of human vision on the pictures, the similar matching of the pictures based on the attribute information is conveniently achieved, and the reliability of matching results is guaranteed.

As another possible implementation manner, in the process of loading and executing the step of obtaining the attribute information of the target picture, the one or more instructions are adapted to specifically execute the following steps:

creating an emotion phrase set and a text label set for the target picture;

judging whether a target article to which the target image belongs is acquired;

weighting the A2, the B2 and the C2 according to a preset weighting rule;

In another aspect, an embodiment of the present invention further provides a service device, including:

a processor adapted to implement one or more instructions; and the number of the first and second groups,

a computer storage medium storing one or more instructions adapted to be loaded by the processor and to perform the steps of:

In the technical scheme, for a target picture to be processed, firstly analyzing characteristic information of the target picture, such as color characteristics, main body region characteristics and marking characteristics; then calling a gallery to identify whether the picture belongs to a copyright picture or a non-copyright picture, if the identification fails, analyzing attribute information of the target picture, such as color attribute, emotion attribute and text attribute, combining the gallery to execute similarity matching so as to search a copyright picture or a non-copyright picture which is similar to the target picture in cognition, and finally outputting recommended content; the whole process can identify whether the target picture has the copyright or not, can search the copyright picture or the non-copyright picture similar to the target picture, and can recommend the non-copyright picture which can be used for replacement, so that legal risks caused by intentional or unintentional use of the copyright picture are avoided, the practicability is high, and the quality of picture copyright related services is improved.

the subject region features comprise at least one feature descriptor;

traversing the color value of each pixel point of the target picture;

creating an active tag set for the target picture;

if not, setting the value of the effective label set to be null;

In the above embodiment, the meaning expressed by the picture is described by one or more effective tags, and the similarity comparison between two pictures is converted into the comparison between two sets describing the picture meaning by the effective tag set, so that the picture processing efficiency is improved.

As still another possible implementation manner, in the process of loading and executing the step of calling the copyright library and the non-copyright library to identify the feature information of the target picture to confirm the type of the target picture by the processor, the one or more instructions are adapted to specifically execute the following steps:

if not, the authentication fails.

weighting the A1, the B1 and the C1 according to a preset weighting rule;

the text attribute comprises a text label set, and if the text label set is not empty, the text label set is indicated to contain at least one text label phrase for describing the meaning of the picture; and if the text label set is empty, the text label set does not contain the text label phrase for describing the picture meaning.

As another possible implementation manner, in the process that the one or more instructions are adapted to be loaded by a processor and executed to obtain the attribute information of the target picture, the following steps are specifically executed:

creating an emotion phrase set and a text label set for the target picture;

judging whether a target article to which the target image belongs is acquired;

respectively calculating first similarity between the attribute information of the target picture and the attribute information of each copyright picture in the copyright gallery;

weighting the A2, the B2 and the C2 according to a preset weighting rule;

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

Fig. 1 is a flowchart of acquiring color features according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a color histogram provided by an embodiment of the present invention;

fig. 3 is a flowchart of acquiring features of a main area according to an embodiment of the present invention;

fig. 4-7 are related schematic diagrams of the process for acquiring the features of the main body region according to the embodiment of the present invention;

FIG. 8 is a schematic diagram of a descriptive label for a picture provided by an embodiment of the invention;

FIG. 9 is a flowchart illustrating an exemplary embodiment of a method for obtaining annotation features;

fig. 10 is a flowchart of establishing an association relationship between a copyrighted picture and a non-copyrighted picture according to an embodiment of the present invention;

FIG. 11 is a flowchart illustrating an embodiment of obtaining emotion attributes;

fig. 12 is a flowchart of acquiring text attributes according to an embodiment of the present invention;

fig. 13 is a flowchart of a picture processing method according to an embodiment of the present invention;

FIG. 14 is a flowchart of another method for processing pictures according to an embodiment of the present invention;

fig. 15 is a schematic structural diagram of a picture processing apparatus according to an embodiment of the present invention;

fig. 16 is a schematic structural diagram of a service device according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention.

The image comparison technology is to extract image features through a feature extraction algorithm and calculate the similarity between the image features so as to achieve the purpose of comparison; common feature extraction algorithms may include, but are not limited to: a pHash algorithm (Perceptual Hash algorithm, PHA for short), a SIFT (Scale Invariant Feature Transform) algorithm, a SURF (Speeded-Up Robust Features) algorithm, and the like. Picture features may include, but are not limited to: texture features (such as picture fingerprints), color features, shape features, spatial relationship features, and the like. The image comparison technology is often applied to scenes such as image recognition, image retrieval, image identification and the like, and specifically, the image comparison technology is used for comparing the characteristics of the image to be processed with the characteristics of the known image to find the known image similar to or matched with the image to be processed, so that the purposes of recognition, retrieval and identification are achieved. In the prior art, most of the picture processing schemes are implemented by using a picture comparison technology, and in addition, some schemes are implemented by adopting model matching on the basis of the picture comparison technology, such as a BoW model, a machine learning training model and the like. In practical applications, it is found that the existing image processing schemes all belong to strict similarity comparison, that is, in comparison results obtained by the existing schemes, two similar images are considered to be strictly similar from the visual point of human or machine (where the machine may refer to an image processing tool such as APP, plug-in, or the like, or an image processing device such as a terminal, a server, or the like).

Copyright (copyright) is copyright rights that authors of literary, artistic, and scientific works enjoy right to their works. If the author of a picture declares copyright protection, the picture is called a copyright picture, and the picture needs to be authorized by the author to be used, otherwise, the copyright of the author is violated. In contrast, pictures that are not declared copyright protected are referred to as non-copyrighted pictures. Since the use of copyright pictures is limited, copyright-related services such as copyright identification and recommendation services have been developed for pictures. The copyright identification and recommendation service is to identify whether the picture to be processed belongs to the copyright picture or not through means such as a picture comparison technology and recommend a non-copyright picture which can be used for replacement when the picture is identified as the copyright picture, so that legal risks possibly caused by intentional or unintentional use of the copyright picture are avoided to a certain extent. As described above, because the existing image processing schemes all belong to strict similarity comparison, the similar or matched image found by the strict similarity comparison is likely to be the image to be processed itself, and both the image to be processed and the found similar or matched image have copyright problems, which cannot provide recommendation service for users and help to avoid use risks caused by the copyright problems. It can be seen that the conventional picture processing scheme with strict similarity comparison is not very suitable for the copyright related service scene for the picture. Many times, the reason why a user intends to use a certain photo is only because the user is interested in the color of the photo or only interested in a certain part of the photo, that is, if the user is only interested in the color of the copyright photo, the user can search the non-copyright photo with the color similar to that of the copyright photo and recommend the non-copyright photo to the user for alternative use, so that the copyright problem can be avoided; also, if the user is only interested in a certain portion of the copyright picture, finding a similar non-copyright picture to the portion is recommended to the user for alternative use to circumvent the copyright problem. Since strict similarity comparison requires that the feature matching degree of each part between two pictures is higher than a matching threshold (set according to actual needs), the requirement that the parts are similar and replaceable by using a strict similarity comparison technology cannot be met, and thus the embodiment of the invention provides the definition of cognitive similarity comparison. The cognitive similarity comparison is opposite to strict similarity comparison, and means that two similar pictures searched by a picture comparison technology cannot be identified as similar from human vision or machine vision, but can be considered to be mutually replaceable by depending on certain characteristics of the pictures (such as colors, main body region characteristics or labels) or depending on certain information outside the pictures (such as context text information where the pictures are located), and when one picture is adopted to replace the other similar picture, the replacement is acceptable from the view of human (user), so that the two pictures are in cognitive similarity; for example: the two pictures are found to be similar in color expression through comparison, and can be used alternatively, so that the two pictures form cognitive similarity; and the following steps: the two pictures are found to be similar in emotion expression through comparison, and can be used alternatively, so that the two pictures form cognitive similarity; another example is: the main body areas of the two pictures are found to be partially similar through comparison, and the two pictures can be used alternatively, so that the two pictures form cognitive similarity; and so on.

The embodiment of the invention provides a picture processing scheme, which can be applied to a picture copyright related service scene, and specifically, the embodiment of the invention at least solves the following two problems: firstly, if a user submits a target picture to acquire a copyright related service, whether the target picture has a copyright or searches for a copyright picture matched with the target picture needs to be identified, and a similar non-copyright picture which can be used for replacing is recommended to the user, so that the use risk caused by the copyright problem is avoided. The second is as follows: if a user submits a URL (Uniform Resource Locator) of an article or submits a request for obtaining a copyright-related service, it is necessary to identify whether all pictures in the article have a copyright according to the content of the article, and to search a matching copyright picture for a picture having a copyright, and it is also necessary to recommend a similar non-copyright picture that can be used for replacement to the user to help avoid a use risk caused by a copyright problem. The picture processing process of the embodiment of the invention mainly comprises the following steps: firstly, analyzing characteristic information of a target picture to be processed, such as color characteristics, main body area characteristics and labeling characteristics, calling a picture library to identify the target picture as a copyright picture or a non-copyright picture, if the identification is successful, finding a similar non-copyright picture which can be used for replacement if the target picture is identified as the copyright picture, and outputting the matched copyright picture and the non-copyright picture which can be used for replacement as identification results; and if the identification target picture is a non-copyright picture, outputting the matched non-copyright picture as an identification result. Secondly, if the identification fails, continuously analyzing attribute information of the target picture, such as color attribute, emotion attribute and text attribute, calling a gallery to execute similar matching to search a copyright picture or a non-copyright picture similar to the target picture, if the similar copyright picture is searched, finding a similar non-copyright picture which can be used for replacement, and outputting the similar copyright picture and the similar non-copyright picture which can be used for replacement as recommended content; and if the similar non-copyright picture is found, outputting the similar non-copyright picture as recommended content. The whole picture processing process utilizes the cognitive similarity comparison technology, whether the target picture has the copyright or not and what copyright may be possessed can be identified, meanwhile, because the output identification result or the recommendation content comprises the non-copyright picture which is similar to the target picture in a mutual cognitive mode, the user can select to replace the target picture with the similar non-copyright picture for use, and therefore the user is helped to avoid the use risk possibly brought by the copyright problem, and the quality of copyright related services is improved. It can be understood that the image processing scheme provided in the embodiment of the present invention may also be compatible with the prior art, that is, a strict similarity comparison technique and a cognitive similarity comparison technique may be combined, for example, before the identification is performed by using the cognitive similarity comparison technique, the strict similarity comparison technique is first used to pre-match the image to be processed with each image in the gallery, and if a strictly similar copyrighted image or non-copyrighted image is not matched, the aforementioned cognitive similarity comparison scheme is executed.

In order to implement a picture processing scheme more conveniently, in the embodiment of the present invention, a gallery is configured in advance, where the gallery may include a copyright gallery and a non-copyright gallery. The copyright gallery comprises at least one copyright picture, characteristic information of each copyright picture, attribute information of each copyright picture and a non-copyright picture associated with each copyright picture. The copyright-free picture library comprises at least one copyright-free picture, characteristic information of each copyright-free picture and attribute information of each copyright-free picture; it should be noted that, a copyright picture is associated with a non-copyright picture, which means that the two pictures are similar to each other in cognition. Wherein, the characteristic information may include: color features, body region features, and labeling features. The attribute information may include color attributes, emotion attributes, and text attributes.

The following describes the configuration process of the copyright picture and the non-copyright gallery in detail, specifically as follows:

1. and acquiring characteristic information.

1.1 Color features) including a global hue vector and a dominant hue vector. Referring to fig. 1, the process of acquiring the color feature of any one of the pictures may include the following steps s11-s15:

and s11, traversing the color value of each pixel point of the picture.

The basic color which can not be decomposed in the color is called as primary color, the primary color can be synthesized into other colors, but the other colors can not restore the original color of the primary color; the primary colors here refer to Red (Red), green (Green), and Blue (Blue), and all the three primary colors can be mixed. That is, any color is composed of three primary colors, and traversing the picture can obtain the color value of each pixel point of the picture, where the color value is represented by (R, G, B), for example: the color value of pixel point one can be expressed as (R) ₁ ，G ₁ ，B ₁ ) The color value of pixel two can be expressed as (R) ₂ ，G ₂ ，B ₂ ) And so on.

And s12, constructing a color histogram of the picture by adopting a color partitioning method according to the color value of each pixel point of the picture, wherein the color partitioning method defines a plurality of color partitions.

Color histograms are color features widely used in the field of picture processing, and describe the proportion of different colors in the whole picture without regard to the spatial position of the colors. Because any color value is composed of three primary colors, a single primary color of each pixel point in the picture is respectively extracted, and three color histograms can be constructed; as shown in fig. 2, extracting single primary colors R, G, and B for the left picture respectively can construct color histograms shown in the first three pictures from top to bottom on the right side. Synthesizing the extracted primary colors to obtain a color histogram; referring to fig. 2 again, the extracted primary colors R, G, B are synthesized to obtain a color histogram shown in the last diagram on the right side. For three primary colors, the value range of each primary color is [0, 255 ]]For a total of 256 values, if each primary color takes 256 values, the color space becomes very large (256 values) ³ A color ofValues) for such a large color space can be time and labor consuming to analyze and calculate. The embodiment of the invention adopts a color partition method, and the color partition method defines a plurality of color partitions, for example: the color value range [0, 255 ] can be obtained by a color partition method]Four color partitions or eight color partitions are divided, and the four color partitions are taken as an example: will take on a value range of [0, 63 ]]Is divided into a zeroth color zone, [64, 127 ]]Is divided into a first color region, [128, 191 ]]Divided into a second color zone, [192, 255]Dividing the color into a third color zone; the color space formed by this color partitioning method described above has a total of 4 ³ The color value is beneficial to improving the efficiency of analyzing and calculating the color space. After the color space is divided by a color partitioning method, each primary color value of each pixel point in the picture is respectively attributed to a corresponding color zone, and a color histogram of the picture can be constructed.

And s13, counting the number of the pixel points in each color partition, and sequentially combining the counting results to obtain the global tone vector of the picture.

The color value of each pixel of the picture falls into a color space combination, for example: the color value of a certain pixel point i is (R) _i ，G _i ，B _i ) Of primary color R _i Falls into the zeroth color zone, primary color G _i Falls in a second color region, primary color B _i If the value of (b) falls into the third color zone, the color value of the pixel point falls into the combination of (the zeroth color zone, the second color zone and the third color zone). This step needs to count the number of pixels included in each color zone combination, for example: the color space is divided into four color divisions according to the color division method, and then 4 is total ³ If the color gamut of the 64 color gamut combinations is different from the color gamut of the image, the pixel number of each color gamut combination in the 64 color gamut combinations is counted, and then the counted pixel number in each color gamut combination is formed into a 64-dimensional vector in sequence, where the vector may be referred to as a global hue vector of the image and may be used to represent the overall hue of the image.

And s14, extracting dominant hue pixel points from the color histogram of the picture.

And s15, counting the number of dominant hue pixel points in each color partition, and sequentially combining the counting results to obtain a dominant hue vector of the picture.

In steps s14-s15, analyzing the color histogram of the picture to obtain dominant hue pixel points of the picture, specifically, selecting pixel points corresponding to color values of high-frequency overlapping regions in the color histogram, then counting the number of dominant hue pixel points included in each color region combination, and forming a 64-dimensional vector by the number of dominant hue pixel points in each color region combination obtained through statistics in sequence, wherein the vector can be called a dominant hue vector of the picture and can be used for representing the dominant hue of the picture.

The embodiment of the invention can acquire a large number of copyright pictures from the Internet by utilizing a crawler technology and the like and store the pictures in the copyright gallery, specifically can crawl the copyright pictures from websites declaring the copyright or crawl the copyright pictures from some websites specially providing the copyright pictures, and can acquire the color characteristics of each copyright picture and store the color characteristics into the copyright gallery by repeating the steps s11-s 15. Similarly, a large number of non-copyright pictures can be crawled from the internet and stored in the non-copyright picture library, specifically, copyright pictures can be crawled from websites without copyright declaration, or non-copyright pictures can be crawled from websites specially providing open source non-copyright pictures, and then the color characteristics of each non-copyright picture are obtained through the steps s11 to s15 and stored in the non-copyright picture library.

1.2 Subject region features including at least one feature descriptor. For any picture, the main body region refers to the main part thereof, such as a scene graph of the eiffel tower, and the main body is the eiffel tower. In order to extract the features of the subject region of a picture, some feature extraction algorithms may be used, such as: ORB (an ordered Brief, a feature extraction algorithm) algorithm, SIFT algorithm, SURF algorithm, and the like. The following describes the process of obtaining the features of the main region of the picture in detail by taking the ORB algorithm as an example, and referring to fig. 3 together, the process includes the following steps s21-s22:

and s21, extracting at least one main body region characteristic pixel point from the picture by adopting a characteristic extraction algorithm.

Step s21 is a process of extracting feature pixels of the main region from the picture. A characteristic pixel of a picture may be understood as a significant pixel in the picture, such as a contour point, a bright point in a darker area, a dark point in a lighter area, and the like. The feature extraction algorithm aims at detecting and extracting feature pixel points in the picture, and the feature extraction algorithm adopted in the ORB algorithm is a FAST algorithm. The core idea of the FAST algorithm is that: and selecting a pixel point to be compared with the surrounding pixel points, and if the selected pixel point is different from most of the surrounding pixel points, considering the selected pixel point as a characteristic pixel point. Referring to fig. 4, the specific extraction process is as follows:

A. randomly selecting a pixel point P in a picture, wherein on a circle which takes the pixel point P as a center and has a radius of R (R is taken according to actual experience, and the value of R in the embodiment of the invention is assumed to be 3), the total number of the pixel points is 16; as shown in fig. 4, pixel 1, pixel 2.

B. Setting a threshold t, wherein the threshold t can be valued according to actual experience; if the absolute value of the difference between the pixel values of the two pixels is larger than t, the two pixels are different. It should be noted that the pixel value of a pixel refers to the gray-level value of the pixel.

C. Whether the pixel point P is a characteristic pixel point is detected, then the 16 pixel points around the pixel point P need to be considered, and the method specifically comprises the following steps:

c1, respectively calculating absolute values of differences of pixel values between pixels (namely pixel 1 and pixel 9) in the vertical direction and the center P, namely respectively calculating the absolute values of the differences of the pixel values between the pixel 1 and the center P and the absolute values of the differences of the pixel values between the pixel 9 and the center P; if the absolute values obtained by calculation are all larger than the threshold value t, performing the step c2;

c2, calculating absolute values of differences between pixel values of pixels in the vertical direction (namely pixel 1 and pixel 9) and pixel values between pixels in the horizontal direction (pixel 5 and pixel 13) and the center P, and if at least 3 of the calculated absolute values exceed a threshold value t, performing step c3;

c3, calculating absolute values of differences of pixel values between the 16 pixel points and the center P, and if at least 9 of the calculated absolute values exceed a threshold value t, determining the pixel point P as a characteristic pixel point.

Through the above a-C, M characteristic pixel points can be detected in the picture, where M is a positive integer greater than or equal to 1.

D. NMS (Non Maximum Suppression, non-Maximum Suppression) is carried out on the picture to select N (N is a positive integer and is more than or equal to 1 and less than or equal to M) main body region characteristic pixel points, and the method specifically comprises the following steps:

d1, calculating a FAST Score value Score of the detected characteristic pixel points, wherein the Score value is the sum of absolute values of differences of pixel values between 16 pixel points and a center, for example: FAST Score value Score of characteristic pixel point P _p ＝|I ₁ -I _p |+|I ₂ -I _p |+......+|I ₁₆ -I _p I wherein _p Representing the pixel value of the characteristic pixel point P; I.C. A ₁ Representing the pixel values of pixel points 1 around the characteristic pixel point P, and so on, I ₁₆ The pixel values of the pixel points 16 around the representative pixel point P;

d2, counting the number of the characteristic pixel points contained in a neighborhood (such as 3x 3) taking the characteristic pixel point P as the center, and if only one characteristic pixel point P exists in the neighborhood, selecting the characteristic pixel point P as a main body region characteristic pixel point; and if other characteristic pixel points except the characteristic pixel point P exist in the neighborhood, selecting the characteristic pixel point with the maximum FAST value from all the characteristic pixel points in the neighborhood to be determined as the characteristic pixel point of the main region.

And s22, generating a feature descriptor of each main body region feature pixel point by adopting a feature description algorithm.

After obtaining N main body region Feature pixels, it is necessary to describe the attributes of these main body region Feature pixels in some way, and the output of these attributes is referred to as descriptors (Feature descriptors) of these main body region Feature pixels. In the ORB algorithm, a BRIEF algorithm is adopted to calculate a descriptor of a characteristic pixel point of a main body region. The core idea of the BRIEF algorithm is to select n point pairs (n is a positive integer) in a certain mode around the characteristic pixel points of the main body region, and combine the comparison results of the n point pairs to serve as a descriptor. Specifically, the method comprises the following steps:

E. filtering the picture by using a gaussian filter, for example: the gaussian filter employed may include the following parameters: variance was 2 and gaussian window was 9 x 9. Furthermore, the integral image can be used for further filtering noise, and the embodiment of the invention can better solve the problem of noise sensitivity through the Gaussian filter and the integral image.

F. Selecting a characteristic pixel point P of a main body region as a center to obtain a neighborhood window, and randomly selecting n (n is a positive integer) pairs of pixel points in the neighborhood window. For convenience of explanation, n =4, and n can be maximally 512 in practical application; referring also to fig. 5, assume that the currently selected 4 dot pairs are labeled as: p1 (X, Y), P2 (X, Y), P3 (X, Y), P4 (X, Y); defining the ratio of T operation to pixel value, the T operation is defined as the following formula (1)

In the above formula (1), I _x Representing the pixel value of the pixel point X; i is _y Representing the pixel value of pixel Y. Then, the T operation is respectively carried out on the selected point pairs, and the obtained results are coded and combined to obtain the feature descriptors of the feature pixel points of the main body region. Suppose that: t (P1 (X, Y)) =1, T (P2 (X, Y)) =0, T (P3 (X, Y)) =0, T (P4 (X, Y)) =1, then the feature descriptor of the subject region feature pixel point P is 1001.

G. In the process of obtaining the feature descriptors, when the point pairs are selected, a coordinate system is established by taking the feature pixel points of the main area as the original points, the horizontal direction as the horizontal axis and the vertical direction as the vertical axis. When the picture rotates, the coordinate system is unchanged, the point pairs taken out in the same mode are different, the descriptors obtained by calculation are different, and the situation is not in accordance with the actual situation; it is therefore necessary to re-establish the coordinate system so that a new coordinate system can be rotated following the rotation of the picture, so that the point pairs taken out in the same pattern have consistency.

The main body region characteristic pixel points extracted by the FAST algorithm do not have directions, and in order to realize rotation consistency, the main directions of the main body region characteristic pixel points need to be calculated. The specific calculation formula is as follows:

in the above formula (2), m _pq Representing a neighborhood matrix of a characteristic pixel point P, C representing the centroid of the picture, (x, y) is a pixel point in the neighborhood, and I (x, y) represents the gray value of (x, y); θ represents the principal direction of the characteristic pixel point P.

When the feature descriptors are calculated by adopting the BRIEF algorithm, n pairs of pixel points are randomly selected and then rotated for each main body region feature pixel point, the rotation angle is calculated by the formula (2), and the feature descriptors are obtained by utilizing the F mode on the basis, so that the problem of rotation consistency is solved. Through the E-G, the feature descriptors of the feature pixel points of each main body region can be obtained. For example, if an ORB algorithm is used to process a picture, the result shown in fig. 6 can be obtained, where the circle represents the identified characteristic pixel point of the main region, and the region indicated by the dashed box is the main region of the picture.

In the embodiment of the present invention, the main area characteristics of each copyright picture can be obtained by repeating the above steps s21 to s22 and stored in the copyright library. Similarly, the main area characteristics of each non-copyright picture are obtained through the steps s21 to s22 and stored in the non-copyright picture library.

When the pictures are compared pairwise, the minimum distance between the main matching feature pixel points can be calculated, and then the pixel point pairs with better matching can be filtered, for example, please refer to fig. 7 together, after the main region feature pixel points of the eiffel tower shot at different angles are extracted, compared and filtered, a better matching result is obtained, referring to fig. 7, it can be found that the feature points of the region of the eiffel tower are matched more accurately, and from the human vision, the main regions of the two pictures are the eiffel tower, if the left side is a picture with copyright, the right side is a picture without copyright, and the right side picture can be used as one of the recommendation results of the left side picture.

1.3 A label feature) including an active label set, and if the active label set is not empty, indicating that the active label set includes at least one active label for describing the meaning of the picture; and if the effective label set is empty, indicating that the effective label set does not contain effective labels for describing the meanings of the pictures.

When a picture is crawled, some extra information of the picture is generally crawled, and the picture description label is included. For example, please refer to fig. 8, when the picture shown in fig. 8 is crawled, a plurality of corresponding description tags are crawled at the same time; if the tags can represent the meaning of the picture to some extent, one or more valid tags that can best express the meaning of the picture need to be found from the tags, and the valid tags are included in the valid tag set of the picture. Assuming that the label marked with a dashed box in fig. 8 is a valid label, the picture expressed by the valid label set of this picture means "temple-prayer hall".

Referring to fig. 9, the following steps s31-s34 are adopted in the embodiment of the present invention to obtain the labeling feature (including the valid label set) of any one of the pictures, which is as follows:

s31, creating an active set of tags for the picture.

And s32, judging whether a label for describing the meaning of the picture expression is acquired.

And s33, if the valid tag set is not acquired, setting the value of the valid tag set to be null.

And s34, if the valid label set is obtained, setting the value of the valid label set to be non-null, screening the obtained labels by adopting a probability statistical algorithm to obtain at least one valid label, and adding the at least one valid label to the valid label set.

In steps s31-s34, if the tags are crawled, the valid tags may be filtered through a probability statistics algorithm, and if the tags are not crawled, the valid tag set is set to null, which indicates that no tags describing the meaning of the picture are included. Specifically, the screening process of the probability statistic algorithm is as follows:

a) Putting all the crawled labels into a dictionary, and putting all the crawled labels into the dictionary without considering repetition;

b) Counting the occurrence frequency of each label in the dictionary;

c) Counting the occurrence probability = occurrence number/total number of labels in the dictionary of each label;

d) The dictionary is reconstructed in a (label, probability) format. For all the tags crawled from any picture, the probability of the tag is searched in the reconstructed dictionary, and the result is obtained as follows: label a,0.12; label b,0.33; label c,0.01; ...

e) And sequencing the labels of the pictures according to the ascending order of the probability from low to high, taking the first K (K is a preset number) labels, determining the first K labels as effective labels, and adding the effective labels into an effective label set of the pictures. The first K tags are most likely to effectively express the meaning of the picture, i.e. the lower the probability the more distinctive.

In the embodiment of the invention, the marked characteristic of each copyright picture can be obtained by repeating the steps s31-s34 and stored in the copyright library. Similarly, the labeling characteristic of each non-copyright picture is obtained through the steps s31-s34 and stored in the copyright gallery.

1.4 Establish an association relationship.

As can be seen from steps 1.1) to 1.3), feature information of each picture can be represented by a triplet (color feature, main region feature, and label feature), and then, referring to fig. 10 together, a process of establishing an association relationship between a copyrighted picture and a non-copyrighted picture according to an embodiment of the present invention includes the following steps s41 to s44:

and s41, selecting any one copyright picture, and acquiring the triple of the copyright picture.

And s42, sequentially selecting a non-copyright picture from the non-copyright picture library, and acquiring the triple of the selected non-copyright picture.

s43, respectively calculating a matching result between the color features of the two selected pictures as A1, a matching result between the features of the main body area as B1 and a matching result between the labeling features as C1; weighting the A1, the B1 and the C1 according to a preset weighting rule; and calculating a total score S1 of the A1, the B1 and the C1 after weighting, wherein the total score is used for representing the matching degree between the two pictures.

Matching for color features: when the pictures with similar colors are searched, the cosine similarity of the tone vectors between every two pictures can be calculated; the cosine similarity threshold may be set relatively high for strict similarity comparison, and relatively low for cognitive similarity comparison. In the embodiment of the invention, when searching the associated copyright picture for the copyright picture from the copyright picture library, firstly, the cosine similarity of the global hue vector between every two copyright pictures and non-copyright pictures is calculated, and the non-copyright pictures which are larger than a threshold value are screened. The similarity threshold value can be set to be 85%, so that more pictures with similar colors can be selected, irrelevant pictures can be introduced to a great extent, and the threshold value has a good adaptation effect on landscape pictures; secondly, calculating the cosine similarity of the dominant hue vector between every two pictures, and screening the non-copyright pictures which are larger than a threshold value. The threshold may be set at 95% at this point, thus ensuring that irrelevant pictures do not appear in the result set.

Matching for subject region features: suppose that the descriptor of a characteristic pixel point X of a certain main body region of a copyright picture is X:10101011, the descriptor of the characteristic pixel point Y in a certain main area of the non-copyright picture is Y:10101010; a threshold value is set, such as 80%. When the similarity of the descriptors of X and Y is greater than 90%, X and Y are matched, in this example, X and Y are different only in the last digit, and the similarity is 87.5% and greater than 80%; x and Y are matched, and the matching degree of X and Y can be easily calculated by carrying out XOR operation on X and Y; then, the matching degree between the main body region characteristic pixel points between every two pictures is sequentially judged by adopting the scheme, and the matching result of the main body region characteristics between every two pictures can be obtained.

And aiming at the matching of the labeling features: judging the matching degree between the effective label sets of every two pictures, for example: setting a threshold value to be 60%, assuming that an effective label set of a copyright picture comprises three effective labels of a, b and c, and an effective label set of a copyright picture comprises three effective labels of a, b and d, wherein the two effective labels are different only by the last effective label, and the matching degree is 2/3 × 100% =66.7% which is greater than the threshold value, and then the two effective labels are matched.

The calculation of the matching degree of the feature information of the two pictures is shown in formula (3) as follows:

S ₁ ＝u*A ₁ +v*B ₁ +w*C ₁ (wherein u, v, w represent weights, u + v + w =1, v ≧ w>u) (3)

And s44, sequencing the non-copyright pictures in a reverse order according to the sequence of the matching degree from high to low, and taking the first L (L is a positive integer and is a preset number) non-copyright pictures to establish an association relation with the copyright pictures.

2. And acquiring the attribute information.

When a copyright picture and a non-copyright picture are crawled, a large number of high-quality graphics and texts mixed articles are crawled at the same time, and generally, the graphics appearing in one article are related to the content of the article or express a certain emotion contained in the article by an author. Here two rules are utilized: articles that describe similar content are easier to use with similar pictures, such as a tour of a certain attraction; the styles of the pictures of the articles with similar emotional expressions are similar, such as expressing the articles in a scattered manner. The following explains the acquisition process of the attribute information as follows:

2.1 Color attributes including a global hue vector.

Color can well express the emotion expressed by a picture, such as: a picture expressing happy emotion, the color of which is usually bright; a picture expressing sad emotions is usually dark in color. The embodiment of the invention can obtain the color attribute of each copyright picture by repeating the steps s11-s13 shown in figure 1 and store the color attribute into the copyright picture library. Similarly, the color attribute of each non-copyright picture can be obtained through steps s11-s13 shown in fig. 1 and stored in the non-copyright picture library.

2.2 The emotion attribute comprises an emotion phrase set, and if the emotion phrase set is not empty, the emotion phrase set is represented by recording at least one key phrase for describing picture emotion; and if the emotion phrase set is empty, indicating that the emotion phrase set does not include key phrases for describing picture emotions.

Referring to fig. 11, the following steps s51-s57 are adopted by the embodiment of the present invention to obtain the emotion attribute of any one of the pictures, which are as follows:

and s51, creating an emotion phrase set for the picture.

And s52, judging whether the article to which the picture belongs is acquired.

And s53, if the emotion phrase set is not acquired, setting the value of the emotion phrase set to be null.

And s54, if the text summary is acquired, extracting the full text summary of the article and the summaries of the upper paragraph and the lower paragraph of the corresponding position of the picture in the article.

And s55, performing word segmentation processing on the full text abstract and the upper and lower paragraph abstracts to obtain a plurality of first alternative phrases for describing emotion.

And s56, screening at least one key phrase from the first candidate phrases by adopting a probability statistical algorithm.

s57, adding the at least one keyword group to the emotion phrase set of the picture.

In steps s51-s57, if the article to which the picture belongs is crawled, a full text abstract of the article and a context paragraph abstract of the corresponding position of the picture in the article can be extracted, and a keyword group expressing the picture emotion is screened through a probability statistical algorithm. If the article to which the picture belongs is not crawled, the emotion phrase set is set to be null, namely, the keyword phrase for describing the emotion of the picture is not included. Specifically, the screening process is as follows:

a) A text segmentation dictionary is prepared. The text segmentation dictionary can be used for collecting meaningless words, and can be used for conveniently performing segmentation processing on the full text abstract and the context paragraph abstract and removing meaningless word groups;

b) A text emotion dictionary is prepared. Some phrases expressing emotion (happy, peaceful, disliked, etc.) can be recorded in the text emotion dictionary, and the emotion phrases can be screened from all phrases after full text abstract and context paragraph abstract word segmentation by using the text emotion dictionary. Referring to the probability statistics algorithm involved in steps s31-s34 shown in fig. 9, a plurality of keyword sets for describing emotion can be obtained by screening, and these keyword sets are added to the emotion phrase set of the picture. And finishing obtaining the emotion attribute of the picture.

In the embodiment of the invention, the emotional attribute of each copyright picture can be obtained by repeating the steps s51-s57 and stored in the copyright gallery. Similarly, the emotional attribute of each non-copyright picture can be obtained through the steps s51-s57 and stored in the non-copyright picture library.

2.3 The text attribute comprises a text label set, and if the text label set is not empty, the text label set is represented to contain at least one text label phrase for describing the meaning of the picture; and if the text label set is empty, the text label set does not contain the text label phrase for describing the picture meaning.

Referring to fig. 12, the following steps s61-s67 are adopted by the embodiment of the present invention to obtain the emotion attribute of any one of the pictures, which are as follows:

and s61, creating a text label set for the picture.

And s62, judging whether the article to which the picture belongs is acquired.

And s63, if the value is not obtained, setting the value of the text label set to be null.

And s64, if the target article is acquired, extracting the full-text abstract of the article and the abstract of the upper paragraph and the lower paragraph of the corresponding position of the picture in the target article.

And s65, performing word segmentation processing on the full text abstract and the upper and lower paragraph abstract to obtain a plurality of second alternative phrases for describing meanings.

And s66, screening at least one text labeling phrase from the second candidate phrases by adopting a probability statistical algorithm.

s67, adding the at least one text labeling phrase to the text labeling set of the picture.

In steps s61-s67, if the article to which the picture belongs is crawled, a full text abstract of the article and a context paragraph abstract of the corresponding position of the picture in the article can be extracted, and a text labeling phrase expressing the picture emotion is screened through a probability statistical algorithm. If the article to which the picture belongs is not crawled, the text label set is set to be null, namely the text label phrase for describing the meaning of the picture is not included. Specifically, the screening process is as follows:

a) The text word segmentation dictionary is adopted, so that the full text abstract and the context paragraph abstract can be conveniently subjected to word segmentation processing, and meaningless phrases are removed;

b) The label dictionary related to steps s31-s34 shown in fig. 9 can be used to screen text-labeled phrases from all phrases after segmentation of the full-text abstract and the context paragraph abstract, and screen a plurality of text-labeled phrases for describing the meaning of the picture by a probability statistical algorithm, and add the text-labeled phrases to the text-labeled set of the picture. And finishing the acquisition of the text attribute of the picture.

The embodiment of the invention can obtain the text attribute of each copyright picture by repeating the steps s61-s67 and store the text attribute into the copyright gallery. Similarly, the text attribute of each non-copyright picture can be obtained through the steps s61-s67 and stored in the non-copyright picture library.

The construction process of the gallery is finished, and the copyright gallery and the non-copyright gallery can be updated in real time or at regular time; the updating here may include: and crawling a new picture from the Internet, storing the new picture in a corresponding image library, and storing the characteristic information, the attribute information and the association relation of the new picture according to the process, or synchronously updating the characteristic information, the attribute information and the association relation of the new picture when the picture in the image library is subjected to editing operation such as deletion and the like.

Based on the above description, an embodiment of the present invention provides a method for processing a picture, please refer to fig. 13, and the method may include the following steps S101 to S105.

S101, obtaining characteristic information of a target picture to be processed, wherein the characteristic information comprises color characteristics, main body region characteristics and labeling characteristics.

In the feature information of the target picture, the color features comprise a global tone vector and a dominant tone vector; the subject region feature comprises at least one feature descriptor; the labeling feature comprises an effective label set, and if the effective label set is not empty, the effective label set is represented to contain at least one effective label for describing the meaning of the target picture; and if the valid tag set is empty, indicating that the valid tag set does not include a valid tag for describing the meaning of the target picture.

In this embodiment, the process of obtaining the color feature of the target picture may include: traversing the color value of each pixel point of the target picture; constructing a color histogram of the target picture by adopting a color partitioning method according to the color value of each pixel point of the target picture, wherein the color partitioning method defines a plurality of color partitions; counting the number of pixel points in each color partition, and sequentially combining the counting results to obtain a global hue vector of the target picture; extracting dominant hue pixel points from the color histogram of the target picture; and counting the number of dominant hue pixel points in each color partition, and sequentially combining the counting results to obtain a dominant hue vector of the target picture. This acquisition process can be seen in steps s11-s15 shown in fig. 1, and is not described in detail here.

The acquisition process of the subject region feature of the target picture may include: extracting at least one main body region characteristic pixel point from the target picture by adopting a characteristic extraction algorithm; and generating a feature descriptor of each main body region feature pixel point by adopting a feature description algorithm. This acquisition process can be seen in steps s21-s22 shown in FIG. 3, and is not described herein.

The obtaining process of the labeling feature of the target picture can include: creating an active tag set for the target picture; judging whether a label for describing the meaning expressed by the target picture is acquired or not; if not, setting the value of the effective label set to be null; and if the effective label set is acquired, setting the value of the effective label set to be non-null, screening at least one effective label from the acquired labels by adopting a probability statistical algorithm, and adding the at least one effective label to the effective label set. This acquisition process can be seen in steps s31-s34 shown in FIG. 9, and is not described in detail herein.

And S102, calling a copyright library and a non-copyright library to identify the characteristic information of the target picture so as to confirm the type of the target picture, wherein the copyright library is associated with the non-copyright library.

The purpose of authentication is to confirm whether the target picture is a copyrighted picture or a non-copyrighted picture. The present embodiment can be identified by using a picture alignment technique, where the picture alignment may include strict similarity alignment or cognitive similarity alignment, for example: firstly, a strict similarity comparison technology is adopted to find whether a copyright picture or a non-copyright picture matched with a target picture exists in a picture library, if so, the identification is successful, otherwise, a cognitive similarity comparison technology is adopted to find whether the picture library contains the copyright picture or the non-copyright picture matched with the target picture, if so, the identification is successful, otherwise, the identification fails. If the identification is successful, the target picture can be confirmed to be a copyright picture or a non-copyright picture, if the target picture is identified to be the copyright picture, a similar non-copyright picture which can be used for replacement is found according to the association relation, the matched copyright picture and the non-copyright picture which can be used for replacement are output as identification results, and a prompt informing that the use of the target picture possibly has copyright risk can be output; and if the target picture is identified to be a non-copyright picture, outputting the matched non-copyright picture as an identification result, and outputting a prompt for informing that the use of the target picture has no copyright risk temporarily. On the contrary, if the identification fails, it indicates that the image comparison technology based on the feature information cannot confirm the type of the target image, and a subsequent cognitive similarity comparison process based on the attribute information needs to be started.

S103, if the identification fails, acquiring attribute information of the target picture, wherein the attribute information comprises a color attribute, an emotion attribute and a text attribute.

In the attribute information of the target picture, the color attribute includes a global hue vector. The emotion attribute comprises an emotion phrase set, and if the emotion phrase set is not empty, the emotion phrase set is represented to contain at least one key phrase for describing picture emotion; and if the emotion word group set is empty, indicating that the emotion word group set does not contain key words for describing picture emotions. The text attribute comprises a text label set, and if the text label set is not empty, the text label set is represented to contain at least one text label phrase for describing the meaning of the picture; and if the text label set is empty, indicating that the text label set does not include text label phrases for describing the meanings of the pictures.

The process of obtaining the color attribute of the target picture can refer to steps s11-s13 shown in fig. 1, which is not described herein again.

The obtaining process of the emotion attribute of the target picture can comprise the following steps: creating an emotion phrase set for the target picture; judging whether a target article to which the target image belongs is acquired; if not, setting the value of the emotion phrase set to be null; if the target article is acquired, extracting a full text abstract of the target article and upper and lower paragraph abstracts of the corresponding position of the target image in the target article; performing word segmentation processing on the full text abstract and the upper and lower paragraph abstract to obtain a plurality of first alternative phrases for describing emotion; screening at least one key phrase from the first candidate phrases by adopting a probability statistical algorithm; and adding the at least one key phrase to the emotion phrase set of the target picture. The obtaining process can refer to steps s51-s57 shown in fig. 11, which are not described herein.

The acquisition process of the text attribute of the target picture comprises the following steps: creating a text label set for the target picture; judging whether a target article to which the target image belongs is acquired; if not, setting the value of the text label set to be null; if the target article is acquired, extracting the full-text abstract of the target article and the upper and lower paragraph abstracts of the corresponding position of the target picture in the target article; performing word segmentation processing on the full text abstract and the upper and lower paragraph abstract to obtain a plurality of second alternative phrases for describing meanings; screening at least one text labeling phrase from the second candidate phrases by adopting a probability statistical algorithm; and adding the at least one text labeling phrase to the text labeling set of the target picture. This process can be seen in steps s61-s67 shown in FIG. 12, which are not described herein.

And S104, performing similar matching in the copyright gallery and the non-copyright gallery according to the attribute information of the target picture to obtain matched recommended content.

And S105, outputting the recommended content, wherein the recommended content comprises a copyright picture similar to the target picture and a non-copyright picture associated with the similar copyright picture, or comprises a non-copyright picture similar to the target picture.

In steps S104-S105, the purpose of similarity matching is to find a copyrighted picture or a non-copyrighted picture that is cognitively similar to the target picture based on the attribute information. Specifically, cognitive similarity comparison is carried out on the copyright pictures in the copyright gallery and the target pictures, cognitive similarity comparison is carried out on the non-copyright pictures in the non-copyright gallery and the target pictures, and matched recommended contents are obtained. If the similar copyright picture is found, finding a similar non-copyright picture which can be used for replacing, and outputting the similar copyright picture and the similar non-copyright picture which can be used for replacing as recommended content; and if the similar non-copyright picture is found, outputting the similar non-copyright picture as recommended content. It can be understood that if similar copyright pictures or non-copyright pictures cannot be found in the gallery by using the cognitive similarity technology, the prompt information of authentication failure can be output to remind the user that the copyright authentication result cannot be obtained currently and the recommendation service cannot be provided.

In the embodiment of the image processing method, for a target image to be processed, firstly, characteristic information of the target image, such as color characteristics, main body region characteristics and labeling characteristics, is analyzed; then calling a gallery to identify whether the picture belongs to a copyright picture or a non-copyright picture, if the identification fails, analyzing attribute information of the target picture, such as color attribute, emotional attribute and text attribute, combining the gallery to execute similarity matching to search a copyright picture or a non-copyright picture which is similar to the target picture in a cognitive mode, and finally outputting recommended content; the whole process can identify whether the target picture has the copyright or not, can search the copyright picture or the non-copyright picture similar to the target picture, and can recommend the non-copyright picture which can be used for replacement, so that legal risks caused by intentional or unintentional use of the copyright picture are avoided, the practicability is high, and the quality of picture copyright related services is improved.

Referring to fig. 14, the method may include the following steps S201 to S216.

S201, obtaining feature information of a target picture to be processed, wherein the feature information comprises color features, main body region features and labeling features.

S202, respectively calculating a first matching degree between the characteristic information of the target picture and the characteristic information of each copyright picture in the copyright gallery.

S203, judging whether the copyright picture matched with the target picture is contained in the copyright picture library according to the first matching degree, if so, turning to the step S204, successfully identifying and confirming that the target picture is the copyright picture. If the determination result is negative, it indicates that the packet is not included, the process proceeds to step S205.

S205, respectively calculating a second matching degree between the feature information of the target picture and the feature information of each non-copyright picture in the non-copyright picture library.

S206, judging whether a non-copyright picture matched with the target picture exists in the non-copyright picture library according to the second matching degree, if so, turning to the step S207, successfully identifying and confirming that the target picture is the non-copyright picture. If the determination result is negative, it indicates that the program is absent, the process proceeds to step S208.

And S208, if the identification does not exist, the identification fails.

Steps S202-S208 may be a specific refinement of step S102 shown in fig. 13. Specifically, the identification process is described, wherein the calculation process of the first matching degree or the second matching degree comprises the following steps: calculating a matching result A1 between color features of the two pictures, a matching result B1 between main body region features and a matching result C1 of an effective label set; weighting the A1, the B1 and the C1 according to a preset weighting rule; and calculating a total score S1 of the A1, the B1 and the C1 after weighting, wherein the total score is used for representing the matching degree between the two pictures. The calculation process can be referred to the description of step s44 shown in fig. 10, and is not described herein again.

S209, if the identification fails, acquiring attribute information of the target picture, wherein the attribute information comprises a color attribute, an emotion attribute and a text attribute.

S210, respectively calculating a first similarity between the attribute information of the target picture and the attribute information of each copyright picture in the copyright gallery.

S211, judging whether the copyright picture similar to the target picture is contained in the copyright picture library or not according to the first similarity, and if the judgment result shows that the copyright picture is contained, turning to S212 to obtain the similar copyright picture and a non-copyright picture associated with the similar copyright picture to generate recommended content; thereafter, the process proceeds to step S216. If the determination result is negative, it indicates that the packet is not included, the process proceeds to step S213.

And S213, respectively calculating a second similarity between the attribute information of the target picture and the attribute information of each non-copyright picture in the non-copyright picture library.

S214, judging whether a non-copyright picture similar to the target picture exists in the non-copyright picture library according to the second similarity, if so, turning to the step S215 to obtain the similar non-copyright picture to generate the recommended content.

Steps S210 to S214 may be a specific refinement step of step S104 shown in fig. 13, specifically describing a process of similarity matching, wherein the calculation process of the first similarity or the second similarity includes: calculating a similar result A2 between color attributes, a similar result B2 between emotion attributes and a similar result C2 between text attributes of the two pictures; weighting the A2, the B2 and the C2 according to a preset weighting rule; and calculating a total score S2 of the A2, the B2 and the C2 after weighting, wherein the total score S2 is used for representing the similarity between the two pictures.

S216, outputting the recommended content, wherein the recommended content comprises a copyright picture similar to the target picture and a non-copyright picture associated with the similar copyright picture, or comprises a non-copyright picture similar to the target picture.

In the embodiment of the image processing method, the cognition similarity comparison technology is utilized, so that whether the target image has the copyright or not and what copyright possibly the target image has can be identified, and meanwhile, because the output identification result or the recommendation content comprises the non-copyright image which is similar to the target image in cognition, the user can select to replace the target image with the similar non-copyright image for use, so that the user is helped to avoid the use risk possibly brought by the copyright problem, and the quality of copyright related services is improved.

The embodiment of the invention is at least suitable for the following two scenes, namely a scene I: the user submits the picture request identification and recommendation service, the target picture at this time is one or more pictures submitted by the user, the submitted pictures can be identified and similar matched through the picture comparison technology by the method of the embodiment shown in the figures 13-14, the copyright pictures similar to the pictures submitted by the user are found, the picture submitted by the user is informed of which kind of copyright is available, and where the user purchases the pictures with complete copyright, and the picture without copyright with a certain close relation to the submitted pictures can be recommended to the user. Scene two: the user submits an article URL or article content, where the target picture is one or more pictures in the article, and through the method of the embodiments shown in fig. 13-14, all pictures in the article can be identified through the picture comparison technology to indicate which pictures are copyrighted, and the non-copyrighted pictures that can be replaced by the user are recommended according to the text passage in which the pictures are located or the entire article content.

Based on the description of the foregoing image processing method embodiment, an embodiment of the present invention further discloses an image processing apparatus, which may be a computer program (including program codes), and the computer program may be run in an independent service device formed by a single server or a cluster service device formed by multiple servers, and may cooperate with a client to implement the image processing method shown in any one of fig. 13 to 14; the client here may be an application client running on a terminal such as a PC (Personal Computer), a mobile phone, a PDA (tablet Computer), etc., including a browser, an instant messaging application, etc., for example: the user can submit the picture and request the copyright related service through the client, and the picture processing device running in the service equipment responds and executes the picture processing method. Referring to fig. 15, the picture processing apparatus operates as follows:

the feature acquiring unit 101 is configured to acquire feature information of a target picture to be processed, where the feature information includes a color feature, a main body region feature, and a labeling feature;

an authentication unit 102, configured to invoke a copyright library and a non-copyright library to authenticate feature information of the target picture to confirm a type of the target picture, where the copyright library is associated with the non-copyright library;

the attribute acquiring unit 103 is configured to acquire attribute information of the target picture if the identification fails, where the attribute information includes a color attribute, an emotion attribute, and a text attribute;

a matching unit 104, configured to perform similar matching in the copyright gallery and the non-copyright gallery according to the attribute information of the target picture, so as to obtain a matched recommended content;

and the recommending unit 105 is used for outputting the recommended content, wherein the recommended content comprises a copyright picture similar to the target picture and a non-copyright picture associated with the similar copyright picture or comprises a non-copyright picture similar to the target picture.

As a possible implementation manner, the copyright library includes at least one copyright picture, feature information of each copyright picture, attribute information of each copyright picture, and a non-copyright picture associated with each copyright picture;

the subject region features comprise at least one feature descriptor;

As another possible implementation manner, in the process of operating the feature obtaining unit 101, the picture processing apparatus specifically includes:

traversing the color value of each pixel point of the target picture;

As still another possible implementation manner, in the process of operating the feature acquiring unit 101, the picture processing apparatus further includes:

extracting at least one main body region characteristic pixel point from the target picture by adopting a characteristic extraction algorithm; and the number of the first and second groups,

As still another possible implementation, in the process of operating the feature acquiring unit 101, the picture processing apparatus further includes:

creating an effective label set for the target picture;

As another possible implementation manner, in the process of operating the authentication unit 102, the picture processing apparatus specifically includes:

if not, the authentication fails.

weighting the A1, the B1 and the C1 according to a preset weighting rule;

As another possible implementation manner, in the process of operating the authentication unit 102, the image processing apparatus further includes:

the text attribute comprises a text label set, and if the text label set is not empty, the text label set is represented to contain at least one text label phrase for describing the meaning of the picture; and if the text label set is empty, indicating that the text label set does not include text label phrases for describing the meanings of the pictures.

As another possible implementation manner, in the process of operating the attribute obtaining unit 103, the picture processing apparatus specifically includes:

creating an emotion phrase set and a text label set for the target picture;

judging whether a target article to which the target image belongs is acquired;

As another possible implementation manner, in the process of operating the matching unit 104, the picture processing apparatus specifically includes:

weighting the A2, the B2 and the C2 according to a preset weighting rule;

According to an embodiment of the present invention, steps S101 to S105 involved in the picture processing method shown in fig. 13 may be performed by respective units in the picture processing apparatus shown in fig. 15. For example, steps S101 to S105 shown in fig. 13 may be performed by the feature acquisition unit 101, the authentication unit 102, the attribute acquisition unit 103, the matching unit 104, and the output unit 105 shown in fig. 15, respectively.

According to another embodiment of the present invention, steps S201 to S216 involved in the picture processing method shown in fig. 14 may be performed by respective units in the picture processing apparatus shown in fig. 15. For example, steps S201, S202 to S208, S209, S210 to S215, S216 shown in fig. 14 may be performed by the feature acquisition unit 101, the authentication unit 102, the attribute acquisition unit 103, the matching unit 104, and the output unit 105 shown in fig. 15, respectively.

According to another embodiment of the present invention, the units in the picture processing apparatus shown in fig. 15 may be respectively or entirely combined into one or several other units to form the unit, or some unit(s) therein may be further split into multiple units with smaller functions to form the unit(s), which may achieve the same operation without affecting the achievement of the technical effect of the embodiment of the present invention. The units are divided based on logic functions, and in practical applications, the functions of one unit can also be implemented by a plurality of units, or the functions of a plurality of units can also be implemented by one unit. In other embodiments of the present invention, the image processing apparatus may also include other units, and in practical applications, these functions may also be implemented by assistance of other units, and may be implemented by cooperation of multiple units.

According to another embodiment of the present invention, the picture processing apparatus device shown in fig. 15 can be configured by running a computer program (including program codes) capable of executing the steps involved in the picture processing methods shown in fig. 13 to 14 on a general-purpose computing device such as a computer including a Central Processing Unit (CPU), a random access storage medium (RAM), a read-only storage medium (ROM), and the like, and a storage element, and the picture processing method of the embodiment of the present invention can be implemented. The computer program may be recorded on a computer-readable recording medium, for example, and loaded and executed in the above-described computing apparatus via the computer-readable recording medium.

In the embodiment of the image processing device, the cognition similarity comparison technology is utilized, so that whether the target image has the copyright or not and what copyright may be possessed by the target image can be identified, and meanwhile, because the output identification result or the recommendation content both comprise the non-copyright image which is similar to the target image in the cognition, the user can select to replace the target image with the similar non-copyright image for use, so that the user is helped to avoid the use risk possibly brought by the copyright problem, and the quality of copyright related services is improved.

Based on the image processing method and the image processing apparatus shown in the foregoing embodiments, embodiments of the present invention further provide a service device, where the service device may be configured to execute corresponding steps of the method flows shown in fig. 13 to fig. 14. In specific implementation, the service device described in the embodiment of the present invention may be an independent service device formed by a single server or a cluster service device formed by multiple servers, for example, in the service device shown in fig. 16, the number of servers may be increased or decreased according to actual business needs; the service device described in the embodiment of the present invention may be used in cooperation with a client to implement the image processing method shown in any one of embodiments in fig. 13 to 14; the client here may be an application client running on a terminal such as a PC (Personal Computer), a mobile phone, a PDA (tablet Computer), etc., including a browser, an instant messaging application, etc., for example: and the user can submit the picture and request the copyright related service through the client, and the service equipment responds and executes the picture processing method. Referring to fig. 16, the internal structure of the service device at least includes a processor, a user interface and a computer storage medium. The processor, the user interface and the computer storage medium in the service device may be connected by a bus or other means, and fig. 16 shows an example of the connection by a bus in the embodiment of the present invention.

The user interface is a medium for implementing interaction and information exchange between a user and a service device, and may be embodied by a Display screen (Display) for outputting, a Keyboard (Keyboard) for inputting, and the like. However, it should be understood that the user interface may also include one or more other physical user interface devices such as a mouse and/or joystick. The processor (or CPU) is a computing core and a control core of the service device, and is adapted to implement one or more instructions, and in particular, is adapted to load and execute one or more instructions to implement corresponding method flows or corresponding functions; for example: the CPU can be used for analyzing a power-on and power-off instruction sent to the service equipment by a user and controlling the service equipment to carry out power-on and power-off operation; the following steps are repeated: the CPU may transmit various types of interactive data between the internal structures of the service device, and so on. A computer storage medium (Memory) is a Memory device in a service device for storing programs and data. It is understood that the computer storage medium herein may include both the built-in storage medium of the service device and, of course, the extended storage medium supported by the service device. The computer storage medium provides a storage space that stores an operating system of the service device. Also, the memory space stores one or more instructions, which may be one or more computer programs (including program code), adapted to be loaded and executed by the processor. The computer storage medium may be a high-speed RAM memory, or may be a non-volatile memory (non-volatile memory), such as at least one disk memory; and optionally at least one computer storage medium located remotely from the processor.

In the embodiment of the present invention, the processor loads and executes one or more instructions stored in the computer storage medium to implement the corresponding steps of the method flows shown in fig. 13-14; in a specific implementation, one or more instructions in a computer storage medium are loaded by a processor and perform the following steps:

In the technical scheme, aiming at a target picture to be processed, firstly analyzing characteristic information of the target picture, such as color characteristics, main body region characteristics and labeling characteristics; then calling a gallery to identify whether the picture belongs to a copyright picture or a non-copyright picture, if the identification fails, analyzing attribute information of the target picture, such as color attribute, emotion attribute and text attribute, combining the gallery to execute similarity matching so as to search a copyright picture or a non-copyright picture which is similar to the target picture in cognition, and finally outputting recommended content; the whole process can identify whether the target picture has the copyright or not, can search the copyright picture or the non-copyright picture similar to the target picture, and can recommend the non-copyright picture which can be used for replacement, so that legal risks caused by intentional or unintentional use of the copyright picture are avoided, the practicability is high, and the quality of picture copyright related services is improved.

the subject region features comprise at least one feature descriptor;

traversing the color value of each pixel point of the target picture;

As another possible implementation manner, during the step of loading and executing the one or more instructions by the processor, the following steps are further executed:

creating an active tag set for the target picture;

if not, setting the value of the effective label set to be null;

and if the effective label set is acquired, setting the value of the effective label set to be non-empty, screening at least one effective label from the acquired labels by adopting a probability statistical algorithm, and adding the at least one effective label to the effective label set.

if not, the authentication fails.

weighting the A1, the B1 and the C1 according to a preset weighting rule;

the emotion attribute comprises an emotion phrase set, and if the emotion phrase set is not empty, the emotion phrase set is represented by the fact that at least one key phrase for describing picture emotion is included in the emotion phrase set; if the emotion word group set is empty, indicating that the emotion word group set does not contain key words for describing picture emotion;

creating an emotion phrase set and a text label set for the target picture;

judging whether a target article to which the target image belongs is acquired;

weighting the A2, the B2 and the C2 according to a preset weighting rule;

In the embodiment, considering that any attribute similarity matching cannot well process all picture types, a weighting score calculation mechanism is adopted, weights are set for the similarity results of each attribute according to actual needs, the total score after the weighting is calculated, and the similarity between pictures is expressed through the total score, so that the results are more comprehensive and accurate.

In the embodiment of the service device, a cognitive similarity comparison technology is utilized, so that whether a target picture has a copyright or not and what copyright may be possessed can be identified, meanwhile, since the output identification result or the recommendation content both include non-copyright pictures which are similar to the target picture in a cognitive manner, a user can select to replace the target picture with the similar non-copyright pictures for use, thereby helping the user avoid the use risk possibly brought by the copyright problem and improving the quality of copyright related services.

It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It is also to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the specification of the present invention and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

It should be further understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

As used in this specification and the appended claims, the term "if" may be interpreted contextually as "when", "upon" or "in response to a determination" or "in response to a detection". Similarly, the phrase "if it is determined" or "if a described condition or event is detected" may be interpreted, depending on the context, to mean "upon determining" or "in response to determining" or "upon detecting a described condition or event" or "in response to detecting a described condition or event".

In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Moreover, various embodiments or examples and features of various embodiments or examples described in this specification can be combined and combined by one skilled in the art without being mutually inconsistent.

Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "a plurality" means at least two, e.g., two, three, etc., unless explicitly specified otherwise.

Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and alternate implementations are included within the scope of the embodiments of the present invention in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the embodiments of the present invention.

It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like. In addition, functional units in the embodiments of the present invention may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium.

The above disclosure is only for the purpose of illustrating the preferred embodiments of the present invention, and it is therefore to be understood that the invention is not limited by the scope of the appended claims.

Claims

1. An image processing method, comprising:

acquiring characteristic information of a target picture to be processed, wherein the characteristic information comprises a color characteristic, a main body area characteristic and a labeling characteristic;

calling a copyright gallery and a non-copyright gallery to identify the characteristic information of the target picture so as to confirm the type of the target picture, wherein the copyright gallery is associated with the non-copyright gallery; the copyright gallery comprises at least one copyright picture, characteristic information of each copyright picture, attribute information of each copyright picture and a non-copyright picture associated with each copyright picture; the copyright-free picture library comprises at least one non-copyright picture, characteristic information of each non-copyright picture and attribute information of each copyright-free picture; the fact that one copyright picture is associated with one non-copyright picture means that the two pictures are similar in cognition;

2. The method of claim 1, wherein the color features comprise a global hue vector and a dominant hue vector; the subject region features comprise at least one feature descriptor; the labeling feature comprises an effective label set, and if the effective label set is not empty, the effective label set is represented to contain at least one effective label for describing the meaning of the picture; if the effective label set is empty, indicating that the effective label set does not contain effective labels for describing the meanings of the pictures;

the color attribute comprises a global hue vector; the emotion attribute comprises an emotion phrase set, and if the emotion phrase set is not empty, the emotion phrase set is represented to contain at least one key phrase for describing picture emotion; if the emotion word group set is empty, indicating that the emotion word group set does not contain key words for describing picture emotion; the text attribute comprises a text label set, and if the text label set is not empty, the text label set is represented to contain at least one text label phrase for describing the meaning of the picture; and if the text label set is empty, the text label set does not contain the text label phrase for describing the picture meaning.

3. The method of claim 2, wherein the obtaining the feature information of the target picture to be processed comprises:

traversing the color value of each pixel point of the target picture;

4. The method of claim 2, wherein the obtaining the feature information of the target picture to be processed comprises:

5. The method of claim 2, wherein the obtaining the feature information of the target picture to be processed comprises:

creating an active tag set for the target picture;

if not, setting the value of the effective label set to be null;

6. The method of any one of claims 2-5, wherein the invoking the copyright gallery and the non-copyright gallery to authenticate feature information of the target picture to confirm the type of the target picture comprises:

if not, the authentication fails.

7. The method of claim 6, wherein the calculating of the first degree of match or the second degree of match comprises:

weighting the A1, the B1 and the C1 according to a preset weighting rule;

8. The method of claim 6, further comprising:

9. The method of claim 2, wherein the obtaining of the attribute information of the target picture comprises:

creating an emotion phrase set and a text label set for the target picture;

judging whether a target article to which the target image belongs is acquired;

10. The method as claimed in claim 9, wherein the performing similarity matching in the copyright gallery and the non-copyright gallery according to the attribute information of the target picture to obtain matching recommended content comprises:

and judging whether the non-copyright picture similar to the target picture exists in the non-copyright picture library according to the second similarity, and if so, acquiring the similar non-copyright picture to generate recommended content.

11. The method of claim 10, wherein the calculating of the first or second similarity comprises:

weighting the A2, the B2 and the C2 according to a preset weighting rule;

12. A picture processing apparatus, comprising:

the characteristic acquisition unit is used for acquiring characteristic information of a target picture to be processed, wherein the characteristic information comprises color characteristics, main body area characteristics and marking characteristics;

an authentication unit, configured to invoke a copyright library and a non-copyright library to authenticate feature information of the target picture to confirm the type of the target picture, wherein the copyright library is associated with the non-copyright library; the copyright gallery comprises at least one copyright picture, characteristic information of each copyright picture, attribute information of each copyright picture and a non-copyright picture associated with each copyright picture; the copyright-free picture library comprises at least one non-copyright picture, characteristic information of each non-copyright picture and attribute information of each copyright-free picture; the fact that one copyright picture is associated with one non-copyright picture means that the two pictures are similar in cognition;

the matching unit is used for performing similar matching on the copyright gallery and the non-copyright gallery according to the attribute information of the target picture to obtain matched recommended content;

13. A computer storage medium having one or more instructions stored thereon, the one or more instructions adapted to be loaded by a processor and to perform the picture processing method according to any of claims 1-11.

14. A service device, comprising:

a processor adapted to implement one or more instructions; and (c) a second step of,

a computer storage medium storing one or more instructions adapted to be loaded by the processor and to perform the picture processing method according to any of claims 1-11.