TW202137079A - Image similarity analyzing system - Google Patents

Abstract

An image similarity analyzing system comprising a trained first deep-learning module, a trained neural-network data processing module, a combination-and-learning unit, and a similarity analyzing unit. The first deep-learning module receives the image to generate an initial image representation. The neural-network data processing module receives the image rule data of the image under the specific image rule to generate an image rule representation. The combination-and-learning unit comprises a combination module and a trained second deep-learning module. The combination module combines the initial image representation and the image rule representation to generate an input data. The second deep-learning module receives the input data to generate a final image representation. The similarity analyzing unit compares the final image representation and a reference image representation. Thereby, the image rule of the IP territory can be included in the analyzing process to improve the weakness of image data comparison within the IP territory.

Description

Image approximation analysis system

The invention relates to an image similarity analysis system, in particular to an image similarity analysis system that uses deep learning to intelligently process image intellectual property rights data.

In the face of international technological competition and impact, the development of intellectual property rights has become an extremely important link in industrial upgrading. With the wave of knowledge economy sweeping the world, there is no doubt about the importance and value of intellectual property rights. However, with the emergence of new technologies and technologies, it has gradually triggered the future service trend of intellectual property rights.

In the past, intellectual property rights required a lot of manpower to be analyzed from the perspectives of technology, law, and commercial interests, and then generated strategies and behaviors beneficial to the right holders.

Among them, the image-related parts of intellectual property rights, such as trademark images, copyright images, or design images, are very labor-intensive, whether it is searching and comparing previous cases, and directly affects the scope of rights, The approval rate, the possibility of infringement and infringement, and the possibility of invalidity or invalidation, legally and commercially, will cause significant profits and losses for the enterprise.

Therefore, it is necessary to use artificial intelligence, which is becoming more and more mature, to solve the problems of labor-consuming, error and controversy, and time-consuming and low efficiency of intellectual property rights.

Therefore, the main purpose of the present invention is to provide an image similarity analysis system that uses deep learning to intelligently process image intellectual property data to solve the above-mentioned problems.

The purpose of the present invention is to provide an image similarity analysis system, which is used in the field of intellectual property rights with specific image rules to analyze the similarity of the image compared to the reference image. The image similarity analysis system includes a trained first deep learning module, a trained neural network data processing module, a combined learning unit, and an approximate analysis unit. The first deep learning module after training receives the image to generate an initial image representation. The trained neural network data processing module receives the graph specification information of the image under a specific image specification, and generates a graph specification representation based on the graph specification information. The combined learning unit includes a combined module and a second deep learning module after training. The combining module is used to combine the initial graph representation and graph specification representation to generate input information. The second deep learning module after training is used to receive input information to generate the final graph representation. The similarity analysis unit is used to compare the final image representation with the reference image representation of the reference image to determine the similarity between the image and the reference image.

In order to achieve at least one of the advantages or other advantages, an embodiment of the present invention provides an image approximation analysis system, which is characterized in that the neural network data processing module uses one hot encoding (One Hot Encode) generates the specification representation of the graph.

In order to achieve at least one of the advantages or other advantages, an embodiment of the present invention provides an image approximation analysis system, which is characterized in that the image specification information uses images corresponding to the specific image specification. A classification database, a knowledge map library with the specific image specification, or a quantitative specification rule corresponding to the specific image specification.

In order to achieve at least one of the advantages or other advantages, an embodiment of the present invention provides an image approximation analysis system, which is characterized in that the combined learning unit combines the initial graph representation with the graph specification representation The method of combining to generate the input information is to adopt Use vector to merge directly.

In order to achieve at least one of the advantages or other advantages, an embodiment of the present invention provides an image approximation analysis system, which is characterized in that the graph rule representation has the same dimension as the initial graph representation. The combined learning unit uses the graph specification representation as a weight to combine the initial graph representation and the graph specification representation.

In order to achieve at least one of the advantages or other advantages, an embodiment of the present invention provides an image approximation analysis system, which is characterized in that the trained first deep learning module and the trained The second deep learning module is selected from at least one of the Convolutional Neural Network (CNN) group consisting of LeNet, AlexNet, VGG, GoogleLeNet, and ResNet.

In order to achieve at least one of the advantages or other advantages, an embodiment of the present invention provides an image approximation analysis system, which is characterized in that the trained first deep learning module is used to receive the reference Image to generate an initial reference image representation, and the trained neural network data processing module is used to receive reference image specification information of the reference image under the specific image specification, and based on the reference image The specification information generates a reference image specification representation, the combining module is used to combine the initial reference image representation and the reference image specification representation to generate reference input information, and the trained second deep learning module is used to receive all The reference input information is used to generate the final reference image representation.

In order to achieve at least one of the advantages or other advantages, an embodiment of the present invention provides an image approximation analysis system, which is characterized in that the dimension of the final image representation is the same as the dimension of the reference image representation.

In order to achieve at least one of the advantages or other advantages, an embodiment of the present invention An image approximation analysis system is proposed, wherein the approximation analysis unit compares the final image representation with the reference image representation to generate a geometric distance in a multi-dimensional space, and judges based on the geometric distance The degree of similarity between the image and the reference image.

In order to achieve at least one of the advantages or other advantages, an embodiment of the present invention provides an image similarity analysis system, wherein the similarity analysis unit sets at least one threshold, and compares the geometry The relationship between the distance and the threshold value is used to determine that the image is similar to or the same as the reference image.

Therefore, the image similarity analysis system for image intellectual property rights data provided by the present invention can effectively incorporate the existing image specifications in the field of intellectual property rights, and solve the problem of image data (such as trademark images) in the field of intellectual property rights. , Copyright images, or design images, etc.) are labor-intensive, errors and disputes, time-consuming and low-efficiency.

The above description is only an overview of the technical solution of the present invention. In order to understand the technical means of the present invention more clearly, it can be implemented in accordance with the content of the description, and in order to make the above and other objectives, features and advantages of the present invention more obvious and understandable. In the following, the preferred embodiments are cited in conjunction with the drawings, and the detailed description is as follows.

100, 200, 300: graph representation generation system

120: The first deep learning module

140: Neural Network Data Processing Module

160: Combined learning unit

I: Image

y: initial graph representation

Ir: Graph specification information

z: Graph specification representation

20: Knowledge Graph Library

162,320: Combining modules

164: The second deep learning module

a: input information

b: final graph representation

280: Training Module

282: Comparison image generation unit

284: Optimization Unit

I’: Compare images

340: Deep Learning Module

400: Image approximation analysis system

480: Approximation analysis unit

I0: Reference image

c: Reference image characterization

The included drawings are used to provide a further understanding of the embodiments of the present application, which constitute a part of the specification, are used to illustrate the embodiments of the present application, and together with the text description, explain the principle of the present application. Obviously, the drawings in the following description are only some embodiments of the application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative labor. In the attached picture:

Fig. 1 is a schematic diagram of an embodiment of a graph characterization generating system of the present invention;

Fig. 2 is a schematic diagram of another embodiment of the graph characterization generating system of the present invention;

Fig. 3 is a flowchart of an embodiment of a method for generating a graph characterization of the present invention;

4 is a flowchart of another embodiment of the method for generating a graph characterization according to the present invention;

FIG. 5 is a schematic diagram illustrating an embodiment of a smart module according to the present invention; and

Fig. 6 is a schematic diagram of an embodiment of the image approximation analysis system of the present invention.

The specific structure and functional details disclosed herein are only representative, and are used for the purpose of describing exemplary embodiments of the present invention. However, the present invention can be embodied in many alternative forms, and should not be construed as being limited only to the embodiments set forth herein.

In the description of the present invention, it should be understood that the terms "center", "lateral", "upper", "lower", "left", "right", "vertical", "horizontal", "top", The directions or positions indicated by "bottom", "inner", and "outer" are based on the positions or positions shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, and are not indicative or implied. The device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the characterization defined with "first" and "second" may explicitly or implicitly include one or more of the characterizations. In the description of the present invention, unless otherwise specified, "plurality" means two or more. In addition, the term "including" and any variations thereof is intended to cover non-exclusive inclusion.

In the description of the present invention, it should be noted that, unless otherwise clearly defined and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, they can be The fixed connection can also be a detachable connection or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, and it can be a connection between two components. For those of ordinary skill in the art, the specific meanings of the above-mentioned terms in the present invention can be understood in specific situations.

The terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments. Unless the context clearly dictates otherwise, the singular forms "a" and "one" used herein are also intended to include the plural. It should also be understood that the terms "comprising" and/or "comprising" used herein specify the existence of the stated features, integers, steps, operations, units and/or elements, and do not exclude the existence or addition of one or more Other characterizations, integers, steps, operations, units, elements, and/or combinations thereof.

Fig. 1 is a schematic diagram of an embodiment of the graph characterization generating system of the present invention. The image representation generation system 100 is used in the field of intellectual property rights with specific image specifications, and is used to transform the image into a domain-adaptable image representation. The aforementioned images may be images in the field of intellectual property rights such as trademark graphics and image designs. The aforementioned specific image specification can be a trademark graphic classification or an industrial design classification, such as the Vienna Classification (a classification established by the Vienna Agreement for trademarks composed of or with graphic elements) or Locarno (Locarno) classification (an international classification for industrial design registration established by the Locarno Agreement).

As shown in the figure, the graph representation generation system 100 includes a first deep learning module 120, a neural network data processing module 140, and a combined learning unit 160.

The first deep learning module 120 is used to receive the image I to generate an initial image representation (representation) y. In an embodiment, the first deep learning module 120 may be selected from the convolutional neural network family consisting of LeNet, AlexNet, VGG, GoogleLeNet, and ResNet At least one in the group.

The neural network data processing module 140 is used to receive the image specification information Ir of the image I under the specific image specification, and generate the image specification representation z according to the image specification information Ir.

In one embodiment, as shown in the figure, a knowledge graph 20 with the specific image specification can be used to analyze the image I to automatically generate the graph specification information Ir. But it is not limited to this. In one embodiment, the image I can be classified by using the image classification database corresponding to the specific image specification to generate image specification information Ir. This sorting operation can be carried out automatically by a calculator or can be assisted by manpower. In one embodiment, the specific image specification may be quantified to generate a quantification specification law, and the quantification specification law may be used to analyze the image I to generate image specification information Ir. For example, you can set the reference image and analyze whether the pixels on the image I and the reference image are the same, and when the ratio of the same pixels exceeds a preset value, the two images are considered to be similar and belong to the same Classification.

In one embodiment, the neural network data processing module 140 may be a simple neural network with a single hidden layer or other shallow neural networks (for example, the number of hidden layers is less than 10), and the number of hidden layers is It is significantly less than the aforementioned first deep learning module 120 to reduce the cost and simplify the complexity of the architecture. However, the present invention is not limited to this. If the image specification is too complex, in order to improve the accuracy of the judgment, in one embodiment, the neural network data processing module 140 may also be a deep learning model with a deep neural network. Group.

In one embodiment, the neural network data processing module 140 uses one-hot encoding to generate the graph specification representation z. The dimension of the graph specification representation z output by the neural network data processing module 140 can be adjusted according to user requirements and the actual training and operating conditions of the graph representation generation system.

The combined learning unit 160 includes a combined module 162 and a second deep learning module 164. The combining module 162 is used to combine the initial graph representation y and the graph specification representation z to generate input information a. In one embodiment, the combination learning unit 160 combines the initial graph representation y with the graph specification representation z to generate the input information a by directly combining vectors. However, it is not limited to this. In one embodiment, the combining module 162 of the combined learning unit 160 combines the initial graph representation y with the graph specification representation z to generate the input information a. The combining method may also use the graph specification representation z as the weight. Merged with the initial graph characterization y. The direct merging of vectors is not limited by the dimensions of the initial graph representation y and graph norm representation z, but it will generate input information a with more dimensions. Using the graph specification representation z as the weight and the initial graph representation y can effectively reduce the dimensionality of the input information a, but the initial graph representation y and the graph specification representation z of the same dimension are required.

The second deep learning module 164 is used to receive the input information a to generate the final graph representation b. In an embodiment, the second deep learning module is selected from at least one of the convolutional neural network group consisting of LeNet, AlexNet, VGG, GoogleLeNet, and ResNet.

Since the input information a received by the second deep learning module 164 includes the graph specification representation z corresponding to the graph specification information Ir, the final graph representation b generated by the second deep learning module 164 can be effectively incorporated into the existing intellectual property field. The image specification of, makes the final image representation b output of the image representation generation system 100 closer to the actual judgment analysis result in the field of intellectual property rights. Secondly, the present invention uses a relatively simple neural network data processing module 140 to process image specifications, which on the one hand helps to reduce costs, and on the other hand, it also helps to increase the computing speed.

The following uses a 2D trademark graphic as an example to illustrate this embodiment. Assume that the image I input to the characterization generating system 100 shows a 2D trademark containing 16x16 gray-scale pixels, and the specific image specification is the eighth edition of the trademark graphic classification of the Vienna Agreement, which has 29 categories. Fake Suppose that the image specification information generated by the image I according to the trademark image classification method is Ir={Ir1,Ir2…Ir29}, Ir1, Ir2…Ir29 is a two-dimensional number 0 or 1 representing each category of the image I and the trademark image classification method Relevance. In other words, if it belongs to this category, fill in 1; if not, fill in 0. In other words, this image specification information Ir is the classification result of this image I according to the trademark image classification method.

As mentioned above, the image I input to the first deep learning unit 120 can be expressed as x={x1,x2...x256}, where x1,x2...x256 represents the gray scale of each pixel; after passing through the first deep learning module 120 The generated initial graph represents y={y1,y2...ym}. According to the graph specification information Ir, the graph specification representation z={z1, z2...zi} generated by one-hot encoding. The aforementioned m and i represent the dimensions of the initial graph representation y and the graph specification representation z, and both can be adjusted according to the actual needs of the user.

If the combination module 162 combines the initial graph representation y with the graph specification representation z to generate input information, the combining method is to use vectors to directly merge the input information a={y1,y2...ym,z1,z2...zi}. If the combination module 162 combines the initial graph representation y with the graph specification representation z to generate the input information a, the combining method is to use the graph specification representation z as the weight to merge with the initial graph representation y, and the dimension i of the graph specification representation z needs to be the same as the initial The dimension m of the graph representation y is the same, and the input information a={y1z1,y2z2...ymzm}. The second deep learning module 164 receives the input information a, and generates according to the final graph representation b={b1, b2...bn}. The aforementioned n represents the dimension of the final graph representation b, which can be adjusted according to actual needs.

The final image representation b will contain the information related to the trademark image classification of this image I. Therefore, using the final graph representation b generated by the graph representation generation system 100 of this embodiment as an object for previous search and comparison processing can be effectively incorporated into the existing image specifications in the field of intellectual property rights and improve the accuracy of judgment. Effectively solve the problems of labor-consuming, error and dispute, time-consuming and low efficiency in the processing of image data in the field of intellectual property rights.

FIG. 2 is a schematic diagram of another embodiment of the graph characterization generating system 200 of the present invention. Mutually Compared with the diagram of FIG. 1, the generating system 100 is represented. The graph representation generation system 200 of this embodiment has an automatic training function, and can directly use the final graph representation b generated by encoding to reversely correct the first deep learning module 120, the neural network data processing module 140, and the second deep learning module 164 parameters.

As shown in the figure, the graph representation generation system 200 of this embodiment includes a training module 280 in addition to the first deep learning module 120, the neural network data processing module 140, and the combined learning unit 160. The training module 280 includes a comparison image generation unit 282 and an optimization unit 284. The comparison image generating unit 282 receives the final graph representation b generated by the second deep learning module 164, and generates the final graph representation according to the first deep learning module 120, the neural network data processing module 140, and the combined learning unit 160 The encoding method of b is to decode and restore the final image characterization b to generate a comparison image I'corresponding to the image I.

The optimization unit 284 receives the compared image I', and calculates the loss function between the compared image I'and the original image I to optimize the first parameter and neural network data of the first deep learning module 120 The second parameter of the processing module 140 and the third parameter of the second deep learning module 164. In other words, the optimization unit 284 of the training module 280 will modify the first parameter, the second parameter, and the third parameter with the goal of reducing the loss function. In an embodiment, the aforementioned loss function may be the mean square error (MSE) of the gray levels of all corresponding pixels of the comparison image I'and the original image I. In an embodiment, the aforementioned loss function may be the mean absolute error (MAE) of the gray levels of all corresponding pixels of the comparison image I'and the original image I. However, the present invention is not limited to this. Any loss function suitable for image comparison, such as Huber loss function and Log-Cosh loss function, can be applied to the present invention.

Through the operation of the aforementioned training module 280, the image representation generation system 200 of this embodiment can automatically restore the final image representation b generated by the encoding to the comparison image I'for the training process to optimize The first parameter of the first deep learning module 120, the second parameter of the neural network data processing module 140, and the third parameter of the second deep learning module 164 are changed without manual intervention.

Fig. 3 is a flowchart of an embodiment of a method for generating a graph characterization of the present invention. This graph representation generation method is used in the field of intellectual property rights with specific image specifications to transform the image into a graph representation with domain adaptability. This graph characterization generation method can be executed using the graph characterization generation system 100 shown in FIG. 1.

As shown in the figure, this graph characterization generation method includes the following steps.

Please refer to FIG. 1 together. First, in step S120, the image I is provided to the first deep learning model to generate the initial image representation y. This step can be performed by the first deep learning module 120 in FIG. 1. In an embodiment, the first deep learning model is provided by at least one selected from the convolutional neural network group consisting of LeNet, AlexNet, VGG, GoogleLeNet, and ResNet.

Subsequently, in step S140, the graph specification information Ir of the image I under the specific image specification is provided to the neural network model to generate the graph specification representation z. This step can be performed by the neural network data processing module 140 in FIG. 1. In one embodiment, this step can use one-hot encoding to generate a graph specification representation z based on graph specification information Ir.

In one embodiment, the image specification information Ir is generated by analyzing the image I using the image classification database corresponding to the specific image specification. In one embodiment, the image specification information Ir is generated by analyzing the image I using a knowledge graph library with the specific image specification. In one embodiment, the image specification information Ir is generated by analyzing the image I using a quantization specification rule generated after quantization of the specific image specification.

Next, in step S160, the initial graph representation y is combined with the graph specification representation z To generate input information a. This step can be performed by the combining module 162 in FIG. 1. In one embodiment, this step is to directly merge the vector of the initial graph representation y and the graph specification representation z. In one embodiment, when the dimension of the graph rule representation y and the initial graph representation z are the same, this step uses the graph specification representation z as a weight to merge with the initial graph representation y.

Finally, in step S180, the input information a is provided to the second deep learning model to generate the final graph representation b. This step can be performed by the second deep learning module 164 in FIG. 1. In an embodiment, the second deep learning model is provided by at least one selected from the convolutional neural network group consisting of LeNet, AlexNet, VGG, GoogleLeNet, and ResNet.

Fig. 4 is a flowchart of another embodiment of a method for generating a graph characterization of the present invention. Compared to the graph in FIG. 3, the generation method is characterized. The graph representation generation method of this embodiment has a training step, and the final graph representation b generated by encoding can be directly used to reversely correct the parameters of the first deep learning model, the neural network model, and the second deep learning model. In an embodiment, this graph characterization generation method can be executed using the graph characterization generation system 200 shown in FIG. 2.

Following step S180 of FIG. 3, as shown in the figure, after the step of generating the final image representation b, this embodiment further includes a step S192 of comparing image generation with a step S194 of parameter optimization, which can automatically correct the first step used in step S120. A deep learning model, the neural network model used in step S140, and the parameters of the second deep learning model used in step S180.

The comparison image generation step S192 is to generate the encoding method of the final image representation b according to the steps S120 to S180, decode and restore the final image representation b, and generate a comparison image I'corresponding to the original image I. Please also refer to FIG. 2. In one embodiment, this step can be performed by the comparison image generating unit 282 of the training module 280.

The parameter optimization step S194 is based on comparing the loss between the image I’ and the original image I. The parameters of the first deep learning model used in step S120, the neural network model used in step S140, and the second deep learning model used in step S180 are optimized according to the loss function. In one embodiment, this step can be performed by the optimization unit 284 of the training module 280. The training module 280 corrects the parameters with the goal of reducing the value of the loss function.

FIG. 5 is a schematic diagram illustrating an embodiment of a smart module according to the present invention. This figure represents that the smart module 300 is used in the field of intellectual property rights with specific image specifications, and is used to transform the image I into a figure representation with domain adaptability. This figure represents that the smart module 300 roughly corresponds to the combined learning unit 160 in FIG. 1.

As shown in the figure, this figure represents that the smart module 300 includes a combining module 320 and a deep learning module 340. The combination module 320 is used to receive the initial image representation y corresponding to the image I and the image specification representation z corresponding to the image I under a specific image specification, and combine the initial image representation y and the image specification representation z to generate input information a. In one embodiment, the graph specification representation z is generated using one-hot encoding.

In one embodiment, the combining module 320 combines the initial graph representation y with the graph specification representation z to generate the input information a by directly combining vectors. But it is not limited to this. In one embodiment, the combining module 320 combines the initial graph representation y with the graph specification representation z to generate the input information a. The combining method may also use the graph specification representation z as a weight to combine with the initial graph representation y. For details of the initial graph representation y and graph specification representation z, please refer to the embodiment of FIG. 1 in this case, and will not be repeated here.

The deep learning module 340 receives the input information a generated by the combining module 320 to generate the final graph representation b. In one embodiment, the deep learning module 340 is at least one selected from the convolutional neural network group consisting of LeNet, AlexNet, VGG, GoogleLeNet, and ResNet. In one embodiment, for the deep learning module 340, please refer to the second part in Figure 1 of this case. The deep learning module 164. Regarding the training process of the deep learning module 340, please refer to the training process of the second deep learning module 164 in FIG. 2 of this case, which is not repeated here.

This figure represents that the smart module 300 can be software, hardware, or a combination of software and hardware. In actual use, it can be combined with the user's existing deep learning module and neural network module to form the graph representation generation system 100 shown in FIG. 1 to generate domain-adaptable graph representations for user analysis and use. For example, this figure represents that the smart module 300 can be achieved by a general programming language or other existing programs, and can be installed in a known computer usable medium; this figure represents that the smart module 300 can use an integrated circuit The manufacturing process is converted to hardware implementation; this figure shows that the smart module 300 can also be implemented by general programming languages or other existing programs, and some modules are converted to hardware implementation by using integrated circuit manufacturing process.

FIG. 6 is a schematic diagram of an embodiment of the image approximation analysis system 400 of the present invention. The image similarity analysis system 400 is used in the field of intellectual property rights with specific image specifications to analyze the similarity of the image I compared to the reference image I0.

The image similarity analysis system 400 includes a trained first deep learning module 120, a trained neural network data processing module 140, a combined learning unit 160, and an approximate analysis unit 480. Wherein, the combined learning unit 160 includes a combined module 162 and a second deep learning module 164 after training.

The trained first deep learning module 120 is used to receive the image I to generate the initial image representation y. In one embodiment, the trained first deep learning module 120 is selected from at least one of the convolutional neural network group consisting of LeNet, AlexNet, VGG, GoogleLeNet, and ResNet. The trained neural network data processing module 140 is used to receive the image specification information Ir of the image I under a specific image specification, and generate the image specification representation z accordingly.

In one embodiment, as shown in the figure, a knowledge graph 20 with the specific image specification can be used to analyze the image I to automatically generate the graph specification information Ir. But it is not limited to this. In one embodiment, the image I can be classified by using the image classification database corresponding to the specific image specification to generate image specification information Ir. This sorting operation can be carried out automatically by a calculator or can be assisted by manpower. In one embodiment, the specific image specification may be quantified to generate a quantification specification law, and the quantification specification law may be used to analyze the image I to generate image specification information Ir.

In one embodiment, the trained neural network data processing module 140 can be a simple neural network with a single hidden layer or other shallow neural networks (for example, the number of hidden layers is less than 10), and the number of hidden layers is obvious The number of the first deep learning module 120 after the training is less than that of the first deep learning module 120, so as to reduce the cost and simplify the complexity of the architecture. In one embodiment, the trained neural network data processing module 140 uses one-hot encoding to generate the graph specification representation z. The dimension of the graph specification representation z can be adjusted according to user requirements and the actual training and operating conditions of the approximation analysis system 400.

The combined learning unit 160 includes a combined module 162 and a second deep learning module 164 after training. The combining module 162 is used to combine the initial graph representation y and the graph specification representation z to generate input information a. In one embodiment, the combination learning unit 160 combines the initial graph representation y with the graph specification representation z to generate the input information a by directly combining vectors. However, it is not limited to this. In one embodiment, the combining module 162 of the combined learning unit 160 combines the initial graph representation y with the graph specification representation z to generate the input information a. The combining method may also use the graph specification representation z as the weight. Merged with the initial graph characterization y. The trained second deep learning module 164 is used to receive the input information a to generate the final graph representation b. The final image representation b will contain information related to the specific image specification of the image I (ie, image specification information Ir). In one embodiment, the second deep learning module is selected from the convolutional neural network group consisting of LeNet, AlexNet, VGG, GoogleLeNet, and ResNet At least one of them.

Regarding the training process of the aforementioned first deep learning module 120, neural network data processing module 140, and second deep learning module 164, for example, the architecture shown in FIG. 2 can be used for training.

The similarity analysis unit 480 is used to compare the final image representation b with the reference image representation c of the reference image I0 to determine the similarity between the image I and the reference image I0. In an embodiment, the reference image representation c may be generated by processing the reference image I0 through the aforementioned training of the first deep learning module 120, the trained neural network data processing module 140, and the combined learning unit 160. . That is to say, the reference image I0 and the image I may undergo the same processing to respectively generate the reference image representation c and the final image representation b for analysis and comparison of similarity.

In one embodiment, the aforementioned reference graph representation c and the final graph representation b may have the same dimension to facilitate the analysis and comparison of the two. The similarity analysis unit 480 may compare the reference image representation c and the final image representation b to generate a geometric distance, and determine the similarity between the image I and the reference image I0 according to the geometric distance. That is, the final graph representation b={b1,b2...bn} and the reference graph representation c={c1,c2..cn} are understood as two points in the n-dimensional representation space vector, and these two points are calculated Here, the geometric distance in the n-dimensional representation space is used to determine the degree of similarity between the image I and the reference image I0.

In one embodiment, considering the particularity of the field of intellectual property rights, the similarity analysis unit 480 may set at least one threshold, and by comparing the geometric distance with the threshold, determine that the image is similar or the same In the reference image. For example, it can be assumed that all publicly registered trademark graphics satisfy the similarity analysis, and the geometric distances between the final graphical representations corresponding to these publicly registered trademark graphics can be counted. The minimum of these geometric distances can be set as the threshold. . In one embodiment, considering the particularity of the field of intellectual property rights, a The same judgment threshold and an approximate judgment threshold. When the geometric distance is less than the same judgment threshold, the judgment image I is the same as the reference image I0, and when the geometric distance is less than the approximate judgment threshold, the judgment image I is similar to the reference image I0. This threshold can be set by analyzing the data in the intellectual property rights database.

If the image similarity analysis system 400 is applied to the trademark field, for example, all registered and public trademark graphics in the trademark database can be input into the first half of the similarity analysis system 400 (or the image representation generated after training) The system 100) to obtain the graphic representations corresponding to these trademark graphics, and these graphic representations can be used as the reference graphic representation c in this embodiment. In this way, the user can use the image similarity analysis system 400 to first confirm whether there are similar or identical trademarks in the registered trademarks before applying for trademarks, so as to evaluate the possibility of approval, and then consider whether the trademark graphics need to be adjusted. design.

In summary, the image similarity analysis system provided by the present invention can effectively incorporate the existing image specifications in the field of intellectual property rights, and solve the problem of image data (such as trademark images, copyright images, etc.) in the field of intellectual property rights. Or design images, etc.) comparison processing is labor-intensive, errors and disputes are large, time-consuming and low-efficiency issues.

The above are only the preferred embodiments of the present invention and do not limit the present invention in any form. Although the present invention has been disclosed as the preferred embodiments, it is not intended to limit the present invention. Anyone familiar with the profession Those skilled in the art, without departing from the scope of the technical solution of the present invention, can use the methods and technical content disclosed above to make slight changes or modifications into equivalent embodiments with equivalent changes, but any content that does not depart from the technical solution of the present invention is based on Any simple modifications, equivalent changes and modifications made to the above embodiments by the technical essence of the present invention still fall within the scope of the technical solutions of the present invention.

400: Image approximation analysis system

120: The first deep learning module

140: Neural Network Data Processing Module

160: Combined learning unit

I: Image

y: initial graph representation

Ir: Graph specification information

z: Graph specification representation

162: Combination Module

164: The second deep learning module

a: input information

b: final graph representation

480: Approximation analysis unit

I0: Reference image

c: Reference image characterization

Claims (11)

An image similarity analysis system used in the field of intellectual property rights with a specific image rule to analyze the similarity of an image compared to a reference image. The image similarity analysis system includes: A trained first deep learning module for receiving the image to generate an initial image representation; A trained neural network data processing module for receiving a graph specification information of the image under the specific image specification, and generating a graph specification representation according to the graph specification information; A combined learning unit includes a combined module and a trained second deep learning module. The combined module is used to combine the initial graph representation and the graph specification representation to generate an input information, so The second deep learning module after training is used to receive the input information to generate a final graph representation; and An approximate degree analysis unit for comparing the final image representation with one of the reference image representations to determine the similarity between the image and the reference image. The image approximation analysis system according to claim 1, wherein the trained neural network data processing module uses One Hot Encode to generate the graph specification representation. The image similarity analysis system according to claim 1, wherein the graph specification information uses a graph classification database corresponding to the specific image specification, and a knowledge graph library having the specific image specification , Or generated corresponding to one of the specific image specifications. The image approximation analysis system according to claim 1, wherein the combination method of the combination learning unit combining the initial graph representation and the graph specification representation to generate the input information is direct combination of vectors. The image approximation analysis system according to claim 1, wherein the dimension of the graph rule representation is the same as the dimension of the initial graph representation. The image approximation analysis system according to claim 5, wherein the combination learning unit uses the graph specification representation as a weight to combine the initial graph representation and the graph specification representation. The image approximation analysis system according to claim 1, wherein the first deep learning module after training and the second deep learning module after training are selected from LeNet, At least one of the convolutional neural network (Convolutional Neural Network; CNN) group composed of AlexNet, VGG, GoogleLeNet, and ResNet. The image approximation analysis system according to claim 1, wherein the trained first deep learning module is used to receive the reference image to generate an initial reference image representation, and the trained nerve The network data processing module is used for receiving one of the reference image specification information of the reference image under the specific image specification, and generating a reference image specification representation according to the reference image specification information, and the combination module is used together The initial reference image representation and the reference image specification representation are combined to generate a reference input information, and the trained second deep learning module is used to receive the reference input information to generate the reference image representation. The image approximation analysis system according to claim 1, wherein the dimension of the final graph representation is the same as the dimension of the reference graph representation. The image approximation analysis system according to claim 1, wherein the approximation analysis unit compares the final image representation with the reference image representation to generate a geometric distance in a multi-dimensional space, and based on the The geometric distance determines the degree of similarity between the image and the reference image. The image similarity analysis system according to claim 10, wherein the similarity analysis unit sets at least one threshold, and judges the image by comparing the relationship between the geometric distance and the threshold. Whether it is similar to the reference image.

Images