WO2017136060A1

WO2017136060A1 - Improving distance metric learning with n-pair loss

Info

Publication number: WO2017136060A1
Application number: PCT/US2016/067946
Authority: WO
Inventors: Kihyuk SOHN
Original assignee: NEC Laboratories America Inc
Current assignee: NEC Laboratories America Inc
Priority date: 2016-02-04
Filing date: 2016-12-21
Publication date: 2017-08-10
Anticipated expiration: 2018-08-04
Also published as: US20170228641A1; DE112016006360T5; JP2019509551A; US10565496B2

Abstract

A method includes receiving N pairs of training examples and class labels therefor. Each pair includes a respective anchor example, and a respective non-anchor example capable of being a positive or a negative training example. The method further includes extracting features of the pairs by applying a DHCNN, and calculating, for each pair based on the features, a respective similarly measure between the respective anchor and non-anchor example. The method additionally includes calculating a similarity score based on the respective similarity measure for each pair. The score represents similarities between all anchor points and positive training examples in the pairs relative to similarities between all anchor points and negative training examples in the pairs. The method further includes maximizing the similarity score for the anchor example for each pair to pull together the training examples from a same class while pushing apart the training examples from different classes.

Description

IMPROVING DISTANCE METRIC LEARNING WITH N-PAIR LOSS

RELATED APPLICATION INFORMATION

[0001 ] This application claims priority to CIS. Provisional Pat. App. Ser, No. 62/2 1 ,025 filed on February 4, 2016, incorporated herein by reference in its entirety.

BACKGROUND

Technical Field

[0002] The present invention relates t computer learning and more particularly- to improving distance metric learning with N-pair loss.

Description of the Related Art

[0003] Deep metric learning has been tackled in many ways but most notably, -conirastive loss and triplet loss have been .used for training objectives, of dee learning. Previous approaches considered pair/wise relationship between two different classes and suffered from slo convergence to an unsatisfactory local n nrnurn. Thus, there is a need for improved metric learning,

SUMMARY

[0004] According to an aspect of the present invention, a computer-implemented method is provided. The method includes receiving, by a processor, N pairs of training examples and class labels tor th training examples that correspond to a- ^'plurality of classes. Each of the N pairs includes a respective anchor example and further includes a respecti ve non-anchor example capable of being a positive training example or a negative training example. The method further includes extracting., by the processor., features of the N pairs by applying a deep convolutional neural network to the N pairs and to the class labels. The method also includes calculating, by the processor for each of the N pairs^' ased OK ^'the features, a respective similarly measure between the respective ancho example and the respective non-anchor example. The method additionally includes calculating, by the processor, a similarity score based o the respective similarity measure for each of the pairs. The similarity score represents one or more similarities between all anchor points and all positive training examples in the N pairs relative to one or more similarities betwee all of the anchor points and all negati ve training examples in the H pairs. The method further includes maximizing, by the processor, the similarity score for the respective anchor example for each of the N pairs to pull together in a distribution space the training examples from a same one of the pluralit of classes while pushing apart in the distribution space the training examples from different ones of the plurality of classes.

[0005] According to another aspect, of the present invention, a system is provided. The system includes a processor. The processor is configured to receive N pairs of training examples and class labels for the training examples that correspond to a plurality o classes. Each of the Ή pairs includes a respective anchor example and further includes a respective non-anchor example capable of being a positi ve training exampl e or a negati ve training example. The processor is further configured to extract features of the pairs by applying a deep convolutional neural network to the N pairs and to the class labels. The processor is also configured to calculate, for each of the N pairs based on the features, a respective similarly measure between the respective anchor example and the respective no»~a»chor example. The processor is additionally configured to calculate a similarity score based on the respective similarity measure for each of the N pairs. The similarity score represents one or more similarities between ail anchor points and all positive training examples in the N pairs relative to one or more^■similarities between all of the anchor points and all negative training examples in the pairs, The processor is further configured to maximize the similarity score for the respective anchor exaaiple for each of the N pairs to pull together in a distribution space the training examples from a same one of the plurality of classes while pushing apart in the distribution space the training examples from different ones of the plurality of classes.

[0006] These and othe features and advantages will become apparent from the following detailed description of illustrative embodiments thereof, which is to e read in connection with the accompanying drawings:

BRIEF DESCRIPTION OF DRAWINGS

[0007] The disclosure will provide details in the following description of preferred embodiments with reference to the following figures wherein:

[0008] FIG, I shows a block diagram of an exemplary processing system 100 to which the present invention ma be applied, in accordance with^' an embodiment of the present invention;

[0009] FIG. 2 shows an exemplary environment 200 to whic the present invention can be applied, in accordance with an embodiment of the present invention ; [0010] FIG. 3 shows a high-level block/flow diagram of an exemplary system/method

300 for deep metric learning with N-pair loss, in accordance with ars .embodiment of the present invention;

[0011] FIG. 4 further shows step 310 of method 300 of FIG. 3, in accordance with an embodiment of the present invention. In particular, FIG. 4 further shows distance metric learning with N-pair loss 400, in accordance with an embodiment of the present invention;

[0012] FIG. 5 is a diagram graphically showing the N-pair loss 400 of FIG. 4 in accordance with an embodiment of the present invention versus a conventional triplet loss 599 id accordance with the prior art; and

[0013] FIGs. 6-8 show a flow diagram of a method 600 for deep metric learning with N-pair loss, in accordance with an embodiment of the present invention,

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

[0014] The present invention is directed to improving distance metric learning with N- pair loss.

[0015] The present invention solves the tundameiital machine learning problem of distance metric learning when the num e of output classes is extremely large, the total nomber of output classes is unknown or the distribution of output classes is variable over^¬ time using deep learning.

[0016] In an embodiment, and in contrast to prior ait approaches, the present invention considers N pairs of examples from N different classes at once. [0017] ί» an embodiment, the present invention introduces a new objective function for deep metric learning. The objective^' function . allows faster convergence to better local optimum,

[0018] The present invention provides an N-patr loss for deep metric learning. The present invention allows training of deep neural networks such that it trains to pull examples from the same class together while pushiag those from different classes apart. The present invention pushes not just one negative example at each update but N - 1 negative examples from all. different classes based on their relative distances to the reference example.

[0019] FIG. 1 shows a block diagram of an exemplary processing system 100 to which the invention principles may be applied, i accordance with an embodiment of the present invention. The processing system 100 includes at least one processor (CPU) 104 operatively coupled to other components via a system bus 10:2. A cache .106, a Read Only Memory (ROM) 108, a Random Access Memory (RAM) 1 10, an input output (I/O) adapter 120, a sound adapter 130, a network adapter 140, a user interface adapter 150, and a display adapter 160, are operatively coupled to the system bus 102.

[0020] A first storage device 122 and a second storage device 124 are operatively coupled to system bus 102 by the I/O. adapter 1 0. The storage devices 122 aid 124 can be any of a disk storage device (e.g., a magnetic or optical disk storage device), a solid state magnetic device, and so forth. The storage devices 122 and 124 can be the same type of storage device or different types of storage devices. [0021] A speaker 132 is operati vely coupled to system bus 102 by the sound adapter 130, A transceiver 1 2 is operativel coupled to system bits 102 b network adapter 1 0, A display device 162 is operaitvely eoispled to system bus 102 by display adapter 160.

[0022] A. first user input device 152, a second user input device 154, and a third user input device .156 are operative!}' coupled .to sy stem bus 102 by user interface adapter 150. The user input devices 152, 154, and 156 can be any of a keyboard, a mouse, a keypad, an image capture device, a motion sensing device, a microphone, a device incorporating the functionality of at least two of the preceding devices, and so forth.. Of coarse, other types of input devices can also be used, while maintaining the spirit of the present invention. The user input devices 152, 154, and 156 can be the same ^'type of user input device or different types of user input devices. The use input devices 152, 154, and 156 are used to input and output information ^'to and from system 100.

[0023] Of course, the processing system 100 may also include oilier elements (not shown), as readily contemplated by one of skill in the art, as well as omit certain elements. For example, various other input devices and/or output devices can be included in processing system 100, depending upon the particular implementation of the same, as readily understood by one of ordinary skill in the art. For example, various types of wireless and/or wired input and/or output devices can be used. Moreover, additional processors, controllers, memories, and so forth, in various configurations can also be utilized as readily appreciated by one of ordinary skill in the art. These and other variations of the processing system 100 are readily contemplated, by one of ordinary skill in the art given the teachings of the present invention provided herein. [0024] Moreover, it is to be appreciated that environment 200 described below with respect to FIG. 2 is an environment tor implementing respective embodiments of the present invention. Part or -all .of processing system 100 may be implem nted, in one or more of the elements of environment 200,

[0O25J Further, it is to be appreciated that processing system Ϊ00 may perform at least part of the method described herein including, for example , at least part of method 300 of FIG. 3 and/or at least part of method 400 of FIG. 4 and or at least part of method 600 of FlGs. 6-8. Similarly, pari or all of environment 200 may be used to perform at least part of method 300 of FIG. 3 and/or at least part of method 400 of FIG. 4 and'or at least part of method 600 of FIGs . 6-8.

[0026] FIG. 2 shows an exemplary environment 200 to which the present invention can b applied, in accordance with an embodiment of the present invention. The environmen 200 is representative of a computer network to which the present invention can be applied. The elements shown relative to FIG. 2 are set forth for the sake of illustration. However, it is to be appreciated that the present inventio can be applied to other network configurations as readily contemplated by one of ordinary skill in the art given, the teachings of the present invention provided herein, while maintaining the spirit of the present invention.

[0027] The environment 200 at, least includes a set of computer processing systems 210. The computer processing systems 210 can be any type of computer processing system including, but not limited to, servers, desktops, laptops, tablets, smart phones, media playback devices, and so forth. For the sake of illustration, the computer processing systems 210 include server -210A, server 21.0B, and server 2 I OC. [0028] ί» an emlx diment, the present invention improves distance metric learning with N-pair loss. The present invention can employ any of the computer processing systems 210 to perform distance metric learning with deep learning as described, herein, in an embodiment, one of the -computer processing systems 210 can classify information recei ved by other ones of the com uter processing systems.

[0029] In the embodiment shown in FIG. 2, the elements thereof are interconnected by a nefwork(s) 201. However, in other embodiments, other types of connections can also be used. Additionally, one or more elements i FIG. 2 .may be implemented by a variety of devices, which include but are not limited to, Digital Signal Processing (DSP) circuits, programmable processors, Application Specific- Integrated Circuits (ASIC^'s), Field Programmable Gate Arrays (FPGAs), Complex Programmable Logic Devices (C LDs), and so forth . These and other var iations of the elements of envi ronment 200 are readily determined by one of ordinary ski ll in the art, given the teachings of the present invention provided herein, while maintaining the spirit of the present invention.

[0030] FIG. 3 shows a high-level block/flow diagram of an exemplary sysiem/method 300 for deep metric learning with N-pair loss, in accordance with an embodiment of the present in vers don.

[0031 J At step 3 i 0, perform distance metric learning with deep learning.

[0032] In an embodiment step 310 includes steps 1. OA, 31 OB, and 310C.

[0033] At ste 31 OA, provide images to a deep convolutions!, neural network 350, The images include N pairs of examples from N different classes at once,

[0034] At step 310B. extract features from the images. [0035] At step 3 I OC, perform distance metric learning with N~pair loss on the features and form a classifier 370.

[0036] At step 320, test the system on image verification.

[0037] In an embodiment, step 320 includes steps 320A, 320B, 320C, 320D, 320E, and 320F.

[0038] At step 320A, receive a first image (image 1).

[0039] At step 32033, receive a second image (image 2).

[0040] At step 320C, extract features using a trained deep convoiutional neural network 35QA. The deep convoiutional neural network 350 is trained to become trained deep convoiutional neural network 350A.

[0041 ] At step 320D, output a first feature (feature 1).

[0042] At step 320E, output a second feature (feature .2).

[0043] At step 320F, input the features (feature 1 and feature 2) and into the classifier

370.

[0044] The classifier 370 can be used to generate predictions, based on which, certain actions can be taken (e.g., see. FIG. 6).

[0045] Regarding step 10, it is to be appreciated that the same differs from previous approaches in at least using N pairs of examples from N different classes at once.

[0046] Regarding step 320. it is to be appreciated that, the N-pair loss can be viewed as a form of neighborhood component analysis.

[0047] FIG. 4 further shows step 310 of method 300 of FIG . 3, in accordance with an embodiment of the present invention. In particular, FIG. 4 further shows distance metric learning with ^'N-pair loss 400. in. accordance with an embodiment of the present invention.

[0048] The deep convolutiona! neural network 350 receives N pairs of images 421 from different classes at once. In FIG; 4, the reference .numeral 401 shows features before training with N-pair loss, and the reference numeral 402 shows the feature after training with N-pair loss.

[0049] In FIG. 4, the following notations apply:

x: input image;

f: output feature;

fi: example from i-th pair;

fT: positive example from i-th pair with

lis having different class labels.

[0050] In an embodiment, N-pair loss can be defined as follows:

[0051] FIG. 5 is a diagram graphically showing the N-pair loss 400 of FIG. 4 in accordance wit an embodiment of the present inventio versas a conventional triplet loss 599 in accordance with the prior art.

[0052] The conventional triplet loss 599 is equivalent to a 2-pair loss.

[0053] The 2-pai loss is a generalization to a N-pair loss for N 2.

[0054] The following equations apply

[0055] FiGs. 6-8 sho a flow diagram of a method 600 for deep metric learning with N-pair loss, in accordance with an embodiment of the present invention.

[0056] At step 610, receive N pairs of training examples and class labels for the ^'framing examples that correspond to a plurality of classes . Each of the N pairs includes a respecti ve anchor example and further includes a respective non-anchor example capable of being a positive training example or a negative training example. In an embodiment, each of the N pairs of the training examples can correspond to a different one of the plurality of classes. In. an embodiment, the plurality of classes can be randomly sejected as a subset from a set of classes, wherein the set of classes includes the plurality of classes and on or more other classes. In an embodiment, the total number of the pluralit of classes at least one of (i) changes over time, (ii) is larger than a threshold amount, and (ni) is unknown.

[0057] At step 620. extract features of the N pairs by applying a deep convolutional neural network to the N pairs and to the class labels.

[0O58J At step 630, calculate, for each of the N pairs based oft the features, a respective similarly measure between the respective anchor example and the respective non-anchor example.

[0059] At step 640, calculate a similarity score based on the respective similarity measure for each of the pairs. The similarity score represents one or more similarities between all anchor points and all positive training examples in the pairs relative to one or more .similarities between all of the anchor points and all negative training examples in the N pairs.

[0060] In an embodiment, ste 640 includes one or more of steps 640 A , 640B, and 640C.

[0061] At step 640A, ^'bound a variable (p₍) used to calculate the respective similarity score of each of the pairs of training examples by at least one of a lower limit and an upper limit the variable representing a relative similarity between the anchor point and the positive training examples with respect to the anchor point and the negative training examples.

[0062] At step 640B, compute a gradient of a logarithm of the similarity score.

[0063] At step 640C, maximize aiV objecti ve function for deep metric learning.

[0064] In an embodiment, step 640C inc!udes step 640C I .

[0065] At step 640CL maximize a portion of the objection function that relates to the anchor points, wherein the objective function include the portion relating to the anchor points and at least one other portion relating to the non-anchor points.

[0066] At step 650. maximize the similarity score for the respective anchor example for each of the N pairs to pull together in a distribution space the training examples from a same one of th plurality of classes while pushing apart in the distribution space the training examples from different ones of the plurality- of classes, in an. embodiment, step 650 is capable of simultaneously pushing N-i examples away from a single reference sample from among the N pairs of training examples, in the distribution, space. In. an embodiment, step 650 is capable of simultaneously pushing N-I examples towards a single reference sample from among the N pairs of training examples, in the distribution space.

[0067] At ste 660, generate prediction using the deep convomtionaS neural network. For example, generate a facial recognition prediction, a speech recognitio prediction, a speaker recognition prediction, and so forth.

[0068] At step 670, perform an action responsive to the prediction. As appreciated by one of ordinary skill in the art, the actio taken is dependent upon the implementation. For example, access to an entity including, but not limited to, device, a system, or a facility, cm be granted responsive to the prediction. It is to be appreciated that the preceding actions are merely illustrative and, thus, other actions can also be performed!^' as readily appreciated by one of ordinary skill in the art, while maintaining the spirit of the present^' invention.

[0069] In a embodiment, step 670 includes step 670A.

[0070] At step 670A, verify a user and provide the user access to an entity, based on the prediction.

[0071] A description will now be given regarding supervised deep metric learning, in accordance with an embodiment of the present invention.

[0072] The description regarding supervised deep metric learning will ^'commence with a description regarding contrastive and triplet loss,

[0073] Supervised deep metric learning aims to learn an embedding vector representation of the data using deep neural networks that preserves the distance between examples from the same class small and those from different classes large. The contrastive loss and the triplet loss functions have been used to trai deep embedding networks:

where /^'(^■; $): χ→ W is a embedding ^'kernel defined by deep neural networks, and yi e{l ₅, ..,L} 's are the label of the data x₍€ χ. Herein, x⁺and x^~ are used to represent positive and negative examples of x, i.e., y⁺— y and y^~≠ y, respectively. [<£]+- m x {0; d) and m > 0 Is a toning parameter for margin. For simplicity,/™j(x) is- used to denote embedding vector representation of * while inheriting all superscripts, and subscripts when exist. Two objective functions^' are similar in the sense that they both optimize embedding kernels to preserve the distance between examples in the label space to the embedding space, but the triplet loss can be considered as a relaxation of th contrastjve loss since it only cares the relative margin of distances between positive and negative pairs, not their absolute values. The loss functions are differentiable with respect to the kernel parameters and therefore they can be readily used as an objective function for training dee neural networks.

[0074] Although it ^'sounds- straightforward., applying contrastiv loss or triplet loss functions to train dee neural networks that can provide highly discriminative embedding vector is non-trivial because the margin constraints of the above loss functions can be easily satisfied for most of the training pairs or triplets after few epochs of training. To avoid bad local minima, different data selection methods have been explored such as an online triplet selection algorithm that, selects {semi-) hard negative but all positive examples within each mini-batch containing few thousands of exemplars. Although the data selection step is essential, it becomes more inefficient for deep metric learning since each data sample should go through the forward pass of deep neural networks to compute the distance.

[0075] A description will now be given regarding A^r~pair Loss for Deep Metric Learning, in accordance with an embodiment of the present invention. Also, theoretical insight is provided regarding why A-pair loss is better than other existing loss functions for deep metric learning by showing relations to those loss functions, such as triplet loss and sof nsa loss.

[0076] A description of <V-pair loss will now be given. Consider N pairs of training examples £{% x† )}£L¾ and labels {( ^*)}!^*_' By ^' definition, >¾ = y * and it is presumed that none of the pairs of examples are from the same class, i.e., >·',·≠ yj Vi≠ . The similarity measure between the anchor point x_t and the positive or negative points {xfy -i is defined as follows:

pij ^ expire r }- and the score ( A pa is a normalized self-similarity, Xe.,

[0077] Not thai i is bounded by (0, .1 ) and it represents the relative similarity between anchor and positive points to the similarities between anchor and negative points. Maximizing the score of all anchor points in N-pair training subset pulls the examples from the same class together, but at the same time, it pushes the examples from different classes away based on their relative dissimilarity, i.e., negative examples in the proximity of anchor point will be pushed away than those already far enough,, as illustrated in FIG, 4. After all, the A-pair loss is defined as follows; mpair ({(¾ **)}f ) ~∑ l - lOgPi (5)

[0078] The gradient of log p_t w.r.t fj can be derived, as follows:

and the gradient wxt Θ can be computed b chain-rule.

[0079] TABLE 1 shows a comparison of Joss functions for deep metric learning. 2- pair loss is equivalent to triplet loss under 6 convergence criteria, while its score function is an approximation to that of .A/~pak loss ft >rA< > 2

TA BLE 1

triplet loss )if - n\² ₂ - wr -r «2 ^'+HL

exp( ^{T +})

2-pair loss ^0gLe p( ^T f⁺) + e p( /-) exp(/^! /^'+)

~pair l ss — t QPi

eS∞p( ⁺) +∑&i ex ^{ /i⁺)

[0080] A description will now be given regarding a comparison of iV-pair loss t triplet loss.

[0081] To illustrate the present invention, iV-pair loss is described with respect to triplet loss and sofhnax loss.

[0082] Regarding the comparison of iV-pair loss to triplet loss, a description of triplet loss and 2-pair loss will now be given,

[0083 J Relation between loss functions can be demonstrated by showing the equivalence between two sets of optimal embedding kernels with respect to each loss function (although the optima! sets of embedding kemeis for two loss ictions are equivalent). To proceed, optimaiity conditions for loss functions are defined as ioilows:

¾ = (f\m^_t(x_tx⁺,x-,'f) - 0,V(x,x+,x-)} {9}

~ V ( χ, X{ , X_z, X )} (10) where £ __pair = ~∑£=i— togpi— 6^']+ and. embedding kemeis are restricted to have unit 12 norm for both 2-pair and triplet losses. In the following, it is shown that 3¾¾ and

T-f-pair ί«¾ equivalent when€^™ ·— ff (~).

^tn' ^c: ?2- αίτ / ^e ^n-i a d consider any valid 2-pair sample

forms a valid triplet sample, we have the followin ;

** ΙΙΛ-Λ⁺ΪΙ^-ΗΛ-Λ Ι^™ on

^ fflfi -flfz >^) ( 3)

and this proves€ z-w r-

■^2- air -¾¾■ Similarly, let ^* 6 -FjL_pairand consider any valid -triplet sample

For any x₂ with y₂ - ^, a 2-pair sample {O&t,**), (x₂,¾^)} can be built that satisfies the following:

— log pi≤ — log σ (~~) ^♦«< (15)

where details in Equation (15) are omitted as it repeats Equation (1.1) - - (Ϊ4) baekwardly. Finally, this proves / 6 3¾ .

[0084] A descriptio will now be given regarding insight from soflraax loss,

[0085] The softmax loss with I classes is written as follows:

£-sa ftm.ax (% >¾)— ~^* > 0^;f I ^xi) ( 17)

where is a weight vector or a template for class /, It is often inefficient of impractical to compute the exact partition function

at training when L is very large. For such cases, the exact partition function can be approximated by randomly selecting a small subset of N templates including a ground-truth template as follows:

where 5 cr {1, .., , Lj, jSj— · iV and ¾ e S. The local partition function Z_s (x) less than ^for any S, and the approximation becomes more accurate with larger (noting that advanced subset sampling methods, such as importance sampling and hashing can be used to reduce approximation error with small /V), This provides a valuable insight when i¥-pair loss is compared to 2-pair loss (or A/-pair loss for M < N) since the self-similarity score of 2-pair loss can be viewed as an approximation to that of JV-pair loss, hi other words, afty self-Similarity score of A-pair loss can be approximated with those of 2-pair loss, but none of them is tight:^'

∑fLi expCtftf^") expCf^) + expCtf )

V/ £ {!, -·- , {i}. This implies that the actual score ofN-pair loss could be concealed behind the overvaiiied scor of the model when it is trained with 2-pair loss, and therefore the model is likely to be sab-opiimai. It. has been determined that the 2-pair loss significantly underilts to the training data compared to the iV-pair loss with /V > 2 or softmax loss models.

[0086] A description will now be given regarding implications of the present invention with respect to various relations.

[0087] The impl ications of these relations are summarised below:

! . The optimal set of embedding kernels for 2-pair loss and triplet loss are equivalent and the performance of the models trained -with these loss functions would he similar,

2. The -pair loss is an approximation of^'A'-pair loss .forM < A⁷.

[0088] A description will now be given regarding L2-Norm Reguiarization.

[0089] Note that the score function in Equation (4) is not designed t he invariant to the norm of embedding vectors. In other words, the score- function can be made to he arbitrarily close to I or 0 by re-sealing embedding vectors. This implies that the self- similarity score function can be maxim ized by increasing the norm of embedding vectors rather than finding right direction, and it is important to regularize the norm of the embedding vector io avoid such situation, e.g., 12 normalization on embedding vectors to compute triplet loss. However, for A-pair loss, applying 12 normalization makes optimization very difficult since the self-similarity score is upper bounded by (for example, the upper bound is 0.88 whea N ~ 2, but it decreases

to 0.105 when N ~ 64). Instead, we regularise b adding following penalty term ~∑LiSf/ilL + ! i⁺I to objective function that promotes the 12-norm of embedding vectors t be small,

[0090] A description will now be given regarding competitive/commercial values of the solution achieved by the present invention.

[00 1] The present invention allows efficient training by (!) removing hard negative data mimng, (2) removing computationally and parameter heavy softmax layer, and (3) faster convergence than previous deep metric learning approaches.

[0092] The present invention is effective for technologies such as face recognition where the number of output classes (e.g., identity) is extremely large.

[0093 The present invention is effective for online learning where the number of output classes is unknow or changi g over time.

[0094] Rather than using two pairs of examples with hard negative mining, N pairs of exanrples are used from a random subset of classes that enables pushing examples from different classes apart quickly,

[0095] Embodiments described herein may be entirely hardware, entirely software or including both hardware and software" elements. 111 a preferred embodiment, the present invention is implemented in software, which includes but is not iimited to firmware, resident software, microcode, etc.

[0096] Embodiments may include a computer program product accessible from a computer-usable or computer-readable medium providing program code for use by or in connection with a computer o any ins traction execution system. A computer-usable or computer readable medium may include any apparatus that stores, conunu-iicates, propagates, or transports the program for use by or m connection with the instruction execution system, apparatus,- of device. The medium can be magnetic, optical, electronic, electromagnetic, infrared, or semiconductor system (or apparatus or device) or a propagation medium. The medium may include a computer-readable storage medium such as a semiconductor or solid state memory, magnetic tape, a removable computer diskette, a random access memory (RAM), a read-only memory (ROM), a rigid magnetic disk and an optical disk, etc.

[0097] Each computer program may be tangibly stored in a machine- readable storage media or device (e.g., program memory or magnetic disk) readable by a general or special purpose programmable computer, for configuring and^' controlling operation, of a computer when the storage medi or device is read by the computer to perform the procedures described herein. The inventive system may also be considered to be embodied in a computer-readable storage medium, configured with a computer program, where the storage medium so configured causes a computer to operate in a specific and predefined manner to perform the functions described herein.

[0098] A data processing system suitable for storin and/or executing program code may include at least one processor coupled directly or indirectly to memory elements through a system bus. The memory elements can include local memory employed during actual execution of the program code, bulk storage, and cache memories which provide temporary storage of at least some program code to reduce the number of times code is retrieved from bulk storage during execution. Input/output or I/O devices (including but not limited to keyboards, displays, pointing devices, etc.) ma be coupled to the -system either directl or through intervening I/O controllers,

[0099] Network adapters m y also he coupled to the system to enable the data processing; system to become eonpied to other dat processing systems or remote printer or storage -devices through intervening private or public networks. Modems, cable motlera and Ethernet cards are just a few of the currently available types of network adapters.

[00100] Reference in the specification to "one embodiment" or "an embodiment:" of the present invention, as well as other variations thereof, means that a particular feature, ^'structure, characteristic, and so forth described in connection with the embodiment is included in at least erne embodiment of the present invention. Thus, the appearances of the -phrase "in one embodiment'⁵ or ¾ an embodiment", as well any other -variations, appearing in various places throughout the specification are not necessaril all referring to the same embodiment.

[00101] It is to be appreciated that the use of any of the following "/". "and/or", and "at least one of, for example, in the cases of "A/B", "A and/or B" and "at least one of A and B", is intended to encompass the selection of the first listed option (A) only, or the selection of the second listed option (B) only, or the selection of both options (A aid B). As a further example, in the cases of "A, B, and/or C" and. "at least one of A, B, and C", such phrasing is intended to encompass the selection of the first listed option (A) only, or the selection of the second listed option (B) only, or the selection of the third listed option (C) only, or the selection of the first and the second listed options (A and 6) only, or the Selection of the first and third listed options (A and C) only, or the- selection of the second and third listed options (B and G) onl.y₅-or the selection of all three options (A and B and C). This may be extended, as readil apparent by one of ordinary skill in this and related arts, for as many items listed,

[00102] The foregoing is to be understood as bein in ever respect illustrative and exemplary, but not restrictive, and the scope of the invention disclosed herein is not to be determined from the Detailed Description, but rather from the claims as interpreted according to the full breadt permitted by the patent laws. It is to be understood that the embodiments shown and described herein are only illustrative of the principles of tire present invention and that those skilled in the art may implement various modifications without departin from the scope and spirit of the invention. Those skilled in the art could implement various other feature combinations without departing from the scope and spirit of the invention. Having thus described aspects of the invention, with the details and particularity required by die patent laws, what is claimed and desired protected by Letters Patent is set forth in the appended claims.

Claims

WHAT IS CLAIMED IS:

1 , A computer-implemented method, comprising:

receiving, b a processor, N pairs of traming examples nd class labels for the training examples that correspond to a plurality of c lasses, w herein each of th e N pairs includes a respective anchor example and further includes a respective non-anchor example capable of being a positive training example or a negative traming example;

extracting, by the processor, features of the N pairs by applying a deep convolutional neural network to the N pairs and to the class labels;

calculating, by the processor for each of the N pairs based on the features, a respective similarly measure between the respective anchor example and the respective non-anchor example;

calculating, by the processor, a similarity score based on the respective similarit measure for each of the N pairs, the similarity score representing one or more similarities betwee all anchor points and all positive training examples in the N pairs relative to one or more similarities between all of the anchor points and all negative training examples in the N pairs; and

maximizing, by the processor, the similarity score for the respective anchor example for each of the N pairs to pull togethe in a distribution space the training examples from a same one of the plurality of classes while pushing apart in the distribution space the training examples from different ones of the plurali ty of classes.

2. The computer-implemented method of claim 1 , wherein each of the pairs of training examples corresponds^' to a differen one of the plurality of classes,

3. The computer-implemented method of claim 2, wherein the plurality Of classes are randomly selected as a subset from a set of classes, and wherein the set of classes includes the plurality of classes and one or more other classes.

4. The computer-implemented method of claim 1. wherem said maximizing step is capable of simultaneously pushing N-1 examples away from, a single reference sample from among the N pairs of training examples, in the distribution space .

5. The computer -implemented method of claim 1 , wherein 'said maximizing step is capable of simultaneously pushing N-1 examples towards a single reference sample from among the pairs of training examples, in the distribution space.

6. The computer-implemented method of claim 1, wherein the deep convolutions! neural network is configured to include embedding vectors that are tramed to satisfy a set of constraints on each loss function in a set of loss functions, wherein the deep convolutions! neural network is trained using the set of loss functions.

7. The computer-implemented method of claim L wherein said maximizing step comprising computing a gradient of a logarithm of the similarity score.

8. The computer implemented method of claim 1 , wherein said maxim zing s ep^'inae&imizes.-aa^'-obj^tiye -function for deep metric learning.

9. The computer im ktrsen ted method of claim 1 , wherein a total number of the plurality of classes at least one of (i) changes over time, (ii) is larger than a threshold amount, and (in) is unknown.

10. The computer-implemented method of claim 1. further comprising verifying a user and providing the user access to an entity, based on a prediction generated using tie deep convoiniioiial neural network.

11. A non-transitory article of manufacture tangibly embodying a -computer- readable program which when executed causes a computer to perform the steps of claim 1.

12. A system, comprising:

a processor, configured to;

receive, N pairs of training examples and class labels for the training examples that correspond to a plurality of classes, wherein eac of the N pairs includes a respective anchor example and further includes a respective non-anchor example capable of being a positive training example or a negative training example; extract features of the N pairs by applying a deep convolutions! neural network to the I pairs and to the class labels;

calculate, for each of the N pairs based on the features, a respective similarly measure between the respective anchor example and the respective non-aftdior example;

calculate a similarity score based o the respective similarity measure for each of the pairs, the similarity score .representing one or more similarities between all anchor points and all positive training examples in the N pairs relative to one or more similarities between all of the anchor points and all negati ve training examples in the N pairs; and

maximize the similarity score for the respective anchor example for each of the N pairs to pull together in a distribution space the training examples from same one of the plurality of classes while pushing apart in the distribution space the training examples from different ones of the plurality of classes.

13, The system of claim 12, wherein each of the pairs of the training examples corresponds to a different one of the plurality of classes and

14. The system of claim 13, wherein the processor is configured to randomly selected the plurality of classes as a subset from a set of classes, and wherein the set of classes includes the plurality of classes and one or more other classes.

15. The system of claim 12, wherein said processor is configured to

simultaneously push, in the distribution space, N-l examples away from a single

reference sample from among the N pairs of trai ing examples, responsive to a

.maximization of the similarity score,

1 . The system of claim 12, -wherein said processor is configured to simultaneously push, in the distribution space, N-l examples towards a single reference sample from among the N pairs of training examples, responsive to a maximization of the similarity score.

17. The system of claim 12, wherein the deep conventional neural network is configured to include embedding vectors that are trained to satisfy a set of constraints on. each loss function in a set of loss functions,^' wherein the deep convokiiional neural network is trained using the set of loss functions.

18. The system, of claim 1.2, wherein said processor is configured to maximize the similarity score by computing ^'a gradient of a logarithm of the similarity score .

19. The computer-implemented method, of claim 12, wherein a total number of the plurality of classes at least one of (i) changes over time, fii) is larger than a threshold amount, and (iii) is unknown. 20, The system of claim 12, w^'iierem said processor i s further configured to verify a user and provide the user access to an entity, based on a prediction generated using the deep coiivoiiitioiia! neural network.