JP2018528521A - Media classification - Google Patents

Abstract

  Multi-label classification is improved by determining threshold and / or scale factors. Selecting a threshold for multi-label classification includes sorting a set of label scores associated with the first label to create an ordered list. A precision value and recall value corresponding to the set of candidate thresholds are calculated from the score values. A threshold is selected from the candidate thresholds for the first label based on the target precision value or recall value. A scale factor is also selected for the activation function for multi-label classification for which the metric of the score within the range is calculated. The scale factor is adjusted when the score metric is not within range.

Description

Cross-reference of related applications
[0001] This application is a US Provisional Patent Application No. 62 / 199,865 entitled "MEDIA CLASSIFICATION", filed July 31, 2015, the entire disclosure of which is expressly incorporated herein by reference. Insist on the interests of.

  [0002] Certain aspects of the present disclosure relate generally to machine learning, and more particularly to improving systems and methods for media classification, particularly for labeling media files, including picture files. .

  [0003] An artificial neural network that may comprise interconnected groups of artificial neurons (eg, neuron models) is a computing device or represents a method to be performed by a computing device.

  [0004] A convolutional neural network is a type of feedforward artificial neural network. A convolutional neural network may include a collection of neurons that each have a receptive field and collectively tile the input space. Convolutional neural networks (CNN) have many applications. In particular, CNN is widely used in the area of pattern recognition and classification.

  [0005] Deep learning architectures, such as deep belief networks and deep convolutional networks, are layered neural network architectures, where the output of the first layer of neurons becomes the input to the second layer of neurons, and the neuron's The output of the second layer becomes the third layer of neurons, inputs, and so on. Deep neural networks can be trained to recognize a hierarchy of features, so they are increasingly used in object recognition applications. Like convolutional neural networks, the computations in these deep learning architectures can be distributed across a collection of processing nodes that can be configured in one or more computation chains. These multi-layer architectures can be trained one layer at a time and can be fine-tuned using back propagation.

  [0006] Other models are also available for object recognition. For example, Support Vector Machine (SVM) is a learning tool that can be applied for classification. The support vector machine includes a separating hyperplane (eg, a decision boundary) that categorizes the data. The hyperplane is defined by supervised learning. The desired hyperplane increases the margin of the training data. In other words, the hyperplane should have the largest minimum distance from the training example.

  [0007] Although these solutions achieve excellent results on some classification benchmarks, their computational complexity can be quite high. In addition, model training can be difficult.

  [0008] In one aspect, a method for selecting a threshold for multi-label classification is disclosed. The method includes sorting a set of label scores associated with the first label to create an ordered list. The method also includes calculating a precision value and a recall value corresponding to the set of candidate threshold values from the plurality of score values. The method also includes selecting a threshold from the candidate threshold for the first label based at least in part on the target precision value or target recall value.

  [0009] Another aspect discloses a method for selecting a scale factor for an activation function for multi-label classification. The method includes calculating a metric for a score within the range and adjusting the scale factor when the score metric is not within the range.

  [0010] In another aspect, an apparatus for selecting a threshold for multi-label classification in wireless communications is disclosed. The apparatus includes means for sorting a set of label scores associated with the first label to create an ordered list. The apparatus also includes means for calculating a precision value and a recall value corresponding to the set of candidate threshold values from the plurality of score values. The apparatus also includes means for selecting a threshold from the candidate threshold for the first label based at least in part on the target precision value or target recall value.

  [0011] Another aspect discloses an apparatus for selecting a scale factor for an activation function for multi-label classification. The apparatus includes means for calculating a metric for a score within the range and means for adjusting a scale factor when the score metric is not within the range.

  [0012] In another aspect, an apparatus for selecting a threshold for multi-label classification in wireless communications is disclosed. The apparatus includes a memory and at least one processor coupled to the memory. The processor (s) is configured to sort the set of label scores associated with the first label to create an ordered list. The processor (s) is also configured to calculate a precision value and a recall value corresponding to the set of candidate threshold values from the plurality of score values. The processor (s) is also configured to select a threshold from the candidate thresholds for the first label based at least in part on the target precision value or target recall value.

  [0013] Another aspect discloses an apparatus for selecting a scale factor for an activation function in wireless communications. The apparatus includes a memory and at least one processor coupled to the memory. The processor (s) is configured to calculate a metric for a score within the range and adjust the scale factor when the score metric is not within the range.

  [0014] In another aspect, a non-transitory computer readable medium for selecting a threshold for multi-label classification is disclosed. The non-transitory computer readable medium has a label associated with the first label to create an ordered list on the processor (s) when executed by the processor (s). It has a non-transitory program code recorded on it that causes it to perform the operation of sorting the set of scores. The program code also causes the processor (s) to calculate the precision and recall values corresponding to the set of candidate threshold values from the plurality of score values. The program code also causes the processor (s) to select a threshold from the candidate thresholds for the first label based at least in part on the target precision value or target recall value.

  [0015] Another aspect discloses a non-transitory computer readable medium for selecting a scale factor for an activation function. The non-transitory computer-readable medium, when executed by the processor (s), calculates to the processor (s) a score metric within the range, and the score metric falls within the range. When not, it has non-temporary program code recorded on it that causes it to perform an operation to adjust the scale factor.

  [0016] The foregoing has outlined rather broadly the features and technical advantages of the present disclosure in order that the detailed description of the invention that follows may be better understood. Additional features and advantages of the present disclosure are described below. Those skilled in the art will appreciate that the present disclosure can be readily utilized as a basis for modifying or designing other structures for carrying out the same purposes of the present disclosure. Those skilled in the art should also realize that such equivalent constructions do not depart from the teachings of the disclosure as set forth in the appended claims. The novel features believed to characterize the present disclosure, both as to the organization and method of operation of the present disclosure, as well as further objects and advantages, will be better understood when the following description is considered in conjunction with the accompanying drawings. It should be clearly understood, however, that each of the figures is provided for purposes of illustration and explanation only and does not define the limitations of the present disclosure.

  [0017] The features, characteristics, and advantages of the present disclosure will become more apparent from the detailed description set forth below when taken in conjunction with the drawings, in which like reference characters refer to like parts throughout.

[0018] FIG. 4 illustrates an example implementation for designing a neural network using a system-on-chip (SOC) that includes a general purpose processor in accordance with certain aspects of the present disclosure. [0019] FIG. 4 illustrates an example implementation of a system in accordance with aspects of the present disclosure. [0020] FIG. 5 illustrates a neural network according to aspects of the disclosure. [0021] FIG. 4 is a block diagram illustrating an example deep convolutional network (DCN), according to aspects of the disclosure. [0022] FIG. 7 is a block diagram illustrating an example software architecture that may modularize artificial intelligence (AI) functions according to aspects of the disclosure. [0023] FIG. 7 is a block diagram illustrating runtime operation of an AI application on a smartphone according to aspects of the disclosure. [0024] FIG. 4 is a block diagram illustrating an exemplary binary classification process. [0025] The figure which shows the concept of precision and recall. [0026] FIG. 7 illustrates a general example of a classification process according to aspects of the disclosure. [0027] FIG. 7 is a block diagram illustrating an example slope selection function of a classification process according to aspects of the disclosure. [0028] FIG. 7 is a block diagram illustrating an exemplary threshold selection function of a classification process according to aspects of the disclosure. [0029] A graph showing the score for a label according to aspects of the disclosure. [0030] FIG. 7 is a graph illustrating threshold selection utilizing an F measure, according to aspects of the disclosure. [0031] FIG. 6 is a flow diagram illustrating a method for selecting a threshold for multi-label classification according to aspects of the disclosure. [0032] FIG. 6 is a flow diagram illustrating a method for selecting a scale factor for an activation function according to aspects of the disclosure.

  [0033] The detailed description set forth below in connection with the appended drawings is intended as a description of various configurations and is not intended to represent the only configurations in which the concepts described herein may be implemented. . The detailed description includes specific details for providing a thorough understanding of various concepts. However, it will be apparent to those skilled in the art that these concepts may be practiced without these specific details. In some instances, well-known structures and components are shown in block diagram form in order to avoid obscuring such concepts.

  [0034] Based on these teachings, the scope of the present disclosure may be implemented independently of other aspects of the present disclosure or in combination with other aspects of the present disclosure. Those skilled in the art should appreciate that they cover any aspect. For example, an apparatus can be implemented or a method can be implemented using any number of the described aspects. Further, the scope of the present disclosure is such an apparatus or method implemented using other structures, functions, or structures and functions in addition to or in addition to the various aspects of the present disclosure described Shall be covered. It should be understood that any aspect of the disclosure disclosed may be practiced by one or more elements of a claim.

  [0035] The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any aspect described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other aspects.

  [0036] Although particular aspects are described herein, many variations and permutations of these aspects fall within the scope of the disclosure. Although some benefits and advantages of the preferred aspects are described, the scope of the disclosure is not intended to be limited to particular benefits, uses, or objectives. Rather, aspects of the present disclosure shall be broadly applicable to various technologies, system configurations, networks, and protocols, some of which are illustrated by way of example in the figures and the following description of preferred aspects. . The detailed description and drawings are merely illustrative of the disclosure rather than limiting, the scope of the disclosure being defined by the appended claims and equivalents thereof.

  [0037] Aspects of the present disclosure are directed to systems and methods for labeling media files. A database of media files may associate each stored media file with one or more labels. In addition, the function calculates a score for each label based on the media file. For example, for a boat photo in a lake, the function may calculate a high score for the labels “boat” and “lake” and a low score for the remaining labels in the database (eg, “car” and “barn”). Can be calculated. The function can be a neural network and the score can be the activation level of the output layer of the neural network.

  [0038] One aspect of the present disclosure is directed to a method of selecting a classifier threshold for a labeling system for each label. For the example of an image of a boat in a lake, the calculated score for “boat” may be 0.8 and the calculated score for “lake” may be 0.9. An image in the database that actually has a boat in the image (and so labeled) ensures that it has a score of 0.6 or higher and includes a lake in the image (and its Images) are reliably determined to have a score of 0.8 or higher. This is because the image in the database for which the function (neural network) calculated a score of 0.7 for “lake” is less likely to contain a lake, and a calculation of 0.7 for “boat” An image with an assigned score means that it is more likely to contain a boat. This information about the database can then be applied to set different thresholds for the classifier system for each label. In this example, the threshold for “boat” may be set to 0.6 and the threshold for “lake” may be set to 0.8.

  [0039] Another aspect of the present disclosure is directed to changing the calculation of scores in the final layer of a neural network. Over the database of images, the original function (neural network) can calculate a set of scores for a given label that can be characterized as having a very narrow distribution. For example, when the tolerance range is between -1.0 and 1.0, all of the values can fall between 0.7 and 0.9. Thus, the threshold setting operation disclosed above may not give sufficient generalization to a new image. For example, images of lakes tend to be scored at values between 0.8 and 0.9, but images that do not include lakes are calculated scores for lakes between 0.75 and 0.79 The labeling system performance becomes very sensitive to the exact placement of the threshold at 0.8.

  [0040] Furthermore, the function (neural network) may be expected to calculate a score slightly below 0.8 for a new image containing a lake due to normal variation of the image. Similarly, a new image that does not include a lake may have a calculated score slightly above 0.8. Thus, setting the threshold for “Lake” to 0.8 can result in many false-negative and false-positive results. To mitigate this sensitivity, aspects of the present disclosure are directed to changing the activation function for the final layer of the neural network. As a result of this change, the distribution of scores for a given label may have a wider and more uniform distribution across the distribution of images. Aspects of the present disclosure provide improved generalization because the calculated score for positive examples and the calculated score for negative examples can be diffused more distantly.

  [0041] FIG. 1 illustrates the above labeling of a media file using a system on chip (SOC) 100 that may include a general purpose processor (CPU) or multi-core general purpose processor (CPU) 102, according to some aspects of the present disclosure. An exemplary implementation is shown. Variables (eg, neural signals and synaptic weights), system parameters (eg, weighted neural networks) associated with computing devices, delays, frequency bin information, and task information are associated with a neural processing unit (NPU) 108. Stored in a block, stored in a memory block associated with the CPU 102, stored in a memory block associated with the graphics processing unit (GPU) 104, or memory block associated with the digital signal processor (DSP) 106 Stored in a dedicated memory block 118, or distributed across multiple blocks. Instructions executed on general purpose processor 102 may be loaded from program memory associated with CPU 102, or may be loaded from dedicated memory block 118.

  [0042] The SOC 100 also includes additional processing blocks adapted to specific functions, such as GPU 104, DSP 106, and fourth generation long term evolution (4G LTE®) connectivity, unlicensed Wi-Fi®. A connectivity block 110 that may include connectivity, USB connectivity, Bluetooth® connectivity, and the like, and a multimedia processor 112 that may detect and recognize gestures, for example, may be included. In one implementation, the NPU is implemented in a CPU, DSP, and / or GPU. The SOC 100 may also include a navigation 120 that may include a sensor processor 114, an image signal processor (ISP), and / or a global positioning system.

  [0043] The SOC may be based on an ARM instruction set. In one aspect of the present disclosure, instructions are loaded into at least one processor, such as general purpose processor 102, which is coupled to memory. The instructions may comprise code for sorting a set of label scores associated with the first label to create an ordered list. The instructions loaded into the general purpose processor 102 may also comprise code for calculating a precision value and a recall value corresponding to the set of candidate threshold values from the set of score values. Further, the instructions loaded into general purpose processor 102 may also comprise code for selecting a threshold from the candidate thresholds for the first label based on the target precision value or target recall value.

  [0044] In another aspect of the present disclosure, the instructions loaded into the general purpose processor 102 may comprise code for calculating a metric for a score within a range. Further, the instructions loaded into the general purpose processor 102 may comprise code for adjusting the scale factor when the score metric is not within range.

  [0045] FIG. 2 illustrates an exemplary implementation of the system 200 in accordance with certain aspects of the present disclosure. As shown in FIG. 2, system 200 may have multiple local processing units 202 that may perform various operations of the methods described herein. Each local processing unit 202 may comprise a local state memory 204 and a local parameter memory 206 that may store the parameters of the neural network. Further, the local processing unit 202 includes a local (neuron) model program (LMP) memory 208 for storing a local model program, a local learning program (LLP) memory 210 for storing a local learning program, and a local connection memory. 212. Further, as shown in FIG. 2, each local processing unit 202 has a configuration processor unit 214 for providing configuration for the local memory of the local processing unit, and a routing connection that provides routing between the local processing units 202. It may interface with the processing unit 216.

  [0046] A deep learning architecture may perform object recognition tasks by learning to represent input at a continuously higher level of abstraction in each layer, thereby accumulating useful feature representations of the input data. In this way, deep learning addresses the major bottleneck of traditional machine learning. Prior to the advent of deep learning, machine learning techniques for object recognition problems may rely heavily on human-designed features, sometimes in combination with shallow classifiers. The shallow classifier can be, for example, a two-class linear classifier in which the weighted sum of feature vector components can be compared to a threshold value to predict which class the input belongs to. A human-designed feature can be a template or kernel adapted to a particular problem area by an engineer with domain expertise. In contrast, deep learning architectures learn to represent features that are similar to what a human engineer can design, but can do so through training. In addition, deep networks can learn to represent and recognize new types of features that humans may not consider.

  [0047] A deep learning architecture may learn a hierarchy of features. For example, if visual data is presented, the first layer may learn to recognize simple features in the input stream, such as edges. When auditory data is presented, the first layer may learn to recognize spectral power at a particular frequency. The second layer, which takes the output of the first layer as input, can learn to recognize a combination of features, such as a simple shape in the case of visual data, or a sound combination in the case of auditory data. Upper layers can learn to represent complex shapes in visual data or words in auditory data. Further higher layers may learn to recognize common visual objects or utterance phrases.

  [0048] Deep learning architectures may work particularly well when applied to problems with natural hierarchical structures. For example, motor vehicle classification may benefit from first learning to recognize wheels, windscreens, and other features. These features can be combined in higher layers in different ways to recognize cars, trucks, and airplanes.

  [0049] Neural networks can be designed with various connectivity patterns. In a feedforward network, information is passed from a lower layer to an upper layer, and each neuron in a given layer communicates with a neuron in the upper layer. As explained above, hierarchical representations can be accumulated in successive layers of the feedforward network. Neural networks can also have recurrent or feedback coupling (also called top-down). In recurrent coupling, the output from a neuron in a given layer is communicated to another neuron in the same layer. A recurrent architecture can help recognize patterns that evolve over time. The connection from a neuron in a given layer to a neuron in a lower layer is called feedback (or top-down) connection. A network with many feedback combinations can be useful when recognition of high-level concepts can help discriminate specific low-level features of the input.

  [0050] Referring to FIG. 3A, the connections between the layers of the neural network may be full connections 302 or local connections 304. In fully connected network 302, a neuron in a given layer can communicate its output to every neuron in the next layer. Alternatively, in the local connection network 304, neurons in a given layer can be coupled to a limited number of neurons in the next layer. The convolutional network 306 may be a local connection, and is a special case where the connection strength associated with each neuron in a given layer is shared (eg, 308). More generally, the local connectivity layer of the network is configured such that each neuron in the layer has the same or similar connectivity pattern, but with connectivity strengths that may have different values (eg, 310 , 312, 314, and 316). The connectivity pattern of local connections is a spatial pattern in the upper layer, because upper layer neurons in a given region can receive input that has been adjusted through training to the properties of a limited portion of the total input to the network. Can result in distinct receptive fields.

  [0051] Locally coupled neural networks may be suitable for problems where the spatial location of the inputs is meaningful. For example, a network 300 designed to recognize visual features from an in-vehicle camera may develop higher layer neurons with different properties depending on their association with the lower part of the image versus the upper part. Neurons associated with the lower part of the image can learn to recognize lane markings, for example, while neurons associated with the upper part of the image can learn to recognize traffic signals, traffic signs, etc. .

  [0052] The DCN may be trained using supervised learning. During training, the DCN can be presented with an image 326, such as a cropped image of a speed limit indicator, and then a “forward path” can be calculated to produce an output 328. The output 328 may be a vector of values corresponding to the features, such as “indicator”, “60”, and “100”. The network designer may note that the DCN corresponds to some of the neurons in the output feature vector, eg, “label” and “60” as shown in the output 328 for the trained network 300. You may wish to output a high score. Prior to training, the output generated by the DCN can be inaccurate, so an error can be calculated between the actual output and the target output. The DCN weights can then be adjusted so that the DCN output score is more closely matched to the target.

  [0053] In order to properly adjust the weights, the learning algorithm may calculate a gradient vector for the weights. The slope may indicate how much the error increases or decreases when the weight is adjusted slightly. In the top layer, the gradient may directly correspond to the value of the weight that combines the activated neurons in the penultimate layer with the neurons in the output layer. In the lower layer, the gradient may depend on the weight value and the calculated error gradient in the upper layer. The weight can then be adjusted to reduce the error. This way of adjusting the weight is sometimes called “back propagation” because it involves a “backward path” through the neural network.

  [0054] In practice, the error gradient of weights may be calculated over a small number of examples so that the calculated gradient approximates the true error gradient. This approximation method is sometimes referred to as stochastic gradient descent. Stochastic gradient descent may be repeated until the overall achievable error rate does not decrease or until the error rate reaches a target level.

  [0055] After learning, the DCN may be presented with a new image 326, and a forward path through the network may result in an output 328 that may be considered an inference or prediction of the DCN.

  [0056] A deep belief network (DBN) is a probabilistic model comprising multiple layers of hidden nodes. The DBN can be used to extract a hierarchical representation of the training data set. The DBN can be obtained by laminating layers of restricted Boltzmann Machine (RBM). An RBM is a type of artificial neural network that can learn a probability distribution over a set of inputs. RBM is often used in unsupervised learning because RBM can learn probability distributions in the absence of information about the class to which it should be categorized. Using a hybrid unsupervised and supervised paradigm, the lower RBM of the DBN can be trained in an unsupervised fashion and can act as a feature extractor, where the upper RBM (simultaneously with the input from the previous tier and the target class Can be trained in a supervised manner (on the distribution) and can act as a classifier.

  [0057] A deep convolutional network (DCN) is a network of convolutional networks composed of an additional pooling layer and a normalization layer. DCN achieves state-of-the-art performance for many tasks. The DCN can be trained using supervised learning, where both input and output targets are known for many samples and are used to change the network weights by using gradient descent methods.

  [0058] The DCN may be a feedforward network. Furthermore, as explained above, the connections from neurons in the first layer of the DCN to groups of neurons in the next higher layer are shared across neurons in the first layer. DCN feedforward and covalent bonding can be exploited for high speed processing. The computational burden of DCN may be much less than that of a similarly sized neural network with recurrent or feedback coupling, for example.

  [0059] The processing of each layer of the convolutional network may be viewed as a spatially invariant template or base projection. When an input is first decomposed into multiple channels, such as the red, green, and blue channels of a color image, the convolutional network trained on that input has two spatial dimensions along the image axis and color information. Can be considered to be three-dimensional, with a third dimension that captures. The output of the convolutional combination is thought to form a feature map in subsequent layers 318, 320, and 322, where each element of the feature map (eg, 320) is from various neurons in the previous layer (eg, 318), And may receive input from each of the plurality of channels. The values in the feature map can be further processed using non-linearities, such as rectification, max (0, x), etc. Values from neighboring neurons can be further pooled 324, which corresponds to downsampling and can provide additional local invariance and dimensionality reduction. Normalization corresponding to whitening can also be applied by lateral suppression between neurons in the feature map.

  [0060] The performance of deep learning architectures may improve as more labeled data points become available or as computing power increases. Modern deep neural networks are routinely trained with computing resources that are thousands of times larger than those available to general researchers just 15 years ago. New architectures and training paradigms can further enhance deep learning performance. A normalized linear unit can reduce a training problem known as vanishing gradients. New training techniques may reduce over-fitting and thus allow larger models to achieve better generalization. Encapsulation techniques can extract data at a given receptive field and further enhance overall performance.

  [0061] FIG. 3B is a block diagram illustrating an exemplary deep convolutional network 350. As shown in FIG. The deep convolutional network 350 may include multiple different types of layers based on connectivity and weight sharing. As shown in FIG. 3B, an exemplary deep convolution network 350 includes a plurality of convolution blocks (eg, C1 and C2). Each of the convolution blocks may be composed of a convolution layer, a normalization layer (LNorm), and a pooling layer. The convolution layer may include one or more convolution filters, which may be applied to the input data to generate a feature map. Although only two convolution blocks are shown, the present disclosure is not so limited, and instead any number of convolution blocks may be included in the deep convolution network 350 according to design preferences. The normalization layer can be used to normalize the output of the convolution filter. For example, the normalization layer can provide whitening or lateral suppression. The pooling layer may perform downsampling aggregation over space for local invariance and dimension reduction.

  [0062] For example, a parallel filter bank of a deep convolution network may be loaded into the CPU 102 or GPU 104 of the SOC 100, optionally based on the ARM instruction set, to achieve high performance and low power consumption. In an alternative embodiment, the parallel filter bank may be loaded into the DSP 106 or ISP 116 of the SOC 100. In addition, the DCN may access other processing blocks that may exist on the SOC, such as processing blocks dedicated to sensors 114 and navigation 120.

  [0063] The deep convolutional network 350 may also include one or more full coupling layers (eg, FC1 and FC2). Deep convolution network 350 may further include a logistic regression (LR) layer. Between each layer of the deep convolutional network 350 is a weight (not shown) to be updated. The output of each layer is a deep convolution network 350 for learning a hierarchical feature representation from the input data (eg, image, audio, video, sensor data and / or other input data) provided in the first convolution block C1. Can serve as input for subsequent layers in.

  [0064] FIG. 4 is a block diagram illustrating an example software architecture 400 that may modularize artificial intelligence (AI) functions. Using the architecture, an application 402 may be designed that may cause various processing blocks of the SOC 420 (eg, CPU 422, DSP 424, GPU 426, and / or NPU 428) to perform supporting computation during the runtime operation of the application 402.

  [0065] The AI application 402 may be configured to invoke functions defined in the user space 404, which may provide, for example, scene detection and recognition indicating the location where the device currently operates. The AI application 402 may configure the microphone and camera differently depending on, for example, whether the recognized scene is an office, a lecture hall, a restaurant, or an outdoor environment such as a lake. The AI application 402 may make a request for compiled program code associated with a library defined in the SceneDetect application programming interface (API) 406 to provide an estimate of the current scene. This requirement may ultimately rely on, for example, the output of a deep neural network configured to provide scene estimation based on video and positioning data.

  [0066] Further, a runtime engine 408, which may be compiled code of the runtime framework, may be accessible to the AI application 402. The AI application 402 may, for example, cause the runtime engine to request a scene estimate that is triggered at a particular time interval or by an event detected by the application user interface. When the scene is estimated, the runtime engine may signal an operating system 410, such as the Linux kernel 412 running on the SOC 420. Operating system 410 may cause computations to be performed on CPU 422, DSP 424, GPU 426, NPU 428, or some combination thereof. CPU 422 may be accessed directly by the operating system, and other processing blocks may be accessed through drivers, such as drivers 414-418 for DSP 424, GPU 426, or NPU 428. In the illustrative example, the deep neural network may be configured to operate on a combination of processing blocks, such as CPU 422 and GPU 426, or may be operated on NPU 428 if present.

FIG. 5 is a block diagram illustrating the runtime operation 500 of the AI application on the smartphone 502. The AI application may include a preprocessing module 504 that may be configured to convert the format of the image 506 and then crop and / or resize the image 508 (eg, using the JAVA programming language). The preprocessed image is then communicated to a classification application 510 that includes a SceneDetect backend engine 512 that can be configured (eg, using the C programming language) to detect and classify a scene based on visual input. Can be done. The SceneDetect backend engine 512 may be configured to further pre-process 514 the image with scaling 516 and cropping 518. For example, the image can be scaled and cropped such that the resulting image is 224 pixels by 224 pixels. These dimensions can be mapped to the input dimensions of the neural network. The neural network may be configured by the deep neural network block 520 to cause various processing blocks of the SOC 100 to further process the image pixels using the deep neural network. The deep neural network results can then be thresholded 522 and passed through an exponential smoothing block 524 in the classification application 510. The smoothed result may then result in settings on the smartphone 502 and / or changes in the display.
Scale factor and threshold selection for classification
[0068] Aspects of the present disclosure are directed to media classification, in particular for labeling media files including picture files. Aspects are directed to binary and multi-label classification. In particular, in the illustrative example, three separate sample images contain different colored soccer balls. The first image includes only a blue soccer ball, the second image includes only a green soccer ball, and the third image includes only a red soccer ball. Each image can be labeled based on the color of the soccer ball in the image. This process of assigning labels is called classification. In another case, a single image contains several colored soccer balls. For the same task, images are labeled with multiple colors. This is called multi-label classification.

  [0069] In machine learning, the classifier provides a score and a decision function for each label. The decision function tests whether the score is above a certain threshold. For a single label classifier, the scores of all labels are taken into account to determine which label is correct.

  [0070] For multi-label classification, each label may be correct regardless of the score of the other labels. The threshold is therefore important for determining which labels belong to the object. Working with a classifier that outputs false positives with very high scores or false negatives with very low scores makes the problem of finding the correct threshold difficult. Aspects of the present disclosure are directed to improving scale factor and threshold selection for classification.

  [0071] FIG. 6 is an exemplary flow diagram 600 illustrating a binary classification process. In one example, the classification process includes a training phase 601 and a prediction phase 602. In the training phase 601, an image is input to the feature extractor 610. Those skilled in the art will appreciate that any type of multimedia file, including sounds or images, can be input to the feature extractor. In this illustrative example, each image is passed through a feature extractor 610 to obtain image features and classifications. In this example, a binary classification of the image is obtained. The binary classification can be a positive response or a negative response. Alternatively, the output can be a “yes” or “no” label. A learning function 612 learns features for a particular concept or element of training.

  [0072] Next, in the prediction phase 602, the image is passed through a feature extractor 620. The features are supplied to the classifier 622, and based on the learning model used by the learning function 612, the classifier 622 outputs a score. Decision function 624 receives the score. In one aspect, the decision function 624 determines whether the score is greater than or less than zero. When the score is greater than 0 and the threshold is 0 (or no threshold), the output is “yes”. In other cases, the output is “no”. The decision function may be based on a global threshold (eg, 0) utilized by the binary classifier.

  [0073] Additional criteria, such as precision and recall, can be utilized in determining the performance of the classifier. The precision is true positive divided by the total number of elements labeled as belonging to the positive class (for example, the sum of true positives and false positives, items that are incorrectly labeled as belonging to the class) (Eg, the number of items correctly labeled as belonging to the positive class). The recall is divided by the total number of elements that actually belong to the positive class (for example, the sum of true positives and false negatives, items that were not labeled but belonged to the positive class). The number of true positives. FIG. 7 shows the concept of precision, recall and F scale formula (based on precision and recall).

  [0074] The following is an illustrative example of media classification. A machine is configured to perform the task of labeling a soccer ball in the sample image. In particular, the machine utilizes a classifier that takes an image as input and outputs a list of labels (eg, colors) for the image. In this example, the machine is given three images with a blue ball, three images with a green ball, and four images with a red ball. The classifier outputs the label “red” to only two of the images with a red ball and erroneously outputs the label “red” to an image with a green ball. The precision is the number of images correctly labeled “red” divided by the total number of images labeled “red”. In this example, the precision for the label “red” is 2/3. The recall is the number of images correctly labeled red and divided by the total number of images that should have been the label “red”. In the previous example, the recall is 2/4 = 1/2.

  [0075] The optimal threshold is 1, where the precision and recall are both 1. This rarely occurs because false positives and false negatives affect accuracy. The precision and recall are equal when the number of objects assigned to a label is equal to the number of objects that should be assigned to that label. In the previous example, labeling the four images as “red” will make the precision and recall equal. Labeling five or more images is very likely to reduce the precision because it is more likely to label the wrong image as red. Labeling less than four images can reduce the reproducibility because it will reduce the molecules if correctly labeled images are excluded. Therefore, there is a compromise between precision and recall. In other words, a higher precision is obtained at the expense of recall and vice versa.

  [0076] FIG. 8A is a block diagram illustrating a general example of a classification process 800 in accordance with aspects of the present disclosure. The classification process includes a training phase 801 and a prediction phase 802. In the training phase 801, a feature extractor 810 receives each image and / or media file and outputs the received image features and binary classification. A learning function 812 learns specific features for specific concepts or elements of training.

  [0077] In the prediction phase 802, the feature extractor 820 receives each image and outputs the image features to the classifier 822. Based on the received features and training model, classifier 822 outputs a raw score to activation function 824. The activation function 824 normalizes the score to fall within a range, for example, the range can be between 0 and 1 or in the range between 1 and -1. Further, the slope selection function 830 determines a scaling factor (eg, slope) for use by the activation function 824. Various parameters may be modified to affect the factors used by the activation function 824 described below. The activation function 824 can be a logistic function, a tan-h function, or a linear normalization function.

  [0078] The normalized score output by the activation function 824 is received by the decision function 826. A threshold selection function 840 determines a threshold for use by the decision function 826. In some aspects, the threshold selection function 840 determines a non-zero threshold. The threshold selection function 840 is described in more detail below.

  [0079] FIG. 8B illustrates an example of a slope selection function 830. The slope selection function 830 uses the image data set to create a list of raw scores for a particular concept / label. In order to obtain the desired distribution of scores, the slope selection function 830 determines a scale factor (eg, slope). In particular, a raw score 832 from a database of images is provided. An activation function 833 is applied to the raw score 832. The scores are then sorted at block 835. In one example, the sorted score is also shown graphically. The percentage of scores that are within a particular range is calculated at block 837. In addition, a target ratio is established. The target percentage indicates the percentage of images that are within a range of values. When the target percentage is met, the scale factor 838 is set to an amount that resulted in the number of images in range. For example, if the target percentage is 90% and 90% of the image is within a certain range, the scale factor 838 is set to a value given that amount of images in that range.

  [0080] Furthermore, the scale factor is adjusted when the target percentage is not met. For example, the scale factor may be adjusted incrementally by an alpha value at block 839. The adjusted scale factor 836 is applied by the activation function at block 833 and the process is repeated. The scale factor is adjusted incrementally repeatedly until the target percentage is achieved. In another aspect, the slope selection function 830 utilizes target slope instead of target percentage. For example, a particular slope may be targeted to a range between “a” and “b”. Optionally, in another aspect, instead of incrementing the scale factor, alternative search functions can be utilized by defining a minimum scale factor and a maximum scale factor. In particular, for example, the scale factor may be adjusted by dividing the difference between the minimum and maximum scale factors by two to determine a new scale factor. In another optional aspect, only range end points are used when iterating through different scaling factors. Furthermore, in another aspect, the scale factor can be approximated by using the inverse of the activation function at the range endpoint.

  [0081] A threshold selection function 840 may be utilized to adjust the threshold, as shown in FIG. 8C. By adjusting the threshold to a value other than 0, improved accuracy can be observed. Furthermore, by adjusting the threshold, a trade-off between precision and recall can be realized. For example, the threshold may be adjusted to obtain a desired precision at the expense of recall and vice versa. Furthermore, adjusting the threshold excludes ambient values (reflecting objects around that particular object in the image). For example, if the image includes trees and chairs on a meadow against a blue sky, the classifier can be trained to consider trees, grass and the sky as a general surrounding. Adjusting the threshold excludes ambient values associated with trees and grass, and therefore takes into account values associated with chairs.

  [0082] In one aspect, the threshold is calculated by sorting the scores for each label, calculating the precision and recall after sorting, and then performing a calculation to select the threshold Can be determined. FIG. 8C shows an example of a threshold selection function 840 that determines the threshold. First, normalized scores for all inputs are obtained for a particular label. Sort function 842 may sort the normalized scores and optionally create an ordered list. For example, the scores can be sorted in descending order. Using the sorted list of scores, the calculation function 844 calculates precision and recall by thresholding each score. In other words, the precision value and recall value are calculated for each corresponding set of candidate threshold values. A threshold can then be selected from the candidate thresholds. The selection may be based at least in part on the target precision value and / or the target recall value.

  [0083] Alternatively, instead of using every score, an average of consecutive scores may be used as the set threshold. After calculating the precision and recall, a threshold is selected by the selection function 846 based on the precision and recall. The selection function analyzes a combination of thresholds and associated precision and / or recall values.

  [0084] Further, in another aspect, the threshold may be based on a value corresponding to a maximum F score. This is the case, for example, when there is no value for which the precision value is above the target precision, when the recall value is above the target recall value, or when the precision or recall is too low when the precision value target is met Can happen. Further, the threshold may be selected based on an F-score that uses a beta value that leans towards precision or recall.

  [0085] FIG. 9 is a graph 900 showing the score for a particular label (eg, “empty”). The classifier can be trained to learn different concepts in the image. Thousands of images pass through the classifier and the sorted and normalized score for “empty” is shown in line 901. Each score has a possible value between -1.0 and 1.0. The precision and recall are then calculated and plotted at lines 902 and 903, respectively. The precision line 902 and recall line 903 are on different scales of 0.0 to 1.0 on the right side of the graph. Line 904 is a threshold line. Line 904 shows the selected threshold, and the selected threshold is the classifier score where the dashed line intersects the sorted score line 901. Each score along line 901 can be selected as a candidate threshold, and a vertical threshold line (eg, 904) is analyzed to determine the relevance and recall for that candidate threshold.

  [0086] Various methods may be used to select thresholds such as, but not limited to, target relevance and maximum F scale. For example, for target precision, a score with a precision that is slightly above the target precision is selected. For example, the threshold may be selected by targeting a 90% match rate.

  [0087] In some scenarios, the threshold may not meet the target percentage and a fallback method is utilized. For example, the F scale function 848 of FIG. 8C may utilize an F scale formula and select a threshold based on the value corresponding to the maximum F score. The F scale formula is as follows.

Here, i is an image count. argrnax (F _β ) is calculated to determine an index into the list of scores. The score at this location is the threshold. The beta (β) parameter provides a way to lean towards recall or precision. When beta is greater than 1 (β> 1), more emphasis is placed on recall. Adjusting the F scale provides feedback on precision and / or recall. Further, the beta value in the F scale equation can be manipulated to affect the precision value or recall value. FIG. 10 is a graph 1000 illustrating threshold selection using the F scale. Lines 1005, 1006 and 1007 are the result of using different beta values for the F scale.

  [0088] Optionally, in an alternative embodiment, a bias value is utilized rather than a threshold value. In particular, instead of using a threshold, the threshold can be embedded in the score by adding a bias or by normalizing the score based on the threshold. Further, in an optional aspect, rather than using an actual score, a score for each concept may be encoded, so the score does not represent a score for each concept.

  [0089] In one configuration, the model is configured to sort a set of label scores associated with the first label to create an ordered list. The model is also configured to calculate a precision value and recall value corresponding to the set of candidate threshold values from a set of score values (eg, multiple score values). Further, the model is configured to select a threshold from the candidate thresholds for the first label based on the target precision or target recall. The model includes means for sorting, means for calculating, and / or means for selecting. In one aspect, the sorting means, calculating means, and / or selecting means are configured to perform the indicated function, the general purpose processor 102, the program memory associated with the general purpose processor 102, the memory block 118, local processing It may be unit 202 and / or routing connection processing unit 216. In another configuration, the means described above may be any module or any device configured to perform the function provided by the means described above.

  [0090] In another configuration, the model is configured to sort a set of label scores associated with the first label to create an ordered list. The model is also configured to calculate a metric for scores within the range and to adjust the scale factor when the score metric is not within range. The model includes means for calculating and / or adjusting the metrics. In one aspect, the metric calculation means and / or adjustment means is a general purpose processor 102, a program memory associated with the general purpose processor 102, a memory block 118, a local processing unit 202, configured to perform the indicated function. And / or a routing connection processing unit 216. In another configuration, the means described above may be any module or any device configured to perform the function provided by the means described above.

  [0091] In addition, the model may also include means for incrementing the scale factor and / or means for dividing. In one aspect, the incrementing means and the dividing means are general purpose processor 102, program memory associated with general purpose processor 102, memory block 118, local processing unit 202, and / or routing configured to perform the indicated function. It may be a connection processing unit 216. In another configuration, the means described above may be any module or any device configured to perform the function provided by the means described above.

  [0092] According to some aspects of the present disclosure, each local processing unit 202 determines network parameters based on the desired one or more functional characteristics of the network, and the determined parameters are further adapted. Can be configured to develop one or more functional features toward the desired functional features as adjusted and updated.

  [0093] FIG. 11 shows a method 1100 for selecting a threshold for multi-label classification. At block 1102, the process sorts the set of label scores associated with the first label to create an ordered list. In block 1104, the process calculates a precision value and recall value corresponding to the set of candidate threshold values from the set of score values. Further, at block 1106, the process selects a threshold from the candidate thresholds for the first label based on the target precision or target recall.

  [0094] FIG. 12 shows a method 1200 for selecting a scale factor for an activation function. At block 1202, the process calculates a metric for scores within the range. In block 1204, the process adjusts the scale factor when the score metric is not within range.

  [0095] Various operations of the methods described above may be performed by any suitable means capable of performing corresponding functions. Such means may include various (one or more) hardware and / or software components and / or modules including, but not limited to, circuits, application specific integrated circuits (ASICs), or processors. . In general, if there are operations shown in the figures, they may have corresponding counterpart means-plus-function components with similar numbers.

  [0096] As used herein, the term "determining" encompasses a wide variety of actions. For example, “determining” means calculating, computing, processing, deriving, examining, looking up (eg, a table, database or another data structure Lookup), confirmation, etc. Further, “determining” can include receiving (eg, receiving information), accessing (eg, accessing data in a memory) and the like. Further, “determining” may include resolving, selecting, selecting, establishing and the like.

  [0097] As used herein, a phrase referring to "at least one of a list of items" refers to any combination of those items, including a single member. By way of example, “at least one of a, b, or c” is intended to include a, b, c, ab, ac, bc, and abc.

  [0098] Various exemplary logic blocks, modules and circuits described in connection with this disclosure may include general purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate array signals ( FPGA) or other programmable logic device (PLD), individual gate or transistor logic, individual hardware components, or any combination thereof designed to perform the functions described herein Or it can be implemented. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any commercially available processor, controller, microcontroller or state machine. The processor is also implemented as a combination of computing devices, eg, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors associated with a DSP core, or any other such configuration. obtain.

  [0099] The method or algorithm steps described in connection with this disclosure may be implemented directly in hardware, in a software module executed by a processor, or in a combination of the two. . A software module may reside in any form of storage medium that is known in the art. Some examples of storage media that may be used include random access memory (RAM), read only memory (ROM), flash memory, erasable programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM ( Registered trademark)), registers, hard disks, removable disks, CD-ROMs, and the like. A software module may comprise a single instruction, or multiple instructions, and may be distributed over several different code segments, between different programs, and across multiple storage media. A storage medium may be coupled to the processor such that the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor.

  [00100] The methods disclosed herein comprise one or more steps or actions for achieving the described method. The method steps and / or actions may be interchanged with one another without departing from the scope of the claims. In other words, unless a specific order of steps or actions is specified, the order and / or use of specific steps and / or actions may be changed without departing from the scope of the claims.

  [00101] The functions described may be implemented in hardware, software, firmware, or any combination thereof. When implemented in hardware, an exemplary hardware configuration may comprise a processing system in the device. The processing system can be implemented using a bus architecture. The bus may include any number of interconnecting buses and bridges depending on the specific application of the processing system and the overall design constraints. The bus may link various circuits including a processor, a machine readable medium, and a bus interface to each other. The bus interface can be used to connect the network adapter, in particular, to the processing system via the bus. Network adapters can be used to implement signal processing functions. In some aspects, a user interface (eg, keypad, display, mouse, joystick, etc.) may also be connected to the bus. A bus may also link various other circuits, such as timing sources, peripherals, voltage regulators, power management circuits, etc., which are well known in the art and are therefore not further described.

  [00102] The processor may be responsible for managing buses and general processing, including execution of software stored on machine-readable media. The processor may be implemented using one or more general purpose and / or dedicated processors. Examples include microprocessors, microcontrollers, DSP processors, and other circuits that can execute software. Software should be broadly interpreted to mean instructions, data, or any combination thereof, regardless of names such as software, firmware, middleware, microcode, hardware description language, and the like. Machine-readable media include, for example, random access memory (RAM), flash memory, read only memory (ROM), programmable read only memory (PROM), erasable programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), register, magnetic disk, optical disk, hard drive, or other suitable storage medium, or any combination thereof. A machine-readable medium may be implemented in a computer program product. The computer program product may comprise packaging material.

  [00103] In a hardware implementation, the machine-readable medium may be part of a processing system that is separate from the processor. However, as those skilled in the art will readily appreciate, the machine-readable medium or any portion thereof may be external to the processing system. By way of example, a machine-readable medium may include a transmission line, a data modulated carrier wave, and / or a computer product separate from the device, all of which may be accessed by a processor via a bus interface. Alternatively or additionally, the machine-readable medium or any portion thereof may be integrated into the processor, as may the cache and / or general purpose register file. Although the various described components, such as local components, may be described as having a particular location, they may also be various, such as some components configured as part of a distributed computing system. Can be configured in various ways.

  [00104] The processing system includes one or more microprocessors that provide processor functionality, all linked together with other support circuitry via an external bus architecture, and an external memory that provides at least a portion of the machine-readable medium. Can be configured as a general-purpose processing system. Alternatively, the processing system may comprise one or more neuromorphological processors for implementing the neuron model and neural system model described herein. As another alternative, a processing system includes an application specific integrated circuit (ASIC) having a processor, a bus interface, a user interface, support circuitry, and at least a portion of a machine readable medium integrated on a single chip. Or one or more field programmable gate arrays (FPGAs), programmable logic devices (PLDs), controllers, state machines, gate logic, discrete hardware components, or other suitable circuitry, or throughout this disclosure It can be implemented using any combination of circuits that can perform the various functions described. Those skilled in the art will understand how best to implement the described functionality for a processing system, depending on the particular application and the overall design constraints imposed on the overall system. .

  [00105] A machine-readable medium may comprise a number of software modules. A software module includes instructions that, when executed by a processor, cause the processing system to perform various functions. The software module may include a transmission module and a reception module. Each software module can reside in a single storage device or can be distributed across multiple storage devices. As an example, a software module can be loaded from a hard drive into RAM when a trigger event occurs. During execution of the software module, the processor may load some of the instructions into the cache to increase access speed. One or more cache lines can then be loaded into a general purpose register file for execution by the processor. When referring to the functionality of a software module below, it will be understood that such functionality is implemented by a processor when executing instructions from that software module. Further, it should be appreciated that aspects of the present disclosure result in improvements in the functionality of processors, computers, machines, or other systems that implement such aspects.

  [00106] When implemented in software, the functions may be stored on or transmitted over as non-transitory computer-readable media as one or more instructions or code. Computer-readable media includes both computer storage media and communication media including any medium that enables transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a computer. By way of example, and not limitation, such computer readable media can be RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage device, or desired program in the form of instructions or data structures. Any other medium that can be used to carry or store the code and that can be accessed by a computer can be provided. In addition, any connection is properly referred to as a computer-readable medium. For example, the software may use a website, server, or other remote, using coaxial technology, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared (IR), wireless, and microwave. When transmitted from a source, coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of the medium. Discs and discs used herein are compact discs (CDs), laser discs (discs), optical discs (discs), digital versatile discs (discs) DVD, floppy disk, and Blu-ray disk, where the disk normally reproduces data magnetically and is a disc. ) Reproduce the data optically with a laser. Thus, in some aspects computer readable media may comprise non-transitory computer readable media (eg, tangible media). In addition, in other aspects computer readable media may comprise transitory computer readable media (eg, signals). Combinations of the above should also be included within the scope of computer-readable media.

  [00107] Accordingly, some aspects may comprise a computer program product for performing the operations presented herein. For example, such a computer program product stores a computer (and / or encoded) with instructions stored thereon that can be executed by one or more processors to perform the operations described herein. A readable medium may be provided. In some aspects, the computer program product may include packaging material.

  [00108] Further, modules and / or other suitable means for performing the methods and techniques described herein may be downloaded by user terminals and / or base stations when applicable, and / or It should be appreciated that it can be obtained in other ways. For example, such a device may be coupled to a server to allow transfer of means for performing the methods described herein. Alternatively, the various methods described herein allow user terminals and / or base stations to store storage means (eg, physical storage media such as RAM, ROM, compact disk (CD) or floppy disk). It can be provided by storage means so that various methods can be obtained when coupled to or provided with. Moreover, any other suitable technique for providing a device with the methods and techniques described herein may be utilized.

  [00109] It is to be understood that the claims are not limited to the precise configuration and components illustrated above. Various modifications, changes and variations may be made in the arrangement, operation and details of the methods and apparatus described above without departing from the scope of the claims.

[00109] It is to be understood that the claims are not limited to the precise configuration and components illustrated above. Various modifications, changes and variations may be made in the arrangement, operation and details of the methods and apparatus described above without departing from the scope of the claims.
The invention described in the scope of claims at the beginning of the application of the present application will be added below.
[C1]
A method for selecting a threshold for multi-label classification,
Sorting the set of label scores associated with the first label to create an ordered list;
Calculating precision and recall values corresponding to a set of candidate thresholds from multiple score values;
Selecting a threshold from the candidate threshold for the first label based at least in part on a target precision value or target recall value.
[C2]
The threshold is
There is no value with a precision value that exceeds the target precision value, or there is no value with the recall value that exceeds the target recall value, or
The value corresponding to the maximum F-score when either the precision value is too low when the target recall value is satisfied, or the recall value is too low when the target precision value is satisfied The method of C1, based at least in part on.
[C3]
The method of C2, wherein the selecting is based at least in part on an F-score using a beta value that leans towards precision or recall.
[C4]
A method for selecting a scale factor for an activation function for multi-label classification comprising:
Calculating a metric for scores within the range;
Adjusting the scale factor when the metric of the score is not within the range.
[C5]
The method of C4, wherein the activation function comprises a logistic function, a tan-h function, or a linear normalization function.
[C6]
The method of C4, wherein the metric of scores comprises a percentage.
[C7]
The method of C4, wherein the metric of scores comprises a slope.
[C8]
Adjusting the scale factor;
Incrementing the scale factor by a value;
The method of C4, comprising one of dividing a difference between a minimum scale factor and a maximum scale factor by two.
[C9]
An apparatus for selecting a threshold for multi-label classification in wireless communication,
Memory,
At least one processor coupled to the memory, the at least one processor comprising:
Sorting the set of label scores associated with the first label to create an ordered list;
Calculating precision and recall values corresponding to a set of candidate thresholds from multiple score values;
Selecting a threshold from the candidate threshold for the first label based at least in part on a target precision value or target recall value;
Configured to do the device.
[C10]
The threshold is
There is no value with a precision value that exceeds the target precision value, or there is no value with the recall value that exceeds the target recall value, or
The value corresponding to the maximum F-score when either the precision value is too low when the target recall value is satisfied, or the recall value is too low when the target precision value is satisfied The device of C9, based at least in part on.
[C11]
The apparatus of C10, wherein the at least one processor is configured to select based at least in part on an F-score that uses a beta value that leans toward precision or recall.
[C12]
An apparatus for selecting a scale factor for an activation function in wireless communication,
Memory,
At least one processor coupled to the memory, the at least one processor comprising:
Calculating a metric for scores within the range;
Adjusting the scale factor when the metric of the score is not within the range;
Configured to do the device.
[C13]
The apparatus of C12, wherein the activation function comprises a logistic function, a tan-h function, or a linear normalization function.
[C14]
The apparatus of C12, wherein the metric of scores comprises a percentage.
[C15]
The apparatus of C12, wherein the metric of scores comprises a slope.
[C16]
The at least one processor comprises:
Incrementing the scale factor by a value;
The apparatus of C12, configured to adjust the scale factor by at least one of dividing a difference between a minimum scale factor and a maximum scale factor by two.
[C17]
A non-transitory computer readable medium for selecting a threshold for multi-label classification, the non-transitory computer readable medium having non-transitory program code recorded thereon, the program code comprising:
Program code for sorting a set of label scores associated with a first label to create an ordered list;
Program code for calculating precision and recall values corresponding to a set of candidate thresholds from multiple score values;
Non-transitory computer readable medium comprising program code for selecting a threshold from the candidate threshold for the first label based at least in part on a target precision value or target recall value.
[C18]
The threshold value is determined when the precision value does not exceed the target precision value, or when the recall value does not exceed the target recall value, or when the target recall value is satisfied. As described in C17, when the precision value is either too low or the recall value is too low when the target precision value is met, based at least in part on the value corresponding to the maximum F-score Non-transitory computer readable medium.
[C19]
The non-transitory computer readable medium of C18, wherein the program code is configured to select based at least in part on an F-score that uses a beta value that leans toward precision or recall.
[C20]
A non-transitory computer readable medium for selecting a scale factor for an activation function, the non-transitory computer readable medium having non-transitory program code recorded thereon, the program code comprising:
Program code to calculate the metric for scores within the range;
A non-transitory computer readable medium comprising: program code for adjusting the scale factor when the metric of the score is not within the range.
[C21]
The non-transitory computer readable medium of C20, wherein the activation function comprises a logistic function, a tan-h function, or a linear normalization function.
[C22]
The non-transitory computer readable medium of C20, wherein the metric of scores comprises a percentage.
[C23]
The non-transitory computer readable medium of C20, wherein the metric of scores comprises a slope.
[C24]
The program code is
Incrementing the scale factor by a value;
The non-transitory computer readable medium of C20, configured to adjust the scale factor by at least one of dividing a difference between a minimum scale factor and a maximum scale factor by two.
[C25]
An apparatus for selecting a threshold for multi-label classification in wireless communication,
Means for sorting a set of label scores associated with the first label to create an ordered list;
Means for calculating precision and recall values corresponding to a set of candidate thresholds from a plurality of score values;
Means for selecting a threshold from the candidate threshold for the first label based at least in part on a target precision value or target recall value.
[C26]
The threshold value is determined when the precision value does not exceed the target precision value, or when the recall value does not exceed the target recall value, or when the target recall value is satisfied. As described in C25, when the precision value is either too low or the recall value is too low when the target precision value is met, based at least in part on the value corresponding to the maximum F-score apparatus.
[C27]
The apparatus of C26, wherein the means for selecting is based at least in part on an F-score that uses a beta value that leans towards precision or recall.
[C28]
An apparatus for selecting a scale factor for an activation function for multi-label classification in wireless communications, comprising:
A means for calculating a metric for a score within a range;
Means for adjusting the scale factor when the metric of the score is not within the range.
[C29]
The apparatus of C28, wherein the activation function comprises a logistic function, a tan-h function, or a linear normalization function.
[C30]
The apparatus of C28, wherein the metric of scores comprises a percentage.
[C31]
The apparatus of C28, wherein the metric of scores comprises a slope.
[C32]
The means for adjusting the scale factor comprises:
Means for incrementing the scale factor by a value;
The apparatus of C28, comprising one of means for dividing a difference between a minimum scale factor and a maximum scale factor by two.

Claims (32)

A method for selecting a threshold for multi-label classification,
Sorting the set of label scores associated with the first label to create an ordered list;
Calculating precision and recall values corresponding to a set of candidate thresholds from multiple score values;
Selecting a threshold from the candidate threshold for the first label based at least in part on a target precision value or target recall value. The threshold is
There is no value with a precision value exceeding the target precision value, or there is no value with the recall value exceeding the target recall value, or the precision value is too low when the target recall value is satisfied Or the recall value is too low when the target precision value is met,
The method of claim 1, wherein the method is based at least in part on a value corresponding to a maximum F score.   The method of claim 2, wherein the selecting is based at least in part on an F-score using a beta value that leans towards precision or recall. A method for selecting a scale factor for an activation function for multi-label classification comprising:
Calculating a metric for scores within the range;
Adjusting the scale factor when the metric of the score is not within the range.   The method of claim 4, wherein the activation function comprises a logistic function, a tan-h function, or a linear normalization function.   The method of claim 4, wherein the metric of scores comprises a percentage.   The method of claim 4, wherein the metric of scores comprises a slope. Adjusting the scale factor;
Incrementing the scale factor by a value;
The method of claim 4, comprising one of dividing a difference between a minimum scale factor and a maximum scale factor by two. An apparatus for selecting a threshold for multi-label classification in wireless communication,
Memory,
At least one processor coupled to the memory, the at least one processor comprising:
Sorting the set of label scores associated with the first label to create an ordered list;
Calculating precision and recall values corresponding to a set of candidate thresholds from multiple score values;
Selecting a threshold value from the candidate threshold values for the first label based at least in part on a target precision value or target recall value;
apparatus. The threshold is
There is no value with a precision value exceeding the target precision value, or there is no value with the recall value exceeding the target recall value, or the precision value is too low when the target recall value is satisfied Or the recall value is too low when the target precision value is met,
10. The apparatus of claim 9, wherein the apparatus is based at least in part on a value corresponding to a maximum F score.   The apparatus of claim 10, wherein the at least one processor is configured to select based at least in part on an F-score that uses a beta value that leans toward precision or recall. An apparatus for selecting a scale factor for an activation function in wireless communication,
Memory,
At least one processor coupled to the memory, the at least one processor comprising:
Calculating a metric for scores within the range;
Adjusting the scale factor when the metric of the score is not within the range; and
apparatus.   The apparatus of claim 12, wherein the activation function comprises a logistic function, a tan-h function, or a linear normalization function.   The apparatus of claim 12, wherein the metric of scores comprises a percentage.   The apparatus of claim 12, wherein the metric of a score comprises a slope. The at least one processor comprises:
Incrementing the scale factor by a value;
The apparatus of claim 12, configured to adjust the scale factor by at least one of dividing a difference between a minimum scale factor and a maximum scale factor by two. A non-transitory computer readable medium for selecting a threshold for multi-label classification, the non-transitory computer readable medium having non-transitory program code recorded thereon, the program code comprising:
Program code for sorting a set of label scores associated with a first label to create an ordered list;
Program code for calculating precision and recall values corresponding to a set of candidate thresholds from multiple score values;
Non-transitory computer readable medium comprising program code for selecting a threshold from the candidate threshold for the first label based at least in part on a target precision value or target recall value.   The threshold value is determined when the precision value does not exceed the target precision value, or when the recall value does not exceed the target recall value, or when the target recall value is satisfied. 18. At least in part based on a value corresponding to a maximum F-score when either the precision value is too low or the recall value is too low when the target precision value is met A non-transitory computer readable medium as described.   The non-transitory computer-readable medium of claim 18, wherein the program code is configured to select based at least in part on an F-score that uses a beta value that leans toward precision or recall. A non-transitory computer readable medium for selecting a scale factor for an activation function, the non-transitory computer readable medium having non-transitory program code recorded thereon, the program code comprising:
Program code to calculate the metric for scores within the range;
A non-transitory computer readable medium comprising: program code for adjusting the scale factor when the metric of the score is not within the range.   21. The non-transitory computer readable medium of claim 20, wherein the activation function comprises a logistic function, a tan-h function, or a linear normalization function.   21. The non-transitory computer readable medium of claim 20, wherein the metric of scores comprises a percentage.   21. The non-transitory computer readable medium of claim 20, wherein the metric of scores comprises a slope. The program code is
Incrementing the scale factor by a value;
21. The non-transitory computer readable medium of claim 20, configured to adjust the scale factor by at least one of dividing a difference between a minimum scale factor and a maximum scale factor by two. . An apparatus for selecting a threshold for multi-label classification in wireless communication,
Means for sorting a set of label scores associated with the first label to create an ordered list;
Means for calculating precision and recall values corresponding to a set of candidate thresholds from a plurality of score values;
Means for selecting a threshold from the candidate threshold for the first label based at least in part on a target precision value or target recall value.   The threshold value is determined when the precision value does not exceed the target precision value, or when the recall value does not exceed the target recall value, or when the target recall value is satisfied. 26. To at least partially based on a value corresponding to a maximum F-score when either the precision value is too low or the recall value is too low when the target precision value is met The device described.   27. The apparatus of claim 26, wherein the means for selecting is based at least in part on an F-score that uses a beta value that leans toward precision or recall. An apparatus for selecting a scale factor for an activation function for multi-label classification in wireless communications, comprising:
A means for calculating a metric for a score within a range;
Means for adjusting the scale factor when the metric of the score is not within the range.   30. The apparatus of claim 28, wherein the activation function comprises a logistic function, a tan-h function, or a linear normalization function.   30. The apparatus of claim 28, wherein the metric of scores comprises a percentage.   30. The apparatus of claim 28, wherein the metric of scores comprises a slope. The means for adjusting the scale factor comprises:
Means for incrementing the scale factor by a value;
29. The apparatus of claim 28, comprising one of means for dividing a difference between a minimum scale factor and a maximum scale factor by two.

Images