Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
  • G06F17/10—Complex mathematical operations
  • G06F17/11—Complex mathematical operations for solving equations, e.g. nonlinear equations, general mathematical optimization problems
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
  • G06F17/10—Complex mathematical operations
  • G06F17/18—Complex mathematical operations for evaluating statistical data, e.g. average values, frequency distributions, probability functions, regression analysis
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F18/00—Pattern recognition
  • G06F18/20—Analysing
  • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
  • G06F18/211—Selection of the most significant subset of features
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F18/00—Pattern recognition
  • G06F18/20—Analysing
  • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
  • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N20/00—Machine learning
  • G06N20/10—Machine learning using kernel methods, e.g. support vector machines [SVM]
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N5/00—Computing arrangements using knowledge-based models
  • G06N5/01—Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N7/00—Computing arrangements based on specific mathematical models
  • G06N7/01—Probabilistic graphical models, e.g. probabilistic networks
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N7/00—Computing arrangements based on specific mathematical models
  • G06N7/08—Computing arrangements based on specific mathematical models using chaos models or non-linear system models

Definitions

• Stochastic optimization is an approach for minimizing or maximizing a function that uses randomness to partially evaluate constituent functions and may thus be applicable to optimize very complex models.
• Methods and systems of the present disclosure advantageously enable smoothing of various objective functions in robust inference problems, thereby making such functions amenable to computation via stochastic-gradient methods using sampling in place of solving the inference problem exactly.
• Such methods and systems advantageously connect the gradient of the smoothed function approximation to a Boltzmann distribution, which can be sampled by a sampling device using a simulated process and/or quantum process, in particular quantum-annealing process, thermal or adiabatic relaxation of a classical computer, semi-classical computer, or a quantum
• processor/device and/or other physical process.
• Systems and methods of the present disclosure may advantageously improve the technical field of data science so that complex inference problems can be solved in various applications in data science, such as clustering of documents, group detection in a crowd, recommender systems, semi- supervised learning, and active learning.
• the systems and methods disclosed herein can also have various applications in natural language processing, such as noun phrase coreference resolution, and computer vision and image processing applications, such as image segmentation.
• the present disclosure advantageously utilizes a sampling device for solving the complex robust inference problem.
• the sampling device can comprise a quantum processor and a quantum device control system for obtaining the schedule of the set of scaling parameters and the data of the robust inference problem.
• the quantum processor may be coupled to the digital computer and to the quantum device control system.
• the quantum processor may comprise a plurality of qubits and a plurality of couplers, each coupler of the plurality of couplers for providing a communicative coupling at a crossing of two qubits of the plurality of qubits.
• the one or more samples of discrete vectors may follow a Boltzmann distribution.
• the sampling device can be a network of optical parametric oscillators, the network can comprise: an optical device, the optical device configured to receive energy from an optical energy source and generate a plurality of optical parametric oscillators; and a plurality of coupling devices, each of which controllably couples an optical parametric oscillator of the plurality of optical parametric oscillators.
• the sampling device may comprise a central processing unit, e.g., a digital computer or a mobile device, and a memory unit coupled to the central processing unit.
• the memory unit may comprise an application for obtaining the schedule of the scaling parameter and the data of the robust inference problem. Such application can be web application or mobile application.
• the sampling device can comprise a reconfigurable digital hardware, a central processing unit and a memory unit, the central processing unit and the memory unit coupled to the
• the reconfigurable digital hardware may be adapted for obtaining the schedule of the scaling parameter and the data of the robust inference problem, and wherein the reconfigurable digital hardware is adapted to perform a Markov Chain Monte Carlo algorithm.
• the Markov Chain Monte Carlo algorithm may be Simulated Quantum Annealing.
• the Markov Chain Monte Carlo algorithm may be Simulated Annealing.
• the Markov Chain Monte Carlo algorithm may be Gibbs Sampling.
• the set of loss functions may comprise one or more loss functions.
• the stochastic optimization of the robust inference problem may be associated with training a structured support vector machine. Each subset of the non-overlapping subsets of loss functions may comprise only two loss functions. The stochastic optimization of the robust inference problem may be associated with image segmentation.
• the stochastic optimization of the robust inference problem may be associated with a dual of the basis pursuit problem from compressed sensing.
• the stochastic optimization of the robust inference problem may be associated with semi- supervised learning.
• the data of the robust inference problem can be associated with one or more image segmentation problems.
• the data of the robust inference problem can be associated with a dual of the basis pursuit problem from one or more compressed sensing problems.
• the data of the robust inference problem may be associated with semi-supervised learning.
• the data of the robust inference problem can be obtained from a noun phrase co-reference resolution problem.
• the data of the robust inference problem may be associated with active learning.
• the data of the robust inference problem may be associated with one or more image tagging problems.
• the data of the robust inference problem may be associated with a recommender system.
• the schedule of the set of scaling parameters can be determined manually by a user or automatically by an algorithm or a computer program.
• the schedule of the set of scaling parameters can be determined using a machine learning algorithm based on history of the set of scaling parameters.
• the digital computer can be remotely located with respect to the sampling device.
• the stopping criterion may be based at least in part on a magnitude of a distance between the current and the updated current continuous vectors.
• the loss functions can comprise of composite functions of the first and second set of arguments.
• the loss functions can comprise of composite functions of the first and second set of arguments.
• each of the one or more gradients may be of the loss function taken with respect to the first argument comprises of iterative applications of chain rule.
• the iterative application of chain rule may be performed using auto-differentiation.
• Computing a search direction may utilize one or more of: stochastic gradient descent (SGD), stochastic average gradient methods (SAG and SAGA), stochastic variance-reduced gradient (SVRG), and stochastic dual coordinate ascent (SDCA).
• SGD stochastic gradient descent
• SAGA stochastic average gradient methods
• SVRG stochastic variance-reduced gradient
• SDCA stochastic dual coordinate ascent
• Step length uses one of the adaptive gradient descent methods may include but may not be limited to Adam, reduced mean square (RMS), RMSProp, and AdaGrad.
• RMS reduced mean square
• RMSProp reduced mean square
• AdaGrad AdaGrad
• a computer-implemented method for stochastic optimization of a robust inference problem using a sampling device may comprise: (a) receiving, by a digital computer, data of said robust inference problem, wherein said data comprises: (i) a set of objective functions or loss functions grouped into non-overlapping subsets, wherein each objective function or loss function in said set of loss functions accepts first and second arguments; and (ii) a set of permissible vectors for each objective function or loss function in said set of said objective functions or loss functions; (b) setting, by said digital computer, a current value of a vector; (c) receiving, by said digital computer, a schedule of a set of scaling parameters; and (d) until a stopping criterion is met: (i) determining current values of said set of scaling parameters based at least in part on said schedule; (ii) selecting a subset of said objective functions or loss functions from said non-overlapping subsets; (iii) iterating the following steps for each objective function or
• Said objective functions or loss functions may comprise one or more composite functions of said first and second arguments.
• Obtaining, by said digital computer, one or more gradients, wherein each of said one or more gradients is of said objective function or loss function taken with respect to said first argument may comprise iterative applications of a chain rule.
• Said chain rule may be performed using auto-differentiation.
• One or more argument functions of said composite functions may comprise differentiable feature extractors.
• Said differentiable feature extractors may comprise deep neural networks.
• a search direction may comprise using one or more of stochastic gradient descent (SGD), stochastic average gradient methods (SAG and SAGA), stochastic variance-reduced gradient (SVRG), or stochastic dual coordinate ascent (SDCA).
• a step length may comprise using one or more of said adaptive gradient descent methods, and wherein said adaptive gradient descent methods comprises Adaptive Moment Estimation (Adam), reduced mean square (RMS), Root Mean Square Propagation (RMSProp), and/or adaptive gradient algorithm.
• Adam Adaptive Moment Estimation
• RMS reduced mean square
• RMSProp Root Mean Square Propagation
• AdaGrad Adaptive Moment Estimation
• Said sampling device may comprise a quantum processor and a quantum device control system for obtaining said schedule of said set of scaling parameters and said data of said robust inference problem.
• Said quantum processor may be coupled to said digital computer and to said quantum device control system.
• Said quantum processor may comprise a plurality of qubits and a plurality of couplers, each coupler of said plurality of couplers for providing a communicative coupling at a crossing of two qubits of said plurality of qubits.
• Said one or more samples of discrete vectors may follow a Boltzmann distribution.
• Said sampling device may comprise a network of optical parametric oscillators, said network comprising: (a) an optical device, said optical device configured to receive energy from an optical energy source and generate a plurality of optical parametric oscillators; and (b) a plurality of coupling devices, each of which controllably couples an optical parametric oscillator of said plurality of optical parametric oscillators.
• Said sampling device may comprise a central processing unit and a memory unit coupled to said central processing unit.
• Said memory unit may comprise an application for obtaining said schedule of said scaling parameter and said data of said robust inference problem, wherein said application is configured to implement a Markov Chain Monte Carlo algorithm.
• Said sampling device may comprise a reconfigurable digital hardware, a central processing unit and a memory unit, said central processing unit and said memory unit coupled to said reconfigurable digital hardware.
• Said reconfigurable digital hardware may be configured to obtain said schedule of said scaling parameter and said data of said robust inference problem, and said reconfigurable digital hardware may be configured to implement a Markov Chain Monte Carlo algorithm.
• Said Markov Chain Monte Carlo algorithm may comprise simulated quantum annealing.
• Said Markov Chain Monte Carlo algorithm may comprise simulated annealing.
• Said Markov Chain Monte Carlo algorithm may comprise Gibbs sampling.
• Said set of objective functions or loss functions may comprise one or more objective functions or loss functions.
• Said stochastic optimization of said robust inference problem may be associated with training a structured support vector machine.
• Each subset of said non-overlapping subsets of objective functions or loss functions may comprise only two objective functions or loss functions.
• Said data of said robust inference problem may be associated with an image segmentation problem.
• Said data of said robust inference problem may be associated with a dual of said basis pursuit problem from a compressed sensing problem.
• Said data of said robust inference problem may be associated with semi-supervised learning.
• Said data of said robust inference problem may be obtained from a noun phrase co reference resolution problem.
• Said data of said robust inference problem may be associated with active learning.
• Said data of said robust inference problem may be associated with an image tagging problem.
• Said data of said robust inference problem may be associated with a recommender system.
• Said schedule of said set of scaling parameters may be determined by a user or automatically by an algorithm,
• Said digital computer may be remotely located with respect to said sampling device.
• Said stopping criterion may be based at least in part on a magnitude of a distance between said current and said updated current vectors.
• Said first and second arguments may be independent, and said first argument may employ a continuous vector as its value, said second argument may employ a discrete vector as its value, and said set of permissible vectors may comprise a set of permissible discrete vectors.
• (1) may comprise generating, by said sampling device, one or more samples of discrete vectors, each sample of said one or more samples being generated from said set of permissible discrete vectors associated with said objective function or loss function, wherein each sample of said one or more samples is generated based on a probability distribution determined at least in part by said set of scaling parameters and said objective function or loss function, wherein said first argument of said objective function or loss function takes said current value of said continuous vector.
• (2) may comprise obtaining, by said digital computer, one or more gradients, wherein each of said one or more gradients is of said loss function taken with respect to said first argument, wherein said first argument of said loss function takes said current value of said continuous vector, and said second argument takes value of a selected sample from said one or more samples, wherein said selected sample is non-repetitively selected.
• Said stopping criterion may comprise a set of rules for determining accuracy of a solution to said robust inference problem.
• Said selection of said subset of objective functions or loss functions may be non-repetitive or repetitive.
• a system for stochastic optimization of a robust inference problem using a sampling device may comprise a digital computer configured to: (a) receive data of said robust inference problem, wherein said data comprises: (i) a set of objective functions or loss functions grouped into non-overlapping subsets, wherein each objective function or loss function in said set of loss functions accepts first and second arguments; and (ii) a set of permissible vectors for each objective function or loss function in said set of said objective functions or loss functions; (b) set a current value of a vector; (c) receive a schedule of a set of scaling parameters; and (d) until a stopping criterion is met: (i) determine current values of said set of scaling parameters based at least in part on said schedule; (ii) select a subset of said objective functions or loss functions from said non-overlapping subsets; (iii) iterate the following steps for each objective function or loss function of said selected subset of said objective functions or loss functions: (1) generating,
• Said objective functions or loss functions may comprise one or more composite functions of said first and second arguments.
• Obtaining, by said digital computer, one or more gradients, wherein each of said one or more gradients is of said objective function or loss function taken with respect to said first argument may comprise iterative applications of a chain rule.
• Said chain rule may be performed using auto-differentiation.
• One or more argument functions of said composite functions may comprise differentiable feature extractors.
• Said differentiable feature extractors may comprise deep neural networks.
• a search direction may comprise using one or more of stochastic gradient descent (SGD), stochastic average gradient methods (SAG and SAGA), stochastic variance-reduced gradient (SVRG), or stochastic dual coordinate ascent (SDCA).
• Said sampling device may comprise a central processing unit and a memory unit coupled to said central processing unit.
• Said memory unit may comprise an application for obtaining said schedule of said scaling parameter and said data of said robust inference problem, wherein said application is configured to implement a Markov Chain Monte Carlo algorithm.
• Said sampling device may comprise a reconfigurable digital hardware, a central processing unit and a memory unit, said central processing unit and said memory unit coupled to said reconfigurable digital hardware.
• Said reconfigurable digital hardware may be configured to obtain said schedule of said scaling parameter and said data of said robust inference problem, and said reconfigurable digital hardware may be configured to implement a Markov Chain Monte Carlo algorithm.
• Said Markov Chain Monte Carlo algorithm may comprise simulated quantum annealing.
• Said data of said robust inference problem may be obtained from a noun phrase co reference resolution problem.
• Said data of said robust inference problem may be associated with active learning.
• Said data of said robust inference problem may be associated with an image tagging problem.
• Said data of said robust inference problem may be associated with a recommender system.
• Said schedule of said set of scaling parameters may be determined by a user or automatically by an algorithm,
• Said digital computer may be remotely located with respect to said sampling device.
• Said stopping criterion may be based at least in part on a magnitude of a distance between said current and said updated current vectors.
• Said first and second arguments may be independent, and said first argument may employ a continuous vector as its value, said second argument may employ a discrete vector as its value, and said set of permissible vectors may comprise a set of permissible discrete vectors.
• (1) may comprise generating, by said sampling device, one or more samples of discrete vectors, each sample of said one or more samples being generated from said set of permissible discrete vectors associated with said objective function or loss function, wherein each sample of said one or more samples is generated based on a probability distribution determined at least in part by said set of scaling parameters and said objective function or loss function, wherein said first argument of said objective function or loss function takes said current value of said continuous vector.
• Another aspect of the present disclosure provides a non-transitory computer readable medium comprising machine executable code that, upon execution by one or more computer processors, implements any of the methods above or elsewhere herein.
• Another aspect of the present disclosure provides a system comprising one or more computer processors and a non-transitory computer readable medium (e.g., computer memory) coupled thereto.
• the non-transitory computer readable medium comprises machine executable code that, upon execution by the one or more computer processors, implements any of the methods above or elsewhere herein.
• Fig 1 shows a flowchart for a non-limiting example of a method for stochastic optimization of a robust inference problem using a sampling device.
• setting a value of a vector e.g., a continuous vector or discrete vector can be setting values for every element of the vector. In other cases, setting a value of a vector can be setting values for one or more element of the vector.
• the present disclosure provides methods and systems that utilize a sampling device within a stochastic optimization method for solving robust inference problems.
• the methods and systems may provide a framework that enables efficient and robust optimization techniques in machine learning methods.
• Nonlimiting examples of machine-learning methods include: structural support vector machines (SSVMs), semi-supervised learning, and active learning. These methods may be useful in applications such as natural language processing (e.g., noun phrase coreference resolution), computer vision and image processing (e.g., image segmentation, image tagging), and data science (e.g. clustering of documents, group detection in a crowd, recommender systems, semi- supervised, and active learning, etc).
• natural language processing e.g., noun phrase coreference resolution
• computer vision and image processing e.g., image segmentation, image tagging
• data science e.g. clustering of documents, group detection in a crowd, recommender systems, semi- supervised, and active learning, etc.
• the robust inference problems herein may be associated with the robustness and/or accuracy of inference or assumption(s) under which a solution may be found for a computational task. In other words, how much deviation(s) from the inference or assumption may occur under which a solution may be obtained.
• the robustness of an inference method may pertain to its resistance to outliers, such as by training models that increase the probability of or guarantee good performance even for non-ideal or least confident predictions.
• the robust inference problems may be expressed as in equation (1):
• C & W 1 can be a set that defines admissible vectors in an n-dimensional real vector space
• R n ® R can be a function mapping vectors in an n-dimensional real vector space to a real value; every $ £i £z : R n x Y iiiz ® R can be a real-valued function; w £ ⁇ £z can be real numbers, Y iliz can be sets of vectors (such as finite set of vectors) from which argument y can take values, ii and h can be independent indexes in the range from 1 to m 1 and 1 to m 2 , respectively, and x andy can be two arguments (such as two independent arguments) which may be any vector from C and Y iliz , respectively.
• optimization of the robust inference problems described herein may refer to solving a minimization problem as described in equation (1) or any other problem that may be equivalent to or can be described by equation (1).
• the set C is a convex set, and the function /(x) and all functions gi l i2 are convex, and all real numbers w £i £z are positive. In such a case, the
• optimization problem described by equation (1) may become a convex optimization problem, allowing convex optimization methods to be employed in order to create convenient approximations of the global optimal solution. Such convex optimization methods may be efficiently implemented using procedures that scale with polynomial time.
• the optimization problem may be non-convex.
• the real numbers w £ ⁇ £z may be negative, as in a latent SSVM optimization problem, the optimization problem can be non-convex.
• the functions g il i2 are non-convex, as in the case that the functions gi l i2 correspond to neural networks, the optimization problem in (1) can be non-convex.
• g(x,y ) may be only subdifferentiable, and the subdifferential may be given by equation (2) as follows:
• dg(x) co ⁇ d x g(x, y)
• for all y such that g(x,y) g(x) ⁇ (2)
• co may be the notation for convex hull and d x may be the partial derivative with respect to x.
• Computing an element of this set i.e., computing dg(x)
• the methods for the stochastic optimization of robust inference problems described herein may include obtaining data of the robust inference problem. Such data may be pre-generated manually or automatically from raw data. Such data may be obtained by a digital computer. Such data may be utilized at least in part by the methods described herein for the stochastic optimization of robust inference problems, such as the robust inference problems described herein.
• each subset may contain at least about 1, 2, 3, 4, 5, 6, 10, 20, 30, 50, 60, 70, 80, 90 100, or more objective functions or loss functions.
• Each subset may contain at least about 1, 2, 3, 4, 5, 6, 10, 20, 30, 40, 50, 60, 70, 80, 90 100, or more objective functions or loss functions. In other cases, each subset may contain at most about 1, 2, 3, 4, 5, 6, 10, 20, 30, 40, 50, 60, 70, 80, 90 100, or less objective functions or loss functions.
• each objective function or loss function may accept a first argument x and a second argument y.
• the first and second arguments may be independent arguments.
• the first and second arguments may be dependent arguments.
• the first argument may take a continuous vector as its value
• the second argument may take a discrete vector as its value.
• Such data of the robust inference problem may include a linear-combination weight for each objective function or loss function: each loss function in equation (1) may be weighted by a weight, which can be a scalar w £ ⁇ £z that influences its contribution to the overall sum.
• Such data of the robust inference problem may include a set of permissible discrete vectors, l , for each loss function from which variable y may take values.
• Such data of the robust inference problem may include an initial continuous vector for the first argument of all loss functions in the first iteration.
• Such data of the robust inference problem may include an initial continuous vector for the first argument of one or more loss functions in the first iteration.
• the methods of the robust inference problem may set the current values of a set of scaling parameters in the first iteration of the iterative optimization process to be initial values of the set of scaling parameters.
• the initial values may be based at least in part on a schedule described herein.
• the current values of the set of scaling parameters may be updated based at least in part on the schedule, as described herein.
• the methods for solving the robust inference problem may receive or generate a schedule of the set of scaling parameters, from which the current values of the set of scaling parameters may be obtained from.
• the schedule may be determined a priori by the user or adjusted automatically by a selected algorithm. Initial values of the set of scaling parameters may be included in the schedule or based at least in part on the schedule.
• the schedule may be generated based on theoretical or empirical knowledge.
• the schedule may be generated using one or more algorithms or procedures selected from: a statistical algorithm or procedure, a pattern recognition algorithm or procedure, a machine learning algorithm or procedure, a deep learning algorithm or procedure, an artificial intelligence algorithm or procedure, a neural network, or the like.
• the schedule may be generated using historical values of the set of scaling parameters.
• the set of scaling parameters may take values of any real numbers.
• the set of scaling parameters may take values of any non-zero real numbers.
• the set of scaling parameters may take values of any positive and/or negative numbers.
• the set of scaling parameters may be used in a“softmax function” for solving the robust inference problems herein.
• The“Softmax function” can approximate the“max” function in equation
• (4. lb) may satisfy one or more of the following three statements:
• the gradient of the smooth approximation /i m may be obtained as an average or weighted average of the gradients.
• the gradient of the approximation /i m may be obtained as an expected value where i may be a random variable.
• the gradient of the approximation /i m may be obtained as an expected value where i may be a random variable that follows a Boltzmann distribution given by equation (5):
• - [1 may be the only element in a set of scaling parameters.
• the methods for stochastic optimization of robust inference problems may include iteratively performing one or more steps in each iteration of the iterative optimization process until at least one stopping criterion is met.
• Such stopping criterion may comprise a set of rules containing one or more rules for determining one or more of an accuracy, sensitivity, or specificity of a solution to the robust inference problem.
• the stopping criterion may be based at least in part on a magnitude of a distance between the current continuous vector in one iteration of the optimization process and the updated current continuous vectors in the same iteration or a different iteration, e.g., a previous or subsequent iteration.
• the one or more steps in the iterative optimization process may include determining current values of the set of scaling parameters.
• the current values may be based at least in part on the schedule of the set of scaling parameters.
• the one or more steps in the iterative optimization process may include selecting a subset of the objective functions or loss functions from the non-overlapping subsets, either non-repetitively or repetitively.
• one or more sub-steps may be performed for each objective function or loss function of the selected subset of the objective functions or loss functions.
• the one or more sub- steps may include generating one or more samples of discrete vectors for variable or argument y in equation (1).
• Each sample of the one or more samples may be selected from the set of permissible discrete vectors associated with the specific objective function or loss function.
• Each sample of the one or more samples may be generated based on a probability distribution. In some cases, the probability distribution may be determined at least in part by the set of scaling parameters and the specific objective function or loss function.
• the first argument of the loss function may take the current value of the continuous vector in the iteration. For instances, each sample may be generated according the probability distribution in equation (6):
• Each of the one or more samples may be generated using the sampling device disclosed herein. For example, a number of k samples may be generated and each sample may be selected from the set of permissible states so that k samples (y 1 ... ,y k ) G T £i£z , and wherein a choice of t ! G ⁇ 1, . .. , m - may represent the selected subset of the objective functions or loss functions, and i 2 G ⁇ 1, . . m 2 ) may represent the function in the selected subset.
• the probability distributions of the samples may be any single probability distribution or any combination of different probability distributions.
• the sampling device herein may include a random or pseudo-random generator that produces samples distributed according to a Boltzmann model.
• a sampling device may include hardware (e.g., a specialized computing device, a quantum processor, a non-classical computer, a quantum computing system, a digital computer, a digital processing device, or the like) and/or software that is configured to perform“Boltzmann sampling.”
• the approximated gradient then can be used to solve the robust inference problem with a pre-selected level of accuracy.
• the utilization of the sampling device and the connection of the sampling device may advantageously connect the gradient of the smoothed function approximation to a Boltzmann distribution, so that a complex robust inference problem can be solved.
• the sampling device may exhibit one or more properties determined by the mathematical definition of a Boltzmann distribution given in equation (6).
• the sampling device may include any hardware, software, or combination of hardware and software that may be configured to exhibit one or more properties determined by the mathematical definition of a Boltzmann
• the systems for solving a robust inference problem may include a sampling device for generating a number of samples.
• the sampling device may comprise a quantum processor and a quantum device control system for obtaining the schedule of the set of scaling parameters, the data of the robust inference problem, or their combination.
• the quantum processor may be coupled to a digital computer and to the quantum device control system.
• the quantum processor can comprise a plurality of qubits and a plurality of couplers, each coupler of the plurality of couplers for providing a communicative coupling at a crossing of two qubits of the plurality of qubits.
• the digital computer may be remotely located with respect to the sampling device.
• the quantum processor or quantum computer may comprise one or more adiabatic quantum computers, quantum gate arrays, one-way quantum computers, topological quantum computers, quantum Turing machines, superconductor-based quantum computers, trapped ion quantum computers, trapped atom quantum computers, optical lattices, quantum dot computers, spin-based quantum computers, spatial-based quantum computers, Loss-DiVincenzo quantum computers, nuclear magnetic resonance (NMR) based quantum computers, solution-state NMR quantum computers, solid-state NMR quantum computers, solid-state NMR Kane quantum computers, electrons-on-helium quantum computers, cavity- quantum-electrodynamics based quantum computers, molecular magnet quantum computers, fullerene- based quantum computers, linear optical quantum computers, diamond-based quantum computers, nitrogen vacancy (NV) diamond-based quantum computers, Bose-Einstein condensate-based quantum computers, transistor-based quantum computers, and rare-earth-metal-ion-doped inorganic crystal based quantum computers.
• the quantum processor or quantum computer may
• quantum annealers Ising solvers, optical parametric oscillators (OPO), and gate models of quantum computing.
• OPO optical parametric oscillators
• the quantum processor or quantum computer may comprise one or more qubits.
• the one or more qubits may comprise superconducting qubits, trapped ion qubits, trapped atom qubits, photon qubits, quantum dot qubits, electron spin-based qubits, nuclear spin-based qubits, molecular magnet qubits, fullerene-based qubits, diamond-based qubits, nitrogen vacancy (NV) diamond-based qubits, Bose- Einstein condensate-based qubits, transistor-based qubits, or rare-earth-metal-ion-doped inorganic crystal based qubits.
• the sampling device may comprise a network of optical parametric oscillators, in which the network includes an optical device configured to receive energy from an optical energy source and generate a plurality of optical parametric oscillators; and a plurality of coupling devices, each of which controllably couples an optical parametric oscillator of the plurality of optical parametric oscillators.
• the sampling device may include a network of optical parametric oscillators simulating two-body, three-body, or many-body interactions via interference of the optical pulses relevant to a reference phase.
• the sampling device may include one or more physical system with tunable and/or controllable many-body interactions that can stay close to its thermal equilibrium or approach its steady states.
• the systems for solving a robust inference problem may include a digital computer, or use of the same.
• the sampling device may include a digital computer, a central processing unit and a memory unit coupled to the central processing unit.
• the sampling device may include an application, a software module, a computer program, a user console, or use of the same, for obtaining the schedule of the scaling parameter, the data of the robust inference problem, or a combination thereof.
• the application, software module, or use of the same may be adapted for performing a Monte Carlo based algorithm.
• the Monte Carlo based algorithm may include Simulated Annealing, Simulated Quantum Annealing, Gibbs Sampling, or any combination thereof.
• the sampling device may include a reconfigurable digital hardware, a central processing unit and a memory unit with the central processing unit and the memory unit coupled to the
• the reconfigurable digital hardware may be adapted for obtaining the schedule of the scaling parameter, the data of the robust inference problem or a combination thereof.
• the reconfigurable digital hardware may be adapted to perform a Monte Carlo based algorithm.
• the Monte Carlo based algorithm may include Simulated Annealing, Simulated Quantum Annealing, Gibbs Sampling, or any combination thereof.
• the stochastic optimization of the robust inference problem herein may be associated with training a structured support vector machine (SSVM).
• SSVM structured support vector machine
• the stochastic optimization of the robust inference problem may be associated with image segmentation, image tagging and/or
• the stochastic optimization of the robust inference problem may be associated with a dual of the basis pursuit problem from compressed sensing.
• the stochastic optimization of the robust inference problem may be associated with unsupervised learning, semi- supervised learning, supervised learning, and/or active learning.
• the one or more sub-steps may include obtaining a gradient of the objective function or loss function taken with respect to the first argument x, wherein the first argument of the objective function or loss function may take the current values of the continuous vector, and the second argument of the objective function or loss function can take value of a selected sample.
• the k samples (y 1 ... , y k ) e y may be generated using a sampling device according to the probabilities using equation (6), where x may be held fixed.
• the index j can be in the range from 1 to &
• the gradient of the function g k t (x, j ) may be evaluated with respect to the continuous variables x evaluated at their current value.
• y may be generated using equation (7):
• k gradients can be generated with each gradient for one of the k samples.
• Each gradient may be obtained with the first argument x taking the same current continuous vector and the second argument y of the objective function or loss function taking values of a selected sample.
• the gradient may be obtained using a digital computer using the samples generated by a sampling device.
• the sampling device may comprise a digital computer, a quantum computer, or any other digital processing device and/or devices.
• the other digital processing devices may include but are not limited to: a hybrid computer including at least a digital computer and a quantum computer.
• the one or more sub-steps may include obtaining an average of the one or more gradients obtained in equation (7) using equation (8):
• k may be an integer greater than one. If k equals one, the average of the one or more gradients may be equal to the single gradient.
• the one or more steps in the iterative optimization process may include obtaining a summation and/or a partial summation of the averages of the one or more gradients, wherein the summation may be for all objective functions or loss functions in the selected subset of the objective functions or loss functions, and the partial summation may be for more than one objective functions or loss functions in the selected subset of the objective functions or loss functions.
• the summation and/or partial summation may be a linear combination of the gradient averages as in equation (9):
• a selected subset of objective functions or loss functions may contain four objective functions or loss functions; and for each objective function or loss function, an average of gradients may be obtained.
• the summation herein may include adding up 4 different averages of gradients multiplied by its associated weight, while the partial summation herein may include adding up any 2 or 3 different averages of gradients multiplied by its associated weight. If there is only one objective function or loss function in the selected subset, the sum may be the average of gradients multiplied by its associated weight for the one objective function or loss function.
• a selected subset of objective functions or loss functions may contain at least about 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, or more objective functions or loss functions.
• a selected subset of objective functions or loss functions may contain at most about 100, 90, 80, 70, 60, 50, 40, 30, 20,
• a selected subset of objective functions or loss functions may contain a number of objective functions or loss functions that is within a range defined by any two of the preceding values.
• Fig. 1 shows a flowchart for a non-limiting example of a method 100 for stochastic optimization of a robust inference problem using a sampling device.
• the method 100 may comprise receiving (for instance, by a digital computer data of the robust inference problem.
• the data may comprise a set of objective functions or loss functions grouped into non-overlapping subsets. Each objective function or loss function in the set of loss functions may accept first and second arguments.
• the data may further comprise: a set of permissible vectors for each objective function or loss function in the set of objective functions or loss functions.
• the method 100 may comprise receiving (for instance, by the digital computer), a schedule of a set of scaling parameters.
• the method 100 may comprise selecting a subset of the objective functions or loss functions from the non-overlapping subsets.
• the method 100 may comprise computing (for instance, by the digital computer) a search direction.
• the search direction may be based at least in part on one or more of: vl) the summation or the partial summation of the averages of the one or more gradients; v2) the current values of the set of scaling parameters; v3) at least part of a history of the summation or partial summation of the averages of the one or more gradients; and v4) at least part of a history of the values of the set of scaling parameters.
• the method 100 may comprise providing the current value of the continuous vector.
• Computing for instance, by the digital computer
• a search direction may comprise using one or more of stochastic gradient descent (SGD), stochastic average gradient methods (SAG and SAGA), stochastic variance-reduced gradient (SVRG), and/or stochastic dual coordinate ascent (SDCA).
• SGD stochastic gradient descent
• SAGA stochastic average gradient methods
• SVRG stochastic variance-reduced gradient
• SDCA stochastic dual coordinate ascent
• the network may comprise an optical device configured to receive energy from an optical energy source and generate a plurality of optical parametric oscillators and a plurality of coupling devices, each of which controllably couples an optical parametric oscillator of the plurality of optical parametric oscillators.
• the sampling device may comprise a central processing unit and a memory unit coupled to the central processing unit.
• the memory unit may comprise an application for obtaining the schedule of the scaling parameter and the data of the robust inference problem, and the application may be configured to implement a Markov Chain Monte Carlo algorithm.
• the sampling device may comprise a reconfigurable digital hardware, a central processing unit and a memory unit. The central processing unit and the memory unit may be coupled to the reconfigurable digital hardware.
• the reconfigurable digital hardware may be configured to obtain the schedule of the scaling parameter and the data of the robust inference problem, and the reconfigurable digital hardware may be configured to implement a Markov Chain Monte Carlo algorithm.
• the Markov Chain Monte Carlo algorithm may comprise simulated quantum annealing.
• the Markov Chain Monte Carlo algorithm may comprise simulated annealing.
• the Markov Chain Monte Carlo algorithm may comprise Gibbs sampling.
• the display device 304 can include a user interface (UI).
• UI user interface
• Examples of UTs include, without limitation, a graphical user interface (GUI) and web-based user interface.
• GUI graphical user interface
• the CPU 302 may be used for processing computer instructions. Various embodiments of the CPU 302 may be provided.
• the central processing unit 302 may be a CPU Core ⁇ 7-3820 running at 3.6 GHz and manufactured by Intel (TM) , for example.
• the display device 304 can be used for displaying data to a user.
• the skilled addressee will appreciate that various types of display device 304 may be used.
• the display device 304 may be a liquid-crystal display (LCD) monitor.
• the display device 304 may have a touchscreen, such as, for example, a capacitive or resistive touchscreen.
• the memory unit 312 may be used for storing computer-executable instructions.
• the memory unit 312 may comprise an operating system module 314.
• the operating system module 314 may be of various types. In some embodiments, the operating system module 314 may be OS X Yosemite manufactured by Apple (TM) .
• the memory unit 312 can further comprise one or more applications.
• One or more of the central processing unit 302, the display device 304, the input devices 306, the communication ports 308 and the memory unit 312 may be interconnected via the data bus 310.
• the system 202 may further comprise a network interface card (NIC) 322.
• NIC network interface card
• the application 320 can send the appropriate signals along the data bus 310 into NIC 322.
• NIC 322 in turn, may send such information to quantum device control system 324.
• the quantum computing system 204 may comprise a plurality of quantum bits and a plurality of coupling devices. Further description of the quantum computing system 204 is disclosed in, for example, U.S. Patent Publication No. 2006/0225165, which is entirely incorporated herein by reference.
• Methods described herein can be implemented by way of machine (e.g., computer processor) executable code stored on an electronic storage location of the computer system 200, such as, for example, on the memory unit 312 or an electronic storage unit.
• the machine executable or machine readable code can be provided in the form of software.
• the code can be executed by the CPU 302.
• the code can be retrieved from the electronic storage unit and stored on the memory unit 312 for ready access by the CPU 302.
• the electronic storage unit can be precluded, and machine-executable instructions are stored on memory unit 312.
• “Storage” type media can include any or all of the tangible memory of the computers, processors or the like, or associated modules thereof, such as various semiconductor memories, tape drives, disk drives and the like, which may provide non-transitory storage at any time for the software programming. All or portions of the software may at times be communicated through the Internet or various other telecommunication networks. Such communications, for example, may enable loading of the software from one computer or processor into another, for example, from a management server or host computer into the computer platform of an application server.
• another type of media that may bear the software elements includes optical, electrical and electromagnetic waves, such as used across physical interfaces between local devices, through wired electrical and/on optical landline networks and/or over various air-links.
• a machine readable medium such as computer-executable code
• a tangible storage medium such as computer-executable code
• Non-volatile storage media include, for example, optical or magnetic disks, such as any of the storage devices in any computer(s) or the like, such as may be used to implement the databases, etc. shown in the drawings.
• Volatile storage media include dynamic memory, such as main memory of such a computer platform.
• Tangible transmission media include coaxial cables; copper wire and fiber optics, including the wires that comprise a bus within a computer system.
• Carrier-wave transmission media may take the form of electric or electromagnetic signals, or acoustic or light waves such as those generated during radio frequency (RF) and infrared (IR) data communications.
• RF radio frequency
• IR infrared
• Common forms of computer-readable media therefore include for example: a floppy disk, a flexible disk, hard disk, magnetic tape, any other magnetic medium, a CD-ROM, DVD or DVD-ROM, any other optical medium, punch cards paper tape, any other physical storage medium with patterns of holes, a RAM, a ROM, a PROM and EPROM, a FLASH-EPROM, any other memory chip or cartridge, a carrier wave transporting data or instructions, cables or links transporting such a carrier wave, or any other medium from which a computer may read programming code and/or data.
• Many of these forms of computer readable media may be involved in carrying one or more sequences of one or more instructions to a processor for execution.
• about 100 meters represents a range of 95 meters to 105 meters (which is +/- 5% of 100 meters), 90 meters to 110 meters (which is +/- 10% of 100 meters), or 85 meters to 115 meters (which is +/- 15% of 100 meters) depending on the embodiments.

Landscapes

• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• Theoretical Computer Science (AREA)
• General Physics & Mathematics (AREA)
• Data Mining & Analysis (AREA)
• Mathematical Physics (AREA)
• General Engineering & Computer Science (AREA)
• Software Systems (AREA)
• Evolutionary Computation (AREA)
• Artificial Intelligence (AREA)
• Mathematical Analysis (AREA)
• Mathematical Optimization (AREA)
• Pure & Applied Mathematics (AREA)
• Computational Mathematics (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Computing Systems (AREA)
• Life Sciences & Earth Sciences (AREA)
• Algebra (AREA)
• Bioinformatics & Cheminformatics (AREA)
• Evolutionary Biology (AREA)
• Bioinformatics & Computational Biology (AREA)
• Operations Research (AREA)
• Databases & Information Systems (AREA)
• Computational Linguistics (AREA)
• Probability & Statistics with Applications (AREA)
• Medical Informatics (AREA)
• Nonlinear Science (AREA)
• Health & Medical Sciences (AREA)
• Biomedical Technology (AREA)
• Biophysics (AREA)
• General Health & Medical Sciences (AREA)
• Molecular Biology (AREA)
• Complex Calculations (AREA)
• Condensed Matter Physics & Semiconductors (AREA)

Priority Applications (5)

Applications Claiming Priority (4)

Related Child Applications (1)

Publications (1)

Family

ID=66664257

Family Applications (1)

Country Status (6)

Cited By (9)

Families Citing this family (11)

Citations (2)

Family Cites Families (230)

• 2018
  • 2018-11-30 WO PCT/CA2018/051534 patent/WO2019104443A1/en not_active Ceased
  • 2018-11-30 EP EP18884196.9A patent/EP3718026B1/en active Active
  • 2018-11-30 CN CN201880088360.8A patent/CN111670438B/zh active Active
  • 2018-11-30 CA CA3083008A patent/CA3083008A1/en active Pending
  • 2018-11-30 JP JP2020529188A patent/JP7288905B2/ja active Active
• 2020
  • 2020-05-29 US US16/888,419 patent/US12423374B2/en active Active

Patent Citations (2)

Non-Patent Citations (3)

Also Published As

Similar Documents

Legal Events