GB2572975A

GB2572975A - Quantum bundle adjustment

Info

Publication number: GB2572975A
Application number: GB1806283.6A
Authority: GB
Inventors: Idini Andrea; Heijnen Huub
Original assignee: Scape Tech Ltd; Scape Technologies Ltd
Current assignee: Scape Tech Ltd; Scape Technologies Ltd
Priority date: 2018-04-17
Filing date: 2018-04-17
Publication date: 2019-10-23
Also published as: WO2019202042A1; GB201806283D0

Abstract

The present invention provides a method of solving the problem of bundle adjustment using a quantum computer. More particularly, the present invention relates to translating the problem of bundle adjustment to find a minimum energy state solution. There is provided a method of bundle adjustment for a quantum computer, comprising: receiving a first data input, generating a first state for the quantum computer and associated with the first data input, applying one or more variational quantum eigensolver and/or derived algorithms e.g. quantum-classical hybrid algorithms, quantum simulators possibly including an annealing function, to the first state, and evolving the first state to output a minimum energy state. Data in relation to a minimum energy state may be provided by a classical addition computer module and may comprise the first data input. The first state may comprise a cost function or a bundle adjustment operator and the first data input may be one or more frames of image data that may be 3D data and such data may be embedded in a quantum representation.

Description

Field

The present invention relates to solving the problem of bundle adjustment using a quantum computer. More particularly, the present invention relates to translating the problem of bundle adjustment to find a minimum energy state solution.

Background

The umbrella term of Computer Vision relates to the problem of conveying structure and meaning of the three-dimensional (3D) world to machines. Within this context, several algorithms have been developed to use an input comprising RGB colour model of video and/or photo feeds, as well as other inputs such as time of flight cameras, to reconstruct the 3D world in a computer model.

One of the steps conventionally considered necessary to perform such a reconstruction comprises finding optimal parameters relating position and characteristics of one or more cameras and/or sensors feeding the input. Doing so, it is possible to construct a faithful 3D model. This optimisation process is known as bundle adjustment.

In the last 10 years, there has been a research effort to increase dimension and reliability of the reconstruction of 3D objects. Bundle adjustment is a considerable bottleneck of this process, requiring significant computational resources and approximations for larger scale solutions. This problem has been addressed by increasing the efficiency of the single computations and accelerating them on multicore and graphics processing units (GPUs), and eventually in distributing the load several computing clusters. However, these achievements are limited to classical, transistor based, computers. The number of operations needed to reach a solution continues to increase significantly with the size of the problem.

Summary of Invention

Aspects and/or embodiments seek to provide a method, system, and apparatus for solving the problem of bundle adjustment using a quantum computer.

According to a first aspect, there is provided a method of bundle adjustment for a quantum computer, comprising: receiving a first data input; generating a first state for the quantum computer and associated with the first data input; applying one or more variational quantum eigensolver and/or derived algorithms to the first state; and evolving the first state to output a minimum energy state.

Quantum computing in the cloud is becoming increasingly common and hence economically viable. Therefore, it may become beneficial to use such a platform for large scale reconstruction. Reconstruction of a 3D structure may be used for the purpose of, for example augmented reality, autonomous navigation, and/or mapping. Application of a quantum variational algorithm to find the minima in parameter space of the bundle adjustment problem may provide an efficient solution, by utilising one or more algorithms with weak or no dependence on the size of the problem, and thereby enable large scale reconstruction fewer operations. The entire problem may therefore be solved in less time and/or with greater precision than when a classical computer arrangement is used exclusively. Using conventional terminology, the use of the word “state” may be used interchangeably with the word “function”.

Optionally, data from the classical addition computer module comprises data in relation to the minimum energy state.

Analysis to find the minimum energy state allows for any absolute minima, and/or relative parameters, to be found and hence the bundle adjustment problem to be solved.

Optionally, the arrangement disclosed herein further comprises the step of applying data from a classical addition computer module as weights to the first state associated with the first data input.

A classical computer module may be obtained at a reduced cost and with wider availability than a quantum computer module. Further, a classical computer module can still provide an efficient and sufficiently fast means for solving certain problems.

Optionally, the first data input comprises data from the classical addition computer module.

The addition of data from the classical addition computer module can provide a starting point for the analysis disclosed herein which is closer to the ultimate solution to the bundle adjustment problem. Therefore, fewer iterations are required to obtain a solution within a predetermined threshold of accuracy, and hence the time taken for the analysis is reduced.

Optionally, the one or more variational quantum eigensolver algorithms comprise quantumclassical hybrid algorithms. Optionally, the first state comprises a cost function. Optionally, the one or more algorithms applied to the first state comprise one or more variational quantum eigensolver algorithms. Optionally, the one or more algorithms applied to the first state comprise one or more quantum-classical hybrid algorithms. Optionally, the one or more algorithms applied to the first state comprise one or more quantum simulators, optionally comprising an annealing function. Optionally, the first state comprises a bundle adjustment operator. Optionally, the one or more algorithms applied to the first state comprise one or more variational quantum algorithms. Optionally, the one or more algorithms applied to the first state comprise one or more quantum-classical hybrid algorithms. Optionally, the one or more algorithms applied to the first state comprise one or simulator. Optionally, the one or more algorithms applied to the first state comprise pure universal quantum gate formalism.

The use of such algorithms, in particular the use of a cost function solved with a variational quantum eigensolver, can provide a preferred method of performing the method disclosed herein. This is due to their practicality, conventional adoption rate, and the economical nature of such a solution.

Optionally, the input data comprises one or more frames of image data. Optionally, the one or more frames of image data comprise 3-dimensional image data. Optionally, the one or more frames of image data comprise video data. Optionally, the one or more frames of image data comprise differential image data.

Image data, in particular video data, can provide a source of input data for bundle adjustment problems. Such data may be difficult and/or computationally expensive to process using conventional bundle adjustment methods.

Optionally, the image data is embedded in a quantum representation. Optionally, the application of the one or more variational quantum eigensolver or derived algorithms to the first state further comprises the use of one or more direct imaging techniques.

By embedding the image data in a quantum representation, the use of a quantum computer may be more efficiently applied.

According to a further aspect, there is provided a computer program product comprising software code and/or a computer readable medium for carrying out the method, system or functional requirements of any preceding claim.

It is understood that the method disclosed herein may be applied to a range of quantum computing applications, not necessarily limited to the problem of bundle adjustment.

Brief Description of Drawings

Embodiments will now be described, by way of example only and with reference to the accompanying drawings having like-reference numerals, in which:

Figure 1 shows the evaluation of a bundle adjustment problem;

Figure 2 shows the evolution of a problem to find a minimum energy state;

Figure 3 shows an example of hybrid optimisation algorithm;

Figure 4 shows a further example of problem solving using a quantum computation; and

Figure 5 shows a further example of algorithms used to minimise cost functions in general.

Specific Description

Referring to Figure 1, an embodiment will now be described. Quantum computing in the cloud is becoming increasingly common and hence economically viable. Therefore it may become beneficial to use such a platform for large scale reconstruction. Reconstruction of a 3D structure may be used for other purposes, for example augmented reality, autonomous navigation, and/or mapping. Application of a quantum variational algorithm to find the minima in parameter space of the bundle adjustment problem may provide an efficient solution, through the use of one or more algorithms with weak or no dependence on the size of the problem. Thereby large scale reconstructions are enabled using fewer operations. This may be implemented by translating the cost function of the problem into a quantum Hamiltonian, that will be evolved reaching the absolute optimal solution, as shown in the associated Figures. Figure 1 specifically shows where a bundle adjustment arrangement may be situated within a reconstruction pipeline.

2D and/or 3D measurement data is provided as input data to an image registrar and triangulator. Initial data relating to the pose and/or other parameters related to the input data may also be provided. Data from the image registrar and triangulator and the initial data is then provided to an outlier filter. The outlier filter processes any data passed through it, passing the processed data through to a bundle adjustment module. The bundle adjusted data may be passed to the outlier filter again as part of an iterative process. Once a predetermined threshold of accuracy has been reached, the bundle adjusted data is then passed to a final module, operable to receive an optimised sensor pose and/or parameters, as well as a 3D model.

A classical computer has transistor states and memory states in binary code, a coding system using the binary digits (“bits”) 0 and 1 to represent data. That can be a simple and efficient way to encode information, however may additionally be relatively limiting when elaborating it. For this reason, to store and multiply two floating points numbers, computers need several bits and binary operations. In the bundle adjustment problem, the limits of traditional binary computing imply severe constraints, namely:

1) The need of computational power that grows at least polynomially with the size of the reconstruction, imposing a practical limit on the size of the optimisation; and

2) The risk of getting stuck in local optimisation minima, without guarantee to explore the best possible solution.

In quantum computing, is possible to use algorithms that scale differently than the corresponding ones allowed in classical computing, thereby lifting or considerably improving the computational limit to the size and scope of the problem. It is also possible to use quantomechanical properties to explore the parameter space. Representations of the tunneling effect may be used to find the absolute minimum or minima of the cost function and thus find the absolute optimal solution.

Quantum computers exploit the nature of superposition, making use of the fact that a quantum bit (“qubit”) is not only either a 0 or 1, but also a weighted superposition of the two. Quantum algorithms make use of this additional information. A quantum computer can embed in the algorithms typical properties of quantum system, for example tunneling effects. For reasons related to the amount of information and special operations that are feasible, quantum optimisers are extremely fast in solving both convex and non-convex optimisation problems.

The evaluation of the eigenvalues of a bundle adjustment problem consists in finding the optimal parameters, that is the parameters that minimise a cost-function. In order to evaluate it in a quantum solver, the cost function for the reprojection error may be converted to an Isinglike Hamiltonian, or any Hamiltonian operable to solve the cost function for the reprojection error in the given quantum computation system, that represent the bundle adjustment matrix.

A Hamiltonian may be chosen which is fastest to solve the cost function for the reprojection error. This is the state preparation.

The output of an iteration can be used to setup the input of the following iteration. Further, an outlier filter may be placed between the output of the optimisation module and/or the addition module after the first step of the optimisation and the state re-input of the quantum eigensolver. This is operable to change the state providing the feedback loop. Outlier evaluation conventionally requires image evaluation, thus may be suitable for analysis by classical machines.

As shown in Figure 2, optimisation is feasible in several ways. Image features and other registered and validated data to be optimised are assembled in a database, before quantum bundle adjustment is applied. The quantum bundle adjustment comprises applying a quantum state preparation to define a cost function landscape. Once so prepared, the quantum state is evolved in such a way that the minimum energy state of the system is reached through extraction of energy and/or annealing to reach a cost maximum. One or more measures of such a state may find the absolute minimum and relative parameters. These parameters may then be used as part of a feedback loop to provide relevant information during the quantum state preparation to define the cost function landscape. Quantum-classical hybrid algorithms are a class of procedures applicable without excessive resources, since the quantum computation is separated from the classical data storage. A variational quantum eigensolver is a specific example of hybrid optimisation algorithm.

As shown in Figure 3, a variational quantum eigensolver comprising an annealing implementation can be used to optimise the state image by image based on camera parameters. Information is then joined and evaluated in a classical computer addition module. The speed of said variational quantum eigensolver is inversely proportional with the requested precision, and may be represented as O(p^_1), since several measures are needed to overcome statistical noise. The speed of the classical computing, on the other hand, is proportional to the size of the parameter space N, usually O(N³) for similar categories of problems.

The answer can eventually be used as ansatz in a classical algorithm for verification and to further refine the solution by passing the answer through an outlier filter and then back to the stage of quantum state preparation using features and camera parameters. The performance optimisation of the quantum and classical computing iterations follows the dimension of the phase space left by the quantum solver. Considering (p^N) is the size of the remaining phase space, may be possible to optimise a function of the quantum and classical computing costs per iteration.

Further optimisation can be obtained by considering explicitly projection and extrinsic camera properties, for example rotations and translation, as operators within the Hamiltonian. These optimisations are important to contain the number of necessary qubits, but more investigation is needed to provide a final idea of the possible implementation. Variational solvers are especially powerful for this field of application, since they can be implemented in specific systems devoted to minimisation. These systems may be referred to as quantum annealers. They may be less complex to develop, and already commercially sold [see “D-Wave” computers].

Another possible implementation, similar to that of Figure 2, comprises the use of a quantum computation of derivatives through the means of an equivalent Hamiltonian implementation, effectively implementing a Jacobian matrix in a quantum system. Several algorithms can be applied using this information, for example conjugate gradients or a form of the LevenbergMarquardt algorithm, as shown in Figure 4.

Other implementations of quantum computing might also be used. Quantum simulators are analogous systems that mock a system to be simulated, for example a lattice of atoms with tunable interactions used to replicate a crystalline structure of another atom. It may be possible to devise such a simulator for computer vision problems operable to provide a direct relation between a quantum and a classical system. In this embodiment the quantum bundle adjustment further comprises an evaluation of a first differential and subsequent higher derivates, for example by the means of a Hamiltonian representation. A preferred minimisation procedure, for example conjugate gradients, is then used to find a convergent solution, optionally passing that solution back to the evaluation stage to allow for a faster convergence.

Orthogonal groups in dimension n may be represented mathematically as O(n) whereby the group comprises one or more distance-preserving transformations of a Euclidian space. In this transformation, a fixed point is preserved and the group operation is provided through the use of one or more composing transformations. A subgroup of the orthogonal group O(n) is the “special orthogonal group”, represented as SO(n) and interchangeably referred to as the rotation group. Affine transformation, comprising a linear mapping method that preserves points, straight lines, and planes, may also be used. Bundle adjustment may be solved through a permutation of rotation, translation and projection operators.

Such components are analogous to operators in SO(3), 0(3), and Aff(3,n) groups respectively, which can manifest themselves in quantum system. The cost function may be set up as an energy landscape in a quantum simulator, and then cooled to extract energy from the system in order to reach a ground state. By measuring the properties of such ground state, the absolute optimal parameters can be found.

While some quantum systems may exhibit SO(3) and 0(3) behaviour, for example crystals and nuclei, affine transformation and projections may require greater computational expense to simulate. Affine transformations are continuous and heteromorphic and quantum systems usually are not. Open quantum systems can provide a solution to this problem. The simulations of affine transformations may also be performed, and optionally empowered through the use of quantum simulators arranged to handle quantum trinary bits (“trits”) instead of conventional qubits.

Universal quantum gates may simulate an entire problem directly using quantum information. This leads to several possible algorithms on images, arising from the fact of representing images with entangled qubits. This is accomplished, for example, using flexible representation of quantum images (FRQI) and novel enhanced quantum representation (NEQR). In relation to the quantum stored images, one or more image manipulation algorithms, such as quantum fast Fourier transform algorithms, can be applied with improved qualitative and speed performance.

A large number of qubits may be required for the application of the method disclosed herein. In the context of bundle adjustment, universal quantum computing may directly solve the bundle adjustment problem using an exact mathematical representation and consequent solution. Regarding quantum simulators, operators that apply SO(3) and 0(3) transformation, together with affine transformation, optionally through reversible gates, may enable efficient bundle adjustment. In particular, using universal quantum gates alongside other algorithms can be used to minimise cost functions in general, and bundle adjustment cost functions in particular. Such algorithms can use sampling and quantum automata, are analogous to simplex and genetic algorithms for classical computing that are particularly useful for finding optimal and sub-optimal solution to ill-defined problems, as exemplified in Figure 5. In that Figure specifically, the evolution of the solution is performed through orthogonal measurement, followed by evaluation by means of sampling-based algorithms and/or quantum automata. Optionally the solution is passed back to the evolution stage to allow for a solution to be reached more quickly.

Such a solution of bundle adjustment problems may be considered an equivalent of 0(1) instead of N polynomial time, where N is the space dimension. This may lead to considerable time savings for large problems, as well as ensuring the finding of an absolute minimum 5 instead of a local minimum.

Any system feature as described herein may also be provided as a method feature, and vice versa. As used herein, means plus function features may be expressed alternatively in terms of their corresponding structure.

Any feature in one aspect may be applied to other aspects, in any appropriate combination. In particular, method aspects may be applied to system aspects, and vice versa. Furthermore, any, some and/or all features in one aspect can be applied to any, some and/or all features in any other aspect, in any appropriate combination.

It should also be appreciated that particular combinations of the various features described and defined in any aspects can be implemented and/or supplied and/or used independently.

Claims

CLAIMS:

1. A method of bundle adjustment for a quantum computer, comprising:

receiving a first data input;

generating a first state for the quantum computer and associated with the first data input;

applying one or more variational quantum eigensolver and/or derived algorithms to the first state; and evolving the first state to output a minimum energy state.

2. The method of claim 1, wherein the data from the classical addition computer module comprises data in relation to the minimum energy state.

3. The method of any preceding claim, further comprising the step of applying data from a classical addition computer module as weights to the first state associated with the first data input.

4. The method of claim 3, wherein the first data input comprises data from the classical addition computer module.

5. The method of any preceding claim, wherein the one or more variational quantum eigensolver algorithms comprise quantum-classical hybrid algorithms.

6. The method of any preceding claim, wherein the first state comprises a cost function.

7. The method of claim 6, wherein the one or more algorithms applied to the first state comprise one or more variational quantum eigensolver algorithms.

8. The method of claim 6, wherein the one or more algorithms applied to the first state comprise one or more quantum-classical hybrid algorithms.

9. The method of claim 6, wherein the one or more algorithms applied to the first state comprise one or more quantum simulators, optionally comprising an annealing function.

10. The method of any one of claims 1 to 5, wherein the first state comprises a bundle adjustment operator.

11. The method of claim 10, wherein the one or more algorithms applied to the first state comprise one or more variational quantum algorithms.

12. The method of claim 10, wherein the one or more algorithms applied to the first state comprise one or more quantum-classical hybrid algorithms.

13. The method of claim 10, wherein the one or more algorithms applied to the first state comprise one or simulator.

14. The method of claim 10, wherein the one or more algorithms applied to the first state comprise pure universal quantum gate formalism.

15. The method of any preceding claim, wherein the input data comprises one or more frames of image data.

16. The method of claim 15, wherein the one or more frames of image data comprise 3dimensional image data.

17. The method of any one of claims 15 or 16, wherein the one or more frames of image data comprise video data.

18. The method of any one of claims 15 to 17, wherein the one or more frames of image data comprise differential image data.

19. The method of any one of claims 15 to 18, wherein the image data is embedded in a quantum representation.

20. The method of any preceding claim, wherein the application of the one or more variational quantum eigensolver or derived algorithms to the first state further comprises the use of one or more direct imaging techniques.

21. A computer program product comprising software code and/or a computer readable medium for carrying out the method, system or functional requirements of any preceding claim.