CN114323010A - Initial feature determination method and device, electronic equipment and storage medium - Google Patents

Abstract

The disclosure relates to an initial feature determination method, an initial feature determination device, electronic equipment and a storage medium, and relates to the technical field of computers. The method comprises the following steps: acquiring relative postures and position information of a target three-dimensional point in a plurality of images; and determining an initial velocity characteristic and an initial gravity characteristic according to the relative posture, the position information of the target three-dimensional point in the plurality of images, the posture of the first coordinate system of each image time relative to the initial first coordinate system, the time difference between each image time and the initial time, the time difference between the first image time and the second image time and the first acceleration integral between the first coordinate system of the first image time and the first coordinate system of the second image time. According to the method and the device, the electronic equipment can accurately and effectively determine the initial speed characteristic and the initial gravity characteristic, and further can accurately determine the motion track of the electronic equipment in the three-dimensional space.

Description

Initial feature determination method and device, electronic equipment and storage medium

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to an initial feature determination method, an initial feature determination apparatus, an electronic device, and a storage medium.

Background

Currently, in a visual inertial navigation odometer system, the motion trajectory of a device in a three-dimensional space can be determined by acquiring an initial state (e.g., an initial velocity, etc.) of the device (or a body). Specifically, it is generally assumed that the initial state of the device is a static state, i.e., the initial velocity of the device is 0, and the motion trajectory of the device in the three-dimensional space is determined.

However, in the existing application scenario, the initial state of the device is not necessarily static. If the initial speed of the device is still defined as 0, the initial state of the device may not be accurately reflected, and thus the motion trajectory of the device in the three-dimensional space may not be accurately determined.

Disclosure of Invention

The present disclosure provides an initial characteristic determination method, an initial characteristic determination device, an electronic device, and a storage medium, which solve the technical problem in the prior art that an initial speed of a device is defined as 0, and therefore an initial state of the device may not be accurately reflected, and further a motion trajectory of the device in a three-dimensional space cannot be accurately determined.

The technical scheme of the embodiment of the disclosure is as follows:

according to a first aspect of embodiments of the present disclosure, there is provided an initial feature determination method. The method can comprise the following steps: acquiring relative postures and position information of a target three-dimensional point in a plurality of images, wherein the relative postures are postures of a first coordinate system relative to a second coordinate system at the same time, the first coordinate system is a coordinate system corresponding to an image sensor included in electronic equipment, the second coordinate system is a coordinate system corresponding to an inertial sensor included in the electronic equipment, and the plurality of images are images respectively acquired by the image sensor at a plurality of image times, wherein each image time is the time when the image sensor acquires one image; determining a pose of a first coordinate system of each of the plurality of image moments relative to an initial first coordinate system, a time difference between each of the plurality of image moments and an initial moment, a time difference between a first image moment and a second image moment, and a first acceleration integral between the first coordinate system of the first image moment and the first coordinate system of the second image moment, the first image moment being one of the plurality of image moments, the second image moment being a moment of the plurality of image moments other than the first image moment, the initial moment being a first moment of the plurality of image moments, the initial first coordinate system of the initial moment; determining an initial velocity characteristic and an initial gravity characteristic according to the relative posture, the position information of the target three-dimensional point in a plurality of images, the posture of the first coordinate system of each image time relative to the initial first coordinate system, the time difference between each image time and the initial time, the time difference between the first image time and the second image time and the first acceleration integral between the first coordinate system of the first image time and the first coordinate system of the second image time, wherein the initial velocity characteristic is a velocity characteristic of the electronic device at the initial time, and the initial gravity characteristic is a gravity characteristic of the electronic device at the initial time.

Optionally, the pose of the first coordinate system of each of the plurality of image time instants with respect to the initial first coordinate system includes a rotation feature of the first coordinate system of each of the plurality of image time instants with respect to the initial first coordinate system, and the initial feature determination method further includes: acquiring the angular velocity of the electronic equipment at each data time in a plurality of data times, wherein the data times are the times when the inertial sensor acquires the velocity data of the electronic equipment; determining a time difference between any two adjacent data moments of the plurality of data moments; determining the rotation characteristic of the first coordinate system of each of the plurality of image time instants with respect to the initial first coordinate system specifically comprises: and determining the rotation characteristic of the first coordinate system of each image moment in the plurality of image moments relative to the initial first coordinate system according to the angular speed of the electronic equipment at each data moment in the plurality of data moments and the time difference between any two adjacent data moments in the plurality of data moments, wherein the time difference between any two adjacent image moments in the plurality of image moments is larger than the time difference between any two adjacent data moments.

Optionally, the relative pose includes a translation feature of the first coordinate system relative to the second coordinate system at the same time, and the initial feature determination method further includes: acquiring the acceleration of the electronic equipment at each data moment in the plurality of data moments; determining a rotation characteristic of each of the plurality of data moments relative to the initial first coordinate system; determining a first acceleration integral between the first coordinate system of the first image time and the first coordinate system of the second image time specifically comprises: determining a first acceleration integral between the first coordinate system of the first image time and the first coordinate system of the second image time according to the acceleration of the electronic device at each of the plurality of data times, the rotation characteristic of each of the plurality of data times relative to the initial first coordinate system, the time difference between any two adjacent data times, the rotation characteristic of the first coordinate system at each of the plurality of image times relative to the initial first coordinate system, and the translation characteristic of the first coordinate system at the same time relative to the second coordinate system.

Optionally, the determining, according to the angular velocity of the electronic device at each of the plurality of data time instants and the time difference between any two adjacent data time instants in the plurality of data time instants, the rotation characteristic of the first coordinate system at each of the plurality of image time instants relative to the initial first coordinate system specifically includes: and determining the rotation characteristic of the first coordinate system of the j image time relative to the initial first coordinate system according to the angular speed of the k data time and the time difference between any two adjacent data times in the plurality of data times, wherein the j image time is one of the plurality of image times, the k data time is one of n data times included between the i image time and the j image time, the i image time is a time except the j image time in the plurality of image times, i is more than or equal to 1, j is more than or equal to 2, n is more than or equal to 1, and k is more than or equal to 1 and less than or equal to n.

Optionally, the determining the first acceleration integral between the first coordinate system of the first image time and the first coordinate system of the second image time according to the acceleration of the electronic device at each of the plurality of data times, the rotation characteristic of each of the plurality of data times relative to the initial first coordinate system, the time difference between any two adjacent data times of the plurality of data times, the rotation characteristic of the first coordinate system at each of the plurality of image times relative to the initial first coordinate system, and the translation characteristic of the first coordinate system at the same time relative to the second coordinate system specifically includes: determining a second acceleration integral between a first coordinate system of a jth image time and the initial first coordinate system according to a time difference between any two adjacent data times of the plurality of data times, a rotation characteristic of a first coordinate system of a kth data time relative to the initial first coordinate system, an acceleration of the electronic device at the kth data time, a rotation characteristic of a first coordinate system of a first f data time relative to the initial first coordinate system, and an acceleration of the electronic device at the f data time, the kth data time being one of n data times included between an ith image time and the jth image time, the f data time being one of m data times included between the ith image time and the kth data time, the jth image time being one of the plurality of image times, the ith moment is a moment except the jth image moment in the plurality of image moments, i is more than or equal to 1, j is more than or equal to 2, n is more than or equal to 1, k is more than or equal to 1 and less than or equal to n, m is more than or equal to 1 and less than or equal to k, and f is more than or equal to 1 and less than or equal to m; and determining a first acceleration integral between the first coordinate system of the ith image moment and the first coordinate system of the jth image moment according to a second acceleration integral between the first coordinate system of the jth image moment and the initial first coordinate system, a second acceleration integral between the first coordinate system of the ith image moment and the initial first coordinate system, a rotation characteristic of the first coordinate system of the jth image moment relative to the initial first coordinate system, a rotation characteristic of the first coordinate system of the ith image moment relative to the initial first coordinate system and a translation characteristic of the first coordinate system at the same moment relative to the second coordinate system.

Optionally, the pose of the first coordinate system of each of the plurality of image moments with respect to the initial first coordinate system comprises a rotation feature and a translation feature of the first coordinate system of each of the plurality of image moments with respect to the initial first coordinate system, and the relative pose comprises a translation feature of the first coordinate system with respect to the second coordinate system at the same moment and a rotation feature of the first coordinate system with respect to the second coordinate system at the same moment; the determining an initial velocity characteristic and an initial gravity characteristic according to the relative pose, the position information of the target three-dimensional point in the plurality of images, the pose of the first coordinate system at each of the plurality of image times relative to the initial first coordinate system, the time difference between each of the image times and the initial time, the time difference between the first image time and the second image time, and the first acceleration integral between the first coordinate system at the first image time and the second image time specifically includes: according to a preset non-unit modular length normal vector, the translation feature of the first coordinate system of the ith image time relative to the initial first coordinate system, the rotation feature of the first coordinate system of the ith image time relative to the initial first coordinate system, the translation feature of the first coordinate system of the same time relative to the second coordinate system, the rotation feature of the first coordinate system of the same time relative to the second coordinate system, the position information of the target three-dimensional point in the ith image, the translation feature of the first coordinate system of the jth image time relative to the initial first coordinate system, the rotation feature of the first coordinate system of the jth image time relative to the initial first coordinate system, the position information of the target three-dimensional point in the jth image, the time difference between the ith image time and the jth image time, the time difference between the jth image time and the initial time, Determining the initial speed characteristic and the initial gravity characteristic by the time difference between the ith image moment and the initial moment and the first acceleration integral between the first coordinate system of the ith image moment and the first coordinate system of the jth image moment, wherein the jth image moment is one of the image moments, the ith moment is a moment except the jth image moment in the image moments, i is not less than 1, and j is not less than 2.

Optionally, the initial characteristic determining method further includes: and determining the preset non-unit mode length normal vector according to the unit normal vector of the target plane and the relative distance between the target plane and the initial first coordinate system, wherein the target plane is a plane corresponding to the target three-dimensional point.

According to a second aspect of the embodiments of the present disclosure, there is provided an initial feature determination apparatus. The apparatus may include: the device comprises an acquisition module and a determination module; the acquisition module is configured to acquire a relative posture and position information of a target three-dimensional point in a plurality of images, wherein the relative posture is a posture of a first coordinate system relative to a second coordinate system at the same time, the first coordinate system is a coordinate system corresponding to an image sensor included in the electronic equipment, the second coordinate system is a coordinate system corresponding to an inertial sensor included in the electronic equipment, and the plurality of images are respectively acquired by the image sensor at a plurality of image moments, and each image moment is a moment when the image sensor acquires one image; the determining module is configured to determine a pose of a first coordinate system of each of the plurality of image moments with respect to an initial first coordinate system, a time difference between each of the plurality of image moments and an initial moment, a time difference between a first image moment and a second image moment, and a first acceleration integral between a first coordinate system of the first image moment and a first coordinate system of the second image moment, the first image moment being one of the plurality of image moments, the second image moment being a moment of the plurality of image moments other than the first image moment, the initial moment being a first one of the plurality of image moments, the initial first coordinate system being the first coordinate system of the initial moment; the determining module is further configured to determine an initial velocity feature and an initial gravity feature according to the relative pose, the position information of the target three-dimensional point in the plurality of images, the pose of the first coordinate system of each of the plurality of image moments relative to the initial first coordinate system, the time difference between each of the plurality of image moments and the initial moment, the time difference between the first image moment and the second image moment, and the first acceleration integral between the first coordinate system of the first image moment and the first coordinate system of the second image moment, wherein the initial velocity feature is a velocity feature of the electronic device at the initial moment, and the initial gravity feature is a gravity feature of the electronic device at the initial moment.

Optionally, the pose of the first coordinate system of each of the plurality of image moments with respect to the initial first coordinate system comprises a rotation characteristic of the first coordinate system of each of the plurality of image moments with respect to the initial first coordinate system; the acquisition module is further configured to acquire the angular velocity of the electronic device at each of a plurality of data moments, wherein the plurality of data moments are moments when the inertial sensor acquires velocity data of the electronic device; the determining module is further configured to determine a time difference between any two adjacent data time instants in the plurality of data time instants; the determining module is specifically configured to determine a rotation characteristic of the first coordinate system of each of the plurality of image time instants relative to the initial first coordinate system according to an angular velocity of the electronic device at each of the plurality of data time instants and a time difference between any two adjacent data time instants, wherein the time difference between any two adjacent image time instants is greater than the time difference between any two adjacent data time instants.

Optionally, the relative pose includes a translation feature of the first coordinate system relative to the second coordinate system at the same time;

the acquisition module is further configured to acquire the acceleration of the electronic device at each of the plurality of data moments; the determination module is further configured to determine a rotation characteristic of each of the plurality of data moments relative to the initial first coordinate system; the determining module is further specifically configured to determine a first acceleration integral between the first coordinate system of the first image time and the first coordinate system of the second image time according to the acceleration of the electronic device at each of the plurality of data times, the rotation characteristic of each of the plurality of data times relative to the initial first coordinate system, the time difference between any two adjacent data times of the plurality of data times, the rotation characteristic of the first coordinate system at each of the plurality of image times relative to the initial first coordinate system, and the translation characteristic of the first coordinate system at the same time relative to the second coordinate system.

Optionally, the determining module is specifically further configured to determine a rotation feature of the first coordinate system of the jth image time with respect to the initial first coordinate system according to an angular velocity of the kth data time and a time difference between any two adjacent data times in the plurality of data times, where the kth data time is one of n data times included between the ith image time and the jth image time, the ith image time is a time of the plurality of image times other than the jth image time, i is greater than or equal to 1, j is greater than or equal to 2, n is greater than or equal to 1, and 1 is greater than or equal to k is less than or equal to n.

Optionally, the determining module is further specifically configured to determine a second acceleration integral between the first coordinate system of the jth image time and the initial first coordinate system according to a time difference between any two adjacent data times in the plurality of data times, a rotation characteristic of the first coordinate system of the kth data time relative to the initial first coordinate system, an acceleration of the electronic device at the kth data time, a rotation characteristic of the first coordinate system of the fth data time relative to the initial first coordinate system, and an acceleration of the electronic device at the fth data time, the kth data time being one of n data times included between the ith image time and the jth image time, the fth data time being one of m data times included between the ith image time and the kth data time, the j image time is one of the image times, the i time is the time except the j image time, i is more than or equal to 1, j is more than or equal to 2, n is more than or equal to 1, k is more than or equal to 1 and less than or equal to n, m is more than or equal to 1 and less than or equal to k, and f is more than or equal to 1 and less than or equal to m; the determining module is specifically further configured to determine a first acceleration integral between the first coordinate system of the ith image time and the first coordinate system of the jth image time according to a second acceleration integral between the first coordinate system of the jth image time and the initial first coordinate system, a rotation feature of the first coordinate system of the jth image time relative to the initial first coordinate system, a rotation feature of the first coordinate system of the ith image time relative to the initial first coordinate system, and a translation feature of the first coordinate system at the same time relative to the second coordinate system.

Optionally, the pose of the first coordinate system of each of the plurality of image moments with respect to the initial first coordinate system comprises a rotation feature and a translation feature of the first coordinate system of each of the plurality of image moments with respect to the initial first coordinate system, and the relative pose comprises a translation feature of the first coordinate system with respect to the second coordinate system at the same moment and a rotation feature of the first coordinate system with respect to the second coordinate system at the same moment; the determining module is specifically configured to determine a translation feature of the first coordinate system at the ith image time relative to the initial first coordinate system, a rotation feature of the first coordinate system at the ith image time relative to the initial first coordinate system, a translation feature of the first coordinate system at the same time relative to the second coordinate system, a rotation feature of the first coordinate system at the same time relative to the second coordinate system, position information of the target three-dimensional point in the ith image, a translation feature of the first coordinate system at the jth image time relative to the initial first coordinate system, a rotation feature of the first coordinate system at the jth image time relative to the initial first coordinate system, position information of the target three-dimensional point in the jth image, a time difference between the ith image time and the jth image time, a translation feature of the first coordinate system at the jth image time relative to the initial first coordinate system, a translation feature of the target three-dimensional point in the jth image, a translation feature of the jth image time relative to the initial first coordinate system, a translation feature of the jth image time, Determining the initial speed characteristic and the initial gravity characteristic by the time difference between the jth image time and the initial time, the time difference between the ith image time and the initial time and the first acceleration integral between the first coordinate system of the ith image time and the first coordinate system of the jth image time, wherein the jth image time is one of the image times, the ith time is a time except the jth image time in the image times, i is not less than 1, and j is not less than 2.

Optionally, the determining module is further configured to determine the preset non-unit modular length normal vector according to a unit normal vector of a target plane and a relative distance between the target plane and the initial first coordinate system, where the target plane is a plane corresponding to the target three-dimensional point.

According to a third aspect of embodiments of the present disclosure, there is provided an electronic device, which may include: a processor and a memory configured to store processor-executable instructions; wherein the processor is configured to execute the instructions to implement any of the above described optional initial feature determination methods of the first aspect.

According to a fourth aspect of embodiments of the present disclosure, there is provided a computer-readable storage medium having instructions stored thereon, which, when executed by an electronic device, enable the electronic device to perform any one of the above-mentioned optional initial feature determination methods of the first aspect.

According to a fifth aspect of embodiments of the present disclosure, there is provided a computer program product comprising computer instructions which, when run on an electronic device, cause the electronic device to perform the optional initial feature determination method as any one of the first aspects.

The technical scheme provided by the embodiment of the disclosure at least brings the following beneficial effects:

based on any one of the above aspects, in the present disclosure, the electronic device may acquire the relative pose and the position information of the target three-dimensional point in the plurality of images, and then determine the pose of the first coordinate system of each of the plurality of image times relative to the initial first coordinate system, the time difference between each of the plurality of image times and the initial time, the time difference between the first image time and the second image time, and the first acceleration integral between the first coordinate system of the first image time and the first coordinate system of the second image time, thereby determining the speed characteristic of the electronic device at the initial time and the gravity characteristic of the electronic device at the initial time. In the embodiment of the disclosure, the electronic device determines the posture of the first coordinate system at different image moments relative to the initial first coordinate system, that is, determines the posture change condition of the image sensor at different image moments; and determining a first acceleration integral between the first coordinate systems of any two image moments (i.e. a first image moment and a second image moment) of the plurality of image moments and a time difference between the any two image moments; the initial speed characteristic and the initial gravity characteristic which can reflect the initial state of the electronic equipment can be accurately and effectively determined, and then the motion trail of the electronic equipment in the three-dimensional space can be accurately determined based on the initial speed characteristic and the initial gravity characteristic.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure and are not to be construed as limiting the disclosure.

Fig. 1 is a schematic flow chart illustrating an initial feature determination method provided by an embodiment of the present disclosure;

fig. 2 is a schematic flow chart diagram illustrating a further initial feature determination method provided by the embodiment of the present disclosure;

fig. 3 is a flowchart illustrating a further initial feature determination method provided by an embodiment of the present disclosure;

fig. 4 is a flowchart illustrating a further initial feature determination method provided by the embodiment of the present disclosure;

fig. 5 is a flowchart illustrating a further initial feature determination method provided by the embodiment of the present disclosure;

fig. 6 is a flowchart illustrating a further initial feature determination method provided by the embodiment of the present disclosure;

fig. 7 is a flowchart illustrating a further initial feature determination method provided by the embodiment of the present disclosure;

fig. 8 is a schematic structural diagram of an initial feature determination apparatus provided in an embodiment of the present disclosure;

fig. 9 is a schematic structural diagram of another initial characteristic determining apparatus provided in an embodiment of the present disclosure.

Detailed Description

In order to make the technical solutions of the present disclosure better understood by those of ordinary skill in the art, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the disclosure described herein are capable of operation in sequences other than those illustrated or otherwise described herein. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, and/or components.

The data to which the present disclosure relates may be data that is authorized by a user or sufficiently authorized by parties.

As described in the background art, since the initial speed of the device is generally defined as 0 in the prior art but the initial state of the device is not necessarily static, the initial state of the device may not be accurately reflected, and thus the motion trajectory of the device in the three-dimensional space may not be accurately determined. Based on this, the embodiments of the present disclosure provide an initial characteristic determining method, which can accurately and effectively determine an initial speed characteristic and an initial gravity characteristic that can reflect an initial state of an electronic device, and further can accurately determine a motion trajectory of the electronic device in a three-dimensional space based on the initial speed characteristic and the initial gravity characteristic.

The initial feature determination method, device, electronic device and storage medium provided by the embodiment of the disclosure are applied to a scene in which a motion trajectory of a certain device (e.g., an electronic device such as a mobile phone) needs to be determined. When the electronic device acquires the relative posture and the position information of the target three-dimensional point in the plurality of images, the speed characteristic of the electronic device at the initial moment and the gravity characteristic of the electronic device at the initial moment can be determined according to the method provided by the embodiment of the disclosure.

It should be noted that the electronic device executing the initial feature determination method provided by the present disclosure may be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a Personal Digital Assistant (PDA), an Augmented Reality (AR) \ Virtual Reality (VR) device, and other devices that can be installed and used with a content community application, and the present disclosure does not particularly limit the specific form of the electronic device. The system can be used for man-machine interaction with a user through one or more modes of a keyboard, a touch pad, a touch screen, a remote controller, voice interaction or handwriting equipment and the like.

As shown in fig. 1, the initial feature determination method provided by the embodiment of the present disclosure may include S101 to S103.

S101, the electronic equipment acquires the relative posture and the position information of the target three-dimensional point in a plurality of images.

The relative posture is a posture of a first coordinate system relative to a second coordinate system at the same time, the first coordinate system is a coordinate system corresponding to an image sensor included in the electronic equipment, the second coordinate system is a coordinate system corresponding to an inertial sensor included in the electronic equipment, and the plurality of images are images respectively acquired by the image sensor at a plurality of image moments, wherein each image moment is a moment when the image sensor acquires one image.

It should be understood that the image sensor and the inertial sensor are included in the electronic device. When the electronic device is in a continuous motion state, the image sensor acquires one image at each of the plurality of image moments to obtain the plurality of images, where the plurality of images may be understood as scene images or environment images acquired by the electronic device at the plurality of image moments, and the target three-dimensional point is a feature point existing in the plurality of scene images in common, or may be understood as the target three-dimensional point included in the plurality of images.

It is understood that, since the electronic device is in a state of continuous motion, the image sensor and the inertial sensor are also in a state of continuous motion, that is, the positions and attitudes of the image sensor and the inertial sensor are constantly changing. And since the image sensor and the inertial sensor are both included in the electronic device and located at different positions in the electronic device (specifically, located at different positions in the internal structure of the electronic device), different coordinate systems may be defined for the image sensor and the inertial sensor in the embodiment of the present disclosure, that is, the first coordinate system and the second coordinate system are defined, so as to combine the first coordinate system and the second coordinate system to determine the motion trajectory of the electronic device.

It should be noted that, after the electronic device is manufactured (or produced), the relative position relationship between the image sensor and the inertial sensor in the electronic device is fixed. Therefore, at the same time (or when the electronic device is located at any position), the posture of the first coordinate system relative to the second coordinate system (i.e. the relative posture) remains unchanged. The electronic device can directly measure and acquire the relative pose.

Alternatively, the image sensor in the embodiment of the present disclosure may be a camera or a camera, and the inertial sensor may be an Inertial Measurement Unit (IMU).

S102, the electronic device determines the posture of the first coordinate system of each image time in the plurality of image times relative to the initial first coordinate system, the time difference between each image time in the plurality of image times and the initial time, the time difference between the first image time and the second image time and the first acceleration integral between the first coordinate system of the first image time and the first coordinate system of the second image time.

The first image time is one of the plurality of image times, the second image time is a time other than the first image time, the initial time is a first time of the plurality of image times, and the first initial coordinate system is a first coordinate system of the initial time.

In connection with the above description of the embodiments, it should be understood that the electronic device (including the image sensor) is in a state of continuous motion, that is, the position and posture of the image sensor may be changed all the time. In the embodiment of the present disclosure, each of the image moments may correspond to a first coordinate system, that is, the first coordinate system of each image coordinate system; the first coordinate system of each of the plurality of image time instants is used to characterize a pose of the image sensor at the each image time instant.

It is understood that the initial first coordinate system is used to characterize the pose of the image sensor at the initial time, and may also be understood as the initial state of the image sensor. In an embodiment of the present disclosure, the pose of the first coordinate system of each of the plurality of image moments relative to the initial first coordinate system is used to characterize a pose change of the image sensor at each of the image moments compared to the pose change of the image sensor at the initial moment.

In an embodiment of the disclosure, the electronic device may further determine the initial first coordinate system as a world coordinate system, such that the electronic device determines a pose of the first coordinate system of each of the plurality of image time instants relative to the initial first coordinate system, i.e. determines a pose of the first coordinate system of each of the plurality of image time instants relative to the world coordinate system.

S103, the electronic device determines an initial speed characteristic and an initial gravity characteristic according to the relative posture, the position information of the target three-dimensional point in the plurality of images, the posture of the first coordinate system of each image time relative to the initial first coordinate system, the time difference between each image time and the initial time, the time difference between the first image time and the second image time and the first acceleration integral between the coordinate system of the first image time and the first coordinate system of the second image time.

The initial speed characteristic is a speed characteristic of the electronic device at the initial time, and the initial gravity characteristic is a gravity characteristic of the electronic device at the initial time.

Optionally, the initial velocity feature and the initial gravity feature may be a three-dimensional vector.

The technical scheme provided by the embodiment can at least bring the following beneficial effects: as can be seen from S101-S103, the electronic device may obtain the relative pose and the position information of the target three-dimensional point in the plurality of images, and then determine the pose of the first coordinate system of each of the plurality of image times relative to the initial first coordinate system, the time difference between each of the plurality of image times and the initial time, the time difference between the first image time and the second image time, and the first acceleration integral between the first coordinate system of the first image time and the first coordinate system of the second image time, so as to determine the speed characteristic of the electronic device at the initial time and the gravity characteristic of the electronic device at the initial time. In the embodiment of the disclosure, the electronic device determines the posture of the first coordinate system at different image moments relative to the initial first coordinate system, that is, determines the posture change condition of the image sensor at different image moments; and determining a first acceleration integral between the first coordinate systems of any two image moments (i.e. a first image moment and a second image moment) of the plurality of image moments and a time difference between the any two image moments; the initial speed characteristic and the initial gravity characteristic which can reflect the initial state of the electronic equipment can be accurately and effectively determined, and then the motion trail of the electronic equipment in the three-dimensional space can be accurately determined based on the initial speed characteristic and the initial gravity characteristic.

With reference to fig. 1, as shown in fig. 2, in an implementation manner of the embodiment of the present disclosure, the pose of the first coordinate system of each of the image time instants relative to the initial first coordinate system includes a rotation feature of the first coordinate system of each of the image time instants relative to the initial first coordinate system, and the method for determining an initial feature provided by the embodiment of the present disclosure may further include S104 to S105.

S104, the electronic equipment acquires the angular speed of the electronic equipment at each data moment in a plurality of data moments.

The plurality of data moments are moments when the inertial sensor collects speed data of the electronic equipment.

It should be understood that the embodiments of the present disclosure include both image time and data time. Specifically, as described in the above embodiments, one image time is the time when the image sensor acquires an image, that is, the image sensor acquires one image at one image time. The plurality of data moments are moments when the inertial sensor collects the speed data of the electronic device, that is, the inertial sensor collects one (or one) time of speed data at one data moment.

In the embodiment of the present disclosure, the inertial sensor may include a gyroscope, and the electronic device may measure and acquire an angular velocity of the electronic device at each of the plurality of data times, where the angular velocity belongs to one of the velocity data.

S105, the electronic equipment determines the time difference between any two adjacent data moments in the multiple data moments.

It should be understood that the electronic device (specifically, the inertial sensor) is uniformly acquired when acquiring the speed data of the electronic device, that is, the inertial sensor acquires the speed data of the electronic device once at a preset time interval, where the preset time interval is a time difference between any two adjacent data moments.

For example, it is assumed that the data times include a first data time, a second data time, a third data time, and a fourth data time in chronological order. The time difference between the first data time and the second data time is the same as the time difference between the second data time and the third data time (or the time difference between the third data time and the fourth data time), and the time difference between the first data time and the second data time (or the time difference between the second data time and the third data time, or the time difference between the third data time and the fourth data time) is the time difference between any two adjacent data times.

Continuing with fig. 2, determining rotation characteristics of the first coordinate system of each of the plurality of image time instants relative to the initial first coordinate system includes S1021.

And S1021, the electronic device determines the rotation characteristic of the first coordinate system of each image time in the plurality of image times relative to the initial first coordinate system according to the angular speed of the electronic device at each data time in the plurality of data times and the time difference between any two adjacent data times in the plurality of data times.

Wherein a time difference between any two adjacent image times among the plurality of image times is greater than a time difference between the any two adjacent data times.

It should be appreciated that the angular velocity of the electronic device at each of the plurality of data instants may characterize a rotation of the electronic device while in motion, and the electronic device may determine a rotation characteristic of the first coordinate system at each of the plurality of image instants relative to the initial first coordinate system based on the rotation.

The technical scheme provided by the embodiment can at least bring the following beneficial effects: as can be seen from S104-S105, and S1021, the electronic device may obtain an angular velocity of the electronic device at each of a plurality of data moments, and determine a time difference between any two adjacent data moments in the plurality of data moments; because the angular speed of each of the multiple data moments can represent the rotation condition of the electronic device during movement, the electronic device can determine the rotation characteristic of the first coordinate system of each of the multiple image moments relative to the initial first coordinate system based on the rotation condition, and can reasonably and accurately determine the posture change condition (specifically, the rotation condition) of the image sensor relative to the initial image sensor at different image moments, thereby improving the accuracy of determining the initial speed characteristic and the initial gravity characteristic.

With reference to fig. 2 and as shown in fig. 3, in an implementation manner of the embodiment of the present disclosure, the above-mentioned relative pose includes a translation feature of the first coordinate system relative to the second coordinate system at the same time, and the method for determining an initial feature provided by the embodiment of the present disclosure further includes: S106-S107.

S106, the electronic equipment acquires the acceleration of the electronic equipment at each data moment in a plurality of data moments.

It should be understood that the acceleration data also represents one of the velocity data described above.

In the embodiment of the present disclosure, the inertial sensor may further include an accelerometer. Specifically, the electronic device may measure and acquire the acceleration of the electronic device at each of the plurality of data times through the accelerometer.

S107, the electronic equipment determines the rotation characteristics of each data moment relative to the initial first coordinate system.

In conjunction with the description of the above embodiments, it should be understood that the rotation characteristic of each of the plurality of data moments relative to the initial first coordinate system may characterize the rotation of the electronic device at each of the data moments relative to the electronic device at the initial moment (which may also be understood as the initial state of the electronic device).

Continuing with fig. 3, determining a first acceleration integral between the first coordinate system at the first image time and the first coordinate system at the second image time may specifically include S1022.

S1022, the electronic device determines a first acceleration integral between the first coordinate system of the first image time and the first coordinate system of the second image time according to the acceleration of the electronic device at each of the plurality of data times, the rotation characteristic of each of the plurality of data times relative to the initial first coordinate system, the time difference between any two adjacent data times, the rotation characteristic of the first coordinate system at each of the plurality of image times relative to the initial first coordinate system, and the translation characteristic of the first coordinate system at the same time relative to the second coordinate system.

In conjunction with the above description of the embodiments, it should be understood that the first coordinate system is a coordinate system corresponding to the image sensor, and the second coordinate system is a coordinate system corresponding to the inertial sensor.

Specifically, the electronic device may determine a second acceleration integral between the first coordinate system of each of the plurality of image time instants and the initial first coordinate system according to an acceleration of the electronic device at each of the plurality of data time instants, a rotation characteristic of each of the plurality of data time instants relative to the initial first coordinate system, and a time difference between any two adjacent data time instants, and then determine a first acceleration integral between the first coordinate system of the first image time instant and the first coordinate system of the second image time instant by combining the rotation characteristic of the first coordinate system of each of the plurality of image time instants relative to the initial first coordinate system and a translation characteristic of the first coordinate system relative to the second coordinate system at the same time instant.

The technical scheme provided by the embodiment can at least bring the following beneficial effects: as known from S106-S107, and S1022, the electronic device may acquire an acceleration of the electronic device at each of a plurality of data moments, and determine a rotation characteristic of each of the plurality of data moments relative to the initial first coordinate system; the electronic device then determines, based on the acceleration of the electronic device at each of the plurality of data instants, the rotation characteristic of each of the plurality of data instants relative to the initial first coordinate system, the time difference between any two adjacent ones of the plurality of data instants, the rotation characteristic of the first coordinate system at each of the plurality of image instants relative to the initial first coordinate system, and the translation characteristic of the first coordinate system at the same instant relative to the second coordinate system, the first acceleration integral between the first coordinate system of the first image time and the first coordinate system of the second image time is determined, the first acceleration integral between the first coordinate systems of any two image times (such as the first image time and the second image time) in the plurality of image times can be accurately and effectively determined, and the accuracy of determining the initial speed characteristic and the initial gravity characteristic is further improved.

In an implementation manner of the embodiment of the present disclosure, as shown in fig. 4 in combination with fig. 2, the determining, according to the angular velocity of the electronic device at each of the plurality of data time instants and the time difference between any two adjacent data time instants, the rotation feature of the first coordinate system at each of the plurality of image time instants relative to the initial first coordinate system may specifically include S1021 a.

And S1021a, the electronic device determines the rotation characteristic of the first coordinate system of the j image time relative to the initial first coordinate system according to the angular velocity of the k data time and the time difference between any two adjacent data times in the plurality of data times.

The jth image time is one of the image times, the kth data time is one of n data times included between the ith image time and the jth image time, the ith image time is a time except the jth image time in the image times, i is not less than 1, j is not less than 2, n is not less than 1, and k is not less than 1 and not more than n.

In one implementation of the embodiment of the disclosure, the electronic device may determine that the rotation characteristic of the first coordinate system at the jth image time relative to the initial first coordinate system satisfies the following formula:

wherein R is_b1bjA rotation characteristic, ω, of the first coordinate system representing the j-th image instant relative to the initial first coordinate system_kIndicates the angular velocity of the k-th data instant, and Δ t indicates the time difference between any two adjacent data instants among the plurality of data instants.

In an alternative implementation, the rotation feature in the present disclosure may also be understood as a rotation matrix, that is, the rotation feature of the first coordinate system of each of the plurality of image time instants relative to the initial first coordinate system is a rotation matrix of the first coordinate system of each of the plurality of image time instants relative to the initial first coordinate system.

In the embodiment of the present disclosure, a time difference between any two adjacent image times in the plurality of image times may be greater than a time difference between any two adjacent data times in the plurality of data times. For example, the time difference between any two adjacent image time instants may be 1min (minute), and the time difference between any two adjacent data time instants may be 10s (second), i.e. 6 data time instants may be included between any two adjacent image time instants.

The technical scheme provided by the embodiment can at least bring the following beneficial effects: from S1021a, the electronic device can determine the rotation characteristic of the first coordinate system at the jth image time relative to the initial first coordinate system according to the angular velocity at the k data times and the time difference between any two adjacent data times. Because the jth image time is one of the image times, the electronic device can accurately determine the rotation matrix of the first coordinate system of each image time in the image times relative to the initial first coordinate system, and the determination efficiency of the rotation matrix of the first coordinate system of each image time relative to the initial first coordinate system can be improved.

In one implementation of the embodiment of the disclosure, as shown in fig. 5 in conjunction with fig. 3, the determining the first acceleration integral between the first coordinate system at the first image time and the first coordinate system at the second image time according to the acceleration of the electronic device at each of the plurality of data times, the rotation characteristic of each of the plurality of data times relative to the initial first coordinate system, the time difference between any two adjacent data times in the plurality of data times, the rotation characteristic of the first coordinate system at each of the plurality of image times relative to the initial first coordinate system, and the translation characteristic of the first coordinate system at the same time relative to the second coordinate system may specifically include S1022a-S1022 b.

S1022a, the electronic device determines a second acceleration integral between the first coordinate system at the jth image time and the initial first coordinate system according to a time difference between any two adjacent data times of the plurality of data times, a rotation characteristic of the first coordinate system at the kth data time with respect to the initial first coordinate system, an acceleration of the electronic device at the kth data time, a rotation characteristic of the first coordinate system at the fth data time with respect to the initial first coordinate system, and an acceleration of the electronic device at the fth data time.

The kth data time is one of n data times included between the ith image time and the jth image time, the fth data time is one of m data times included between the ith image time and the kth data time, the jth image time is one of the multiple image times, the ith time is a time except the jth image time in the multiple image times, i is not less than 1, j is not less than 2, n is not less than 1, k is not less than 1, m is not less than 1, and f is not less than 1.

S1022b, the electronic device determines, according to the second acceleration integral between the first coordinate system and the initial first coordinate system at the jth image time, the second acceleration integral between the first coordinate system and the initial first coordinate system at the ith image time, the rotation feature of the first coordinate system at the jth image time relative to the initial first coordinate system, the rotation feature of the first coordinate system at the ith image time relative to the initial first coordinate system, and the translation feature of the first coordinate system at the same time relative to the second coordinate system, the first acceleration integral between the first coordinate system at the ith image time and the first coordinate system at the jth image time.

It is understood that the electronic device may first determine a second acceleration integral between the first coordinate system of the jth image time and the initial first coordinate system, and then determine a first acceleration integral between the first coordinate system of the ith image time and the first coordinate system of the jth image time.

In one implementation of the disclosed embodiment, the electronic device may determine that a first acceleration integral between the first coordinate system of the ith image time and the first coordinate system of the jth image time satisfies the following equation:

S_ij＝Δp_1j-Δp_1i+(R_b1bj-R_b1bi)p_bc (2)

wherein S is_ijRepresents a first acceleration integral, Δ p, between the first coordinate system of the ith image instant and the first coordinate system of the jth image instant_1jA second integral of acceleration, Δ p, between the first coordinate system characterizing the j-th image instant and the initial first coordinate system_1iA second integral of acceleration, R, between the first coordinate system characterizing the ith image instant and the initial first coordinate system_b1bjA rotation characteristic, R, of the first coordinate system representing the j-th image instant relative to the initial first coordinate system_b1biA rotation characteristic, p, of the first coordinate system representing the instant of the ith image relative to the initial first coordinate system_bcRepresenting the translational characteristics of the first coordinate system relative to the second coordinate system at the same time.

In another implementation of the embodiment of the present disclosure, the electronic device may further determine that a second acceleration integral between the first coordinate system of the jth image time and the initial first coordinate system satisfies the following formula:

wherein, Δ p_1jA second acceleration integral between the first coordinate system characterizing the jth image instant and the initial first coordinate system, Δ t representing a time difference between any two adjacent data instants of the plurality of data instants, R_b1bkA rotation characteristic of the first coordinate system representing the k-th data instant relative to the initial first coordinate system, a_kRepresenting the acceleration, R, of the electronic device at the kth data time_b1bfA rotation characteristic of the first coordinate system relative to the initial first coordinate system representing the f-th data instant, a_fIndicating the acceleration of the electronic device at the f-th data time.

In conjunction with the description of the above embodiments, it should be understood that n data time instants may be included between the ith image time instant and the jth image time instant, and the kth data time instant is one of the n data time instants. Similarly, m data moments may be included between the ith image moment and the kth data moment, and the fth data moment is one of the m data moments.

It is understood that the jth data time point may correspond to the first image time point in the embodiment of the present disclosure, that is, the ith image time point may correspond to the second image time point in the embodiment of the present disclosure.

It should be noted that the electronic device determines a second acceleration integral (i.e., Δ p) between the first coordinate system of the ith image time and the initial first coordinate system_1i) And the second acceleration integral (i.e., Δ p) between the first coordinate system at the moment of the j-th image determination and the initial first coordinate system as described above_1j) The methods of (a) are the same or similar and will not be described in detail herein.

The technical scheme provided by the embodiment can at least bring the following beneficial effects: from S1022a-S1022b, the electronic device may determine a second acceleration integral between the first coordinate system of the jth image time and the initial first coordinate system according to a time difference between any two adjacent data times of the plurality of data times, a rotation characteristic of the first coordinate system of the kth data time relative to the initial first coordinate system, an acceleration of the electronic device at the kth data time, a rotation characteristic of the first coordinate system of the fth data time relative to the initial first coordinate system, and an acceleration of the electronic device at the fth data time, and determine a second acceleration integral between the first coordinate system of the jth image time and the initial first coordinate system according to the second acceleration integral between the first coordinate system of the jth image time and the initial first coordinate system, the second acceleration integral between the first coordinate system of the ith image time and the initial first coordinate system, the rotation characteristic of the first coordinate system of the jth image time relative to the initial first coordinate system, The rotation feature of the first coordinate system of the ith image time relative to the initial first coordinate system and the translation feature of the first coordinate system at the same time relative to the second coordinate system determine a first acceleration integral between the first coordinate system of the ith image time and the first coordinate system of the jth image time. In the embodiment of the present disclosure, the electronic device may determine a second acceleration integral between a first coordinate system of one image time (for example, the jth image time) of the multiple image times and the initial first coordinate system, and then determine a first acceleration integral between the image time and a first coordinate system of another image time (for example, the ith image time) of the multiple image times, so that the determination efficiency of each first acceleration integral can be improved, and the determination efficiency of the initial velocity feature and the initial gravity feature can be further improved.

In one implementation of the embodiment of the disclosure, as shown in fig. 6 in conjunction with fig. 1, the posture of the first coordinate system of each of the plurality of image moments with respect to the initial first coordinate system includes a rotation feature and a translation feature of the first coordinate system of each of the plurality of image moments with respect to the initial first coordinate system, and the relative posture includes a translation feature of the first coordinate system with respect to the second coordinate system at the same moment and a rotation feature of the first coordinate system with respect to the second coordinate system at the same moment.

The determining of the initial velocity characteristic and the initial gravity characteristic according to the relative pose, the position information of the target three-dimensional point in the plurality of images, the pose of the first coordinate system at each of the plurality of image times relative to the initial first coordinate system, the time difference between each of the plurality of image times and the initial time, the time difference between the first image time and the second image time, and the first acceleration integral between the first coordinate system at the first image time and the first coordinate system at the second image time may specifically include S1031.

S1031, the electronic device, according to a preset non-unit modular length normal vector, a translation feature of the first coordinate system at the ith image time relative to the initial first coordinate system, a rotation feature of the first coordinate system at the ith image time relative to the initial first coordinate system, a translation feature of the first coordinate system at the same time relative to the second coordinate system, a rotation feature of the first coordinate system at the same time relative to the second coordinate system, position information of the target three-dimensional point in the ith image, a translation feature of the first coordinate system at the jth image time relative to the initial first coordinate system, a rotation feature of the first coordinate system at the jth image time relative to the initial first coordinate system, position information of the target three-dimensional point in the jth image, a time difference between the ith image time and the jth image time, a time difference between the jth image time and the initial time, a translation feature of the first coordinate system at the jth image time relative to the initial coordinate system, a translation feature of the jth image time, And determining an initial speed characteristic and an initial gravity characteristic according to the time difference between the ith image moment and the initial moment and the first acceleration integral between the first coordinate system of the ith image moment and the first coordinate system of the jth image moment.

Wherein, the j image time is one of the plurality of image times, the i time is the time except the j image time, i is more than or equal to 1, and j is more than or equal to 2.

In one implementation of the disclosed embodiment, the electronic device may determine that the initial velocity characteristic and the initial gravity characteristic satisfy the following formula:

wherein the content of the first and second substances,

representing the predetermined non-unit norm vector, p_b1biA translation feature, R, of the first coordinate system representing the i-th image instant relative to the initial first coordinate system_b1biA rotation characteristic, p, of the first coordinate system representing the instant of the ith image relative to the initial first coordinate system_bcRepresenting the translation characteristics of the first coordinate system relative to the second coordinate system at the same time, R_bcRepresenting the rotation characteristic of the first coordinate system relative to the second coordinate system at the same time,

indicating the position information of the target three-dimensional point in the ith image, p_b1bjA translation feature, R, of the first coordinate system representing the j-th image instant relative to the initial first coordinate system_b1bjA rotation feature of the first coordinate system representing the j-th image instant relative to the initial first coordinate systemThe step of performing the sign operation,

representing the position information of the target three-dimensional point in the jth image,

representing the initial speed characteristic, Δ t_i,jRepresenting the time difference between the i-th image time and the j-th image time, g^b1Representing the initial gravitational characteristic, Δ t_1,jRepresenting the time difference, Δ t, between the jth image instant and the initial instant_1,iRepresenting the time difference between the i-th image time and the initial time, S_ijA first acceleration integral between the first coordinate system of the ith image instant and the first coordinate system of the jth image instant.

It will be understood that in the above formula

To represent

The transposed matrix of (2).

In an alternative implementation, a certain translation feature of an embodiment of the present disclosure may be a translation vector. For example, the translation feature of the first coordinate system of the ith image time relative to the initial first coordinate system may be a translation vector of the first coordinate system of the ith image time relative to the initial first coordinate system.

Alternatively, the position information of the target three-dimensional point in one of the images (for example, the ith image) may be coordinates of the target three-dimensional point in the ith image.

In one implementation of the embodiment of the present disclosure, a translation feature of the first coordinate system of one of the image time instants (for example, the jth image time instant and/or the ith image time instant) may also be unknown relative to the initial first coordinate system, so that a certain conversion may be required to be performed on the translation feature to determine the initial velocity feature and the initial gravity feature.

Specifically, the translation feature of the first coordinate system at the j-th image time relative to the initial first coordinate system may be converted based on the following formula (i.e., formula (5)), or the following formula may be substituted into the above formula (4) to determine the initial velocity feature and the initial gravity feature.

Wherein p is_b1bjA translation feature, p, representing the first coordinate system of the j-th image instant relative to the initial first coordinate system_b1b1Representing a translation characteristic of the initial first coordinate system relative to the initial first coordinate system,

representing the speed characteristic of the electronic device at the k-th data time, Δ t representing the time difference between any two adjacent data times of the plurality of data times, g^b1Representing the initial speed characteristic, R_b1bkA rotation characteristic representing the first coordinate system of the kth data instant relative to the initial first coordinate system, a_kThe acceleration of the electronic equipment at the kth data moment is represented, j is more than or equal to 2, n is more than or equal to 1, and k is more than or equal to 1 and less than or equal to n.

The technical scheme provided by the embodiment can at least bring the following beneficial effects: from S1031, the electronic device may obtain, according to the predetermined non-unit norm vector, the translation feature of the first coordinate system at the ith image time with respect to the initial first coordinate system, the rotation feature of the first coordinate system at the ith image time with respect to the initial first coordinate system, the translation feature of the first coordinate system at the same time with respect to the second coordinate system, the rotation feature of the first coordinate system at the same time with respect to the second coordinate system, the position information of the target three-dimensional point in the ith image, the translation feature of the first coordinate system at the jth image time with respect to the initial first coordinate system, the rotation feature of the first coordinate system at the jth image time with respect to the initial first coordinate system, the position information of the target three-dimensional point in the jth image, the time difference between the ith image time and the jth image time, the time difference between the jth image time and the initial time, the translation feature of the jth image time and the initial time, The time difference between the ith image moment and the initial moment and the first acceleration integral between the first coordinate system of the ith image moment and the first coordinate system of the jth image moment are used for accurately and effectively determining the initial speed characteristic and the initial gravity characteristic, and then the motion track of the electronic equipment can be accurately and effectively determined based on the initial speed characteristic and the initial gravity characteristic.

With reference to fig. 6, as shown in fig. 7, the initial feature determining method provided in the embodiment of the present disclosure further includes S108.

S108, the electronic equipment determines a preset non-unit modular length normal vector according to the unit normal vector of the target plane and the relative distance between the target plane and the initial first coordinate system.

Wherein, the target plane is a plane corresponding to the target three-dimensional point.

It is to be understood that the target three-dimensional point may correspond to a plurality of planes, and the target plane is a plane including the most three-dimensional points (or feature points) included in the plurality of planes.

In an alternative implementation, the electronic device may determine that the preset non-unity norm vector satisfies the following formula:

wherein the content of the first and second substances,

the method further includes the steps of representing the predetermined non-unit norm vector, x representing the unit normal vector of the target plane, and d representing the relative distance between the target plane and the initial first coordinate system.

The technical scheme provided by the embodiment can at least bring the following beneficial effects: from S108, the electronic device can conveniently and quickly determine the preset non-unit modular length normal vector according to the unit normal vector of the target plane and the relative distance between the target plane and the initial first coordinate system, so as to quickly determine the initial speed characteristic and the initial gravity characteristic.

A process of obtaining the above formula (4) in the embodiment of the present disclosure is described in detail below as an example.

In conjunction with the above description of the embodiments, it should be understood that the image sensor acquires one image at each of the plurality of image moments to obtain the plurality of images. When the target three-dimensional point is observed by any two images (or any two frames of images) in the plurality of images (or the target three-dimensional point is included in any two images), the visual observation under the multi-view angle may form an epipolar geometric constraint shown in the following formula (7):

wherein p is_b1biA translation feature, R, of the first coordinate system representing the i-th image instant relative to the initial first coordinate system_b1biA rotation characteristic, p, of the first coordinate system representing the instant of the ith image relative to the initial first coordinate system_bcRepresenting the translational characteristics of the first coordinate system relative to the second coordinate system at the same time,

representing the depth, R, of the target three-dimensional point in the first coordinate system at the instant of the i-th image_bcRepresenting the rotation characteristic of the first coordinate system relative to the second coordinate system at the same time,

representing the position information, p, of the target three-dimensional point in the ith image_b1bjA translation feature, R, of the first coordinate system representing the j-th image instant relative to the initial first coordinate system_b1bjA first coordinate system representing the j-th image instant relative to the initial first seatThe rotation characteristic of the mark system is that,

representing the depth of the target three-dimensional point in the first coordinate system at the j-th image time instant,

and the j is larger than or equal to 1 and larger than or equal to 2, and represents the position information of the target three-dimensional point in the jth image.

It should be noted that the target plane may include a plurality of three-dimensional points, and the target three-dimensional point is one of the plurality of three-dimensional points. The above formula (7) should be satisfied also for other three-dimensional points, i.e., three-dimensional points other than the target three-dimensional point among the plurality of three-dimensional points.

In one implementation of the embodiment of the present disclosure, the electronic device may further determine a speed characteristic of the jth image time and a translation characteristic of the first coordinate system of the jth image time relative to the initial first coordinate system, which respectively satisfy the following two formulas (i.e., formula (8) and formula (9)):

wherein the content of the first and second substances,

representing the velocity characteristic at the instant of the j-th image,

representing the initial velocity characteristic, g^b1Representing the initial gravitational characteristic, Δ t_1,jRepresenting the time difference, R, between the jth image instant and the initial instant_b1bkA rotation characteristic representing the first coordinate system of the kth data instant relative to the initial first coordinate system, a_kThe acceleration of the electronic equipment at the kth data time is shown, delta t represents the time difference between any two adjacent data times in the plurality of data times, and the kth data time is the ith image timeAnd in one of n data moments included between the moment and the jth image moment, i is more than or equal to 1, j is more than or equal to 2, n is more than or equal to 1, and k is more than or equal to 1 and less than or equal to n.

Wherein p is_b1bjA translation feature, p, representing the first coordinate system of the j-th image instant relative to the initial first coordinate system_b1b1A translation feature of the first coordinate system representing the 1 st image instant relative to the initial first coordinate system,

representing the velocity characteristic of the kth data time, Δ t representing the time difference between any two adjacent data times of the plurality of data times, g^b1Representing the initial gravity characteristic, R_b1bkA rotation characteristic representing the first coordinate system of the kth data instant relative to the initial first coordinate system, a_kAnd the acceleration of the electronic equipment at the kth data moment is represented, the kth data moment is one of n data moments included between the ith image moment and the jth image moment, i is not less than 1, j is not less than 2, n is not less than 1, and k is not less than 1 and not more than n.

Substituting equation (8), equation (9) and equation (1) into equation (7) can obtain a new constraint equation (10):

wherein the content of the first and second substances,

representing the depth, R, of the target three-dimensional point in the first coordinate system at the instant of the i-th image_b1biA rotation characteristic, R, of the first coordinate system representing the i-th image instant relative to the initial first coordinate system_bcRepresenting the rotation characteristic of the first coordinate system relative to the second coordinate system at the same time,

indicating the position information of the target three-dimensional point in the ith image,

representing the depth, R, of the target three-dimensional point in the first coordinate system at the jth image time_b1bjA rotation feature representing the first coordinate system of the jth image instant relative to the initial first coordinate system,

representing the position information of the target three-dimensional point in the jth image,

representing the initial speed characteristic, Δ t_i,jRepresenting the time difference between the i-th image time and the j-th image time, g^b1Representing the initial gravitational characteristic, Δ t_1,jRepresenting the time difference, Δ t, between the jth image instant and the initial instant_1,iRepresenting the time difference between the i-th image time and the initial time, S_ijAnd a first acceleration integral between the first coordinate system of the ith image moment and the first coordinate system of the jth image moment is represented, i is larger than or equal to 1, and j is larger than or equal to 2.

In the disclosed embodiments, planar constraints are also taken into account. Specifically, it is assumed that the target plane can be expressed in the upper first initial coordinate system as: pi ═ x^T,d]^T. Wherein the three-dimensional vector x represents a unit normal vector of the target plane, and d represents a relative distance between the target plane and the initial first coordinate system.

In connection with the above description of the embodiments, it is to be understood that x^TRepresenting the transpose of x.

Since the target three-dimensional point falls on the target plane, the following formula can be obtained:

where x represents the unit normal vector of the target plane, p_b1biA translation feature, R, of the first coordinate system representing the i-th image instant relative to the initial first coordinate system_b1biA rotation characteristic, p, of the first coordinate system representing the instant of the ith image relative to the initial first coordinate system_bcRepresenting the translational characteristics of the first coordinate system relative to the second coordinate system at the same time,

representing the depth, R, of the target three-dimensional point in the first coordinate system at the instant of the i-th image_bcRepresenting the rotation characteristic of the first coordinate system relative to the second coordinate system at the same time,

the position information of the target three-dimensional point in the ith image is represented, d represents the relative distance between the target plane and the initial first coordinate system, and i is larger than or equal to 1.

From this, it can be derived that the depth of the target three-dimensional point in the first coordinate system at the ith image time satisfies the following formula:

wherein the content of the first and second substances,

representing the depth of the target three-dimensional point in the first coordinate system at the i-th image time, d representing the relative distance between the target plane and the initial first coordinate system, x representing the unit normal vector of the target plane, p_b1biA translation feature, R, of the first coordinate system representing the i-th image instant relative to the initial first coordinate system_b1biA rotation characteristic, p, of the first coordinate system representing the instant of the ith image relative to the initial first coordinate system_bcRepresenting the plane of the first coordinate system relative to the second coordinate system at the same timeCharacterised by the fact that R_bcRepresenting the rotation characteristic of the first coordinate system relative to the second coordinate system at the same time,

and the position information of the target three-dimensional point in the ith image is shown, wherein i is more than or equal to 1.

To this end, formula (12) is substituted into formula (10) above, and formula (6) above is combined (specifically, x is converted to

) That is, equation (4) in the present disclosure can be obtained.

It is understood that, in practical implementation, the electronic device according to the embodiments of the present disclosure may include one or more hardware structures and/or software modules for implementing the corresponding initial feature determination method, and the executing hardware structures and/or software modules may constitute an electronic device. Those of skill in the art will readily appreciate that the present disclosure can be implemented in hardware or a combination of hardware and computer software for implementing the exemplary algorithm steps described in connection with the embodiments disclosed herein. Whether a function is performed as hardware or computer software drives hardware depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

Based on such understanding, the embodiment of the present disclosure further provides an initial feature determining apparatus correspondingly, and fig. 8 illustrates a schematic structural diagram of the initial feature determining apparatus provided by the embodiment of the present disclosure. As shown in fig. 8, the initial characteristic determination device 10 may include: an acquisition module 101 and a determination module 102.

The acquiring module 101 is configured to acquire a relative pose and position information of a target three-dimensional point in a plurality of images, where the relative pose is a pose of a first coordinate system relative to a second coordinate system at the same time, the first coordinate system is a coordinate system corresponding to an image sensor included in an electronic device, the second coordinate system is a coordinate system corresponding to an inertial sensor included in the electronic device, and the plurality of images are images respectively acquired by the image sensor at a plurality of image times, where each image time is a time when one image is acquired by the image sensor.

A determining module 102 configured to determine a pose of the first coordinate system of each of the plurality of image moments with respect to an initial first coordinate system, a time difference between each of the plurality of image moments and an initial moment, a time difference between a first image moment and a second image moment, and a first acceleration integral between the first coordinate system of the first image moment and the first coordinate system of the second image moment, the first image moment being one of the plurality of image moments, the second image moment being a moment of the plurality of image moments other than the first image moment, the initial moment being a first one of the plurality of image moments, the initial first coordinate system of the initial moment.

The determining module 102 is further configured to determine an initial velocity feature and an initial gravity feature according to the relative pose, the position information of the target three-dimensional point in the plurality of images, the pose of the first coordinate system of each of the plurality of image moments relative to the initial first coordinate system, the time difference between each of the plurality of image moments and the initial moment, the time difference between the first image moment and the second image moment, and the first acceleration integral between the first coordinate system of the first image moment and the first coordinate system of the second image moment, wherein the initial velocity feature is a velocity feature of the electronic device at the initial moment, and the initial gravity feature is a gravity feature of the electronic device at the initial moment.

Optionally, the pose of the first coordinate system of each of the plurality of image time instants with respect to the initial first coordinate system comprises a rotation characteristic of the first coordinate system of each of the plurality of image time instants with respect to the initial first coordinate system.

The obtaining module 101 is further configured to obtain an angular velocity of the electronic device at each of a plurality of data moments, where the plurality of data moments are moments when the inertial sensor collects velocity data of the electronic device.

The determining module 102 is further configured to determine a time difference between any two adjacent data time instants of the plurality of data time instants.

The determining module 102 is specifically configured to determine a rotation characteristic of the first coordinate system of each of the plurality of image time instants relative to the initial first coordinate system according to the angular velocity of the electronic device at each of the plurality of data time instants and a time difference between any two adjacent data time instants, where the time difference between any two adjacent image time instants is greater than the time difference between any two adjacent data time instants.

Optionally, the relative pose includes a translation feature of the first coordinate system relative to the second coordinate system at the same time.

The obtaining module 101 is further configured to obtain an acceleration of the electronic device at each of the plurality of data moments.

The determining module 102 is further configured to determine a rotation characteristic of each of the plurality of data instants with respect to the initial first coordinate system.

The determining module 102 is further specifically configured to determine a first acceleration integral between the first coordinate system of the first image time and the first coordinate system of the second image time according to the acceleration of the electronic device at each of the plurality of data times, the rotation characteristic of each of the plurality of data times relative to the initial first coordinate system, the time difference between any two adjacent data times of the plurality of data times, the rotation characteristic of the first coordinate system at each of the plurality of image times relative to the initial first coordinate system, and the translation characteristic of the first coordinate system at the same time relative to the second coordinate system.

Optionally, the determining module 102 is specifically further configured to determine a rotation characteristic of the first coordinate system of the jth image time with respect to the initial first coordinate system according to an angular velocity of the kth data time and a time difference between any two adjacent data times in the plurality of data times, where the kth data time is one of n data times included between the ith image time and the jth image time, the ith image time is a time of the plurality of image times except the jth image time, i is greater than or equal to 1, j is greater than or equal to 2, n is greater than or equal to 1, and 1 is greater than or equal to k is less than or equal to n.

Optionally, the determining module 102 is specifically further configured to determine a second acceleration integral between the first coordinate system of the jth image time and the initial first coordinate system according to a time difference between any two adjacent data times in the plurality of data times, a rotation characteristic of the first coordinate system of the kth data time relative to the initial first coordinate system, an acceleration of the electronic device at the kth data time, a rotation characteristic of the first coordinate system of the fth data time relative to the initial first coordinate system, and an acceleration of the electronic device at the fth data time, where the kth data time is one of n data times included between the ith image time and the jth image time, the fth data time is one of m data times included between the ith image time and the kth data time, the j image time is one of the image times, the i time is the time except the j image time, i is more than or equal to 1, j is more than or equal to 2, n is more than or equal to 1, k is more than or equal to 1 and less than or equal to n, m is more than or equal to 1 and less than or equal to k, and f is more than or equal to 1 and less than or equal to m.

The determining module 102 is specifically further configured to determine a first acceleration integral between the first coordinate system of the ith image time and the first coordinate system of the jth image time according to a second acceleration integral between the first coordinate system of the jth image time and the initial first coordinate system, a rotation feature of the first coordinate system of the jth image time relative to the initial first coordinate system, a rotation feature of the first coordinate system of the ith image time relative to the initial first coordinate system, and a translation feature of the first coordinate system at the same time relative to the second coordinate system.

Optionally, the pose of the first coordinate system of each of the plurality of image time instants with respect to the initial first coordinate system comprises a rotation feature and a translation feature of the first coordinate system of each of the plurality of image time instants with respect to the initial first coordinate system, and the relative pose comprises a translation feature of the first coordinate system with respect to the second coordinate system at the same time instant and a rotation feature of the first coordinate system with respect to the second coordinate system at the same time instant.

The determining module 102 is specifically configured to determine a translation feature of the first coordinate system at the ith image time relative to the initial first coordinate system, a rotation feature of the first coordinate system at the ith image time relative to the initial first coordinate system, a translation feature of the first coordinate system at the same time relative to the second coordinate system, a rotation feature of the first coordinate system at the same time relative to the second coordinate system, position information of the target three-dimensional point in the ith image, a translation feature of the first coordinate system at the jth image time relative to the initial first coordinate system, a rotation feature of the first coordinate system at the jth image time relative to the initial first coordinate system, position information of the target three-dimensional point in the jth image, a time difference between the ith image time and the jth image time, a translation feature of the first coordinate system at the jth image time relative to the initial first coordinate system, a translation feature of the target three-dimensional point in the jth image, a translation feature of the jth image time relative to the initial first coordinate system, a translation feature of the jth image time, Determining the initial speed characteristic and the initial gravity characteristic by the time difference between the jth image time and the initial time, the time difference between the ith image time and the initial time and the first acceleration integral between the first coordinate system of the ith image time and the first coordinate system of the jth image time, wherein the jth image time is one of the image times, the ith time is a time except the jth image time in the image times, i is not less than 1, and j is not less than 2.

Optionally, the determining module 102 is further configured to determine the preset non-unit modular length normal vector according to a unit normal vector of a target plane and a relative distance between the target plane and the initial first coordinate system, where the target plane is a plane corresponding to the target three-dimensional point.

As described above, the embodiments of the present disclosure may perform the division of the functional modules on the initial feature determination device according to the above method example. The integrated module can be realized in a hardware form, and can also be realized in a software functional module form. In addition, it should be further noted that the division of the modules in the embodiments of the present disclosure is schematic, and is only a logic function division, and there may be another division manner in actual implementation. For example, the functional blocks may be divided for the respective functions, or two or more functions may be integrated into one processing block.

Regarding the initial characteristic determining apparatus in the foregoing embodiment, the specific manner in which each module performs the operation and the beneficial effects thereof have been described in detail in the foregoing method embodiment, and are not described herein again.

Fig. 9 is a schematic structural diagram of another initial characteristic determination apparatus provided by the present disclosure. As shown in fig. 9, the initial characteristic determination apparatus 20 may include at least one processor 201 and a memory 203 for storing processor-executable instructions. Wherein the processor 201 is configured to execute instructions in the memory 203 to implement the initial feature determination method in the above-described embodiments.

In addition, the initial characteristic determination apparatus 20 may further include a communication bus 202 and at least one communication interface 204.

The processor 201 may be a Central Processing Unit (CPU), a micro-processing unit, an ASIC, or one or more integrated circuits for controlling the execution of programs according to the present disclosure.

The communication bus 202 may include a path that conveys information between the aforementioned components.

The communication interface 204 may be any device, such as a transceiver, for communicating with other devices or communication networks, such as an ethernet, a Radio Access Network (RAN), a Wireless Local Area Network (WLAN), etc.

The memory 203 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a Random Access Memory (RAM) or other type of dynamic storage device that can store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disk storage, optical disk storage (including compact disc, laser disc, optical disc, digital versatile disc, blu-ray disc, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to these. The memory may be self-contained and connected to the processing unit by a bus. The memory may also be integrated with the processing unit.

The memory 203 is used for storing instructions for executing the disclosed solution, and is controlled by the processor 201. The processor 201 is configured to execute instructions stored in the memory 203 to implement the functions of the disclosed method.

In particular implementations, processor 201 may include one or more CPUs such as CPU0 and CPU1 in fig. 9 for one embodiment.

In a specific implementation, the initial feature determination apparatus 20 may include a plurality of processors, such as the processor 201 and the processor 207 in fig. 9, as an embodiment. Each of these processors may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. A processor herein may refer to one or more devices, circuits, and/or processing cores for processing data (e.g., computer program instructions).

In one embodiment, the initial characteristic determining apparatus 20 may further include an output device 205 and an input device 206. The output device 205 is in communication with the processor 201 and may display information in a variety of ways. For example, the output device 205 may be a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display device, a Cathode Ray Tube (CRT) display device, a projector (projector), or the like. The input device 206 is in communication with the processor 201 and can accept user input in a variety of ways. For example, the input device 206 may be a mouse, a keyboard, a touch screen device, or a sensing device, among others.

Those skilled in the art will appreciate that the configuration shown in fig. 9 does not constitute a limitation of the initial characteristic determination apparatus 20, and may include more or fewer components than those shown, or combine certain components, or adopt a different arrangement of components.

In addition, the present disclosure also provides a computer-readable storage medium including instructions that, when executed by an electronic device, cause the electronic device to perform the initial feature determination method provided as the above embodiment.

In addition, the present disclosure also provides a computer program product comprising instructions that, when executed by an electronic device, cause the electronic device to perform the initial feature determination method as provided in the above embodiments.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

Claims (10)

1. An initial feature determination method, comprising:

acquiring relative postures and position information of a target three-dimensional point in a plurality of images, wherein the relative postures are postures of a first coordinate system relative to a second coordinate system at the same time, the first coordinate system is a coordinate system corresponding to an image sensor included in electronic equipment, the second coordinate system is a coordinate system corresponding to an inertial sensor included in the electronic equipment, and the plurality of images are images respectively acquired by the image sensor at a plurality of image times, wherein each image time is the time when the image sensor acquires one image;

determining a pose of a first coordinate system of each of the plurality of image moments relative to an initial first coordinate system, a time difference between each of the plurality of image moments and an initial moment, a time difference between a first image moment and a second image moment, and a first acceleration integral between the first coordinate system of the first image moment and the first coordinate system of the second image moment, the first image moment being one of the plurality of image moments, the second image moment being a moment of the plurality of image moments other than the first image moment, the initial moment being the first one of the plurality of image moments, the initial first coordinate system being the first coordinate system of the initial moment;

determining an initial velocity characteristic and an initial gravity characteristic according to the relative pose, the position information of the target three-dimensional point in a plurality of images, the pose of the first coordinate system of each image time in the plurality of image times relative to the initial first coordinate system, the time difference between each image time in the plurality of image times and the initial time, the time difference between the first image time and the second image time, and the first acceleration integral between the first coordinate system of the first image time and the first coordinate system of the second image time, wherein the initial velocity characteristic is a velocity characteristic of the electronic device at the initial time, and the initial gravity characteristic is a gravity characteristic of the electronic device at the initial time.

2. The initial feature determination method of claim 1, wherein the pose of the first coordinate system of each of the plurality of image time instants relative to the initial first coordinate system comprises a rotation feature of the first coordinate system of each of the plurality of image time instants relative to the initial first coordinate system, the method further comprising:

acquiring the angular velocity of the electronic equipment at each data time in a plurality of data times, wherein the data times are the times when the inertial sensor acquires the velocity data of the electronic equipment;

determining a time difference between any two adjacent data moments of the plurality of data moments;

determining a rotation characteristic of the first coordinate system of each of the plurality of image time instants relative to the initial first coordinate system, comprising:

and determining the rotation characteristic of the first coordinate system of each image moment in the plurality of image moments relative to the initial first coordinate system according to the angular speed of the electronic equipment at each data moment in the plurality of data moments and the time difference between any two adjacent data moments in the plurality of data moments, wherein the time difference between any two adjacent image moments in the plurality of image moments is larger than the time difference between any two adjacent data moments.

3. The initial feature determination method of claim 2, wherein the relative pose comprises a translation feature of the first coordinate system relative to the second coordinate system at the same time, the method further comprising:

acquiring the acceleration of the electronic equipment at each data moment in the plurality of data moments;

determining a rotation characteristic of each of the plurality of data moments relative to the initial first coordinate system;

determining a first acceleration integral between the first coordinate system of the first image time and the first coordinate system of the second image time, comprising:

determining a first acceleration integral between a first coordinate system of the first image time and a first coordinate system of the second image time according to the acceleration of the electronic device at each of the plurality of data times, the rotation characteristic of each of the plurality of data times relative to the initial first coordinate system, the time difference between any two adjacent data times of the plurality of data times, the rotation characteristic of the first coordinate system at each of the plurality of image times relative to the initial first coordinate system, and the translation characteristic of the first coordinate system at the same time relative to the second coordinate system.

4. The initial feature determination method of claim 2, wherein determining the rotation feature of the first coordinate system of each of the plurality of image time instants relative to the initial first coordinate system according to the angular velocity of the electronic device at each of the plurality of data time instants and the time difference between any two adjacent data time instants of the plurality of data time instants comprises:

and determining the rotation characteristic of a first coordinate system of a j-th image time relative to the initial first coordinate system according to the angular speed of the k-th data time and the time difference between any two adjacent data times in the plurality of data times, wherein the j-th image time is one of the plurality of image times, the k-th data time is one of n data times included between the i-th image time and the j-th image time, the i-th image time is a time except the j-th image time in the plurality of image times, i is more than or equal to 1, j is more than or equal to 2, n is more than or equal to 1, and k is more than or equal to 1 and less than or equal to n.

5. The initial feature determination method of claim 3, wherein determining the first acceleration integral between the first coordinate system of the first image time and the first coordinate system of the second image time according to the acceleration of the electronic device at each of the plurality of data times, the rotation feature of each of the plurality of data times relative to the initial first coordinate system, the time difference between any two adjacent data times of the plurality of data times, the rotation feature of the first coordinate system at each of the plurality of image times relative to the initial first coordinate system, and the translation feature of the first coordinate system at the same time relative to the second coordinate system comprises:

determining a second acceleration integral between a first coordinate system of a jth image time and the initial first coordinate system according to a time difference between any two adjacent data times of the plurality of data times, a rotation characteristic of a first coordinate system of a kth data time relative to the initial first coordinate system, an acceleration of the electronic device at the kth data time, a rotation characteristic of a first coordinate system of a first f data time relative to the initial first coordinate system, and an acceleration of the electronic device at the f data time, the kth data time being one of n data times included between an ith image time and the jth image time, the f data time being one of m data times included between the ith image time and the kth data time, the jth image moment is one of the image moments, the ith moment is a moment except the jth image moment in the image moments, i is more than or equal to 1, j is more than or equal to 2, n is more than or equal to 1, k is more than or equal to 1 and less than or equal to n, m is more than or equal to 1 and less than or equal to k, and f is more than or equal to 1 and less than or equal to m;

and determining a first acceleration integral between the first coordinate system of the ith image moment and the first coordinate system of the jth image moment according to a second acceleration integral between the first coordinate system of the jth image moment and the initial first coordinate system, a second acceleration integral between the first coordinate system of the ith image moment and the initial first coordinate system, a rotation characteristic of the first coordinate system of the jth image moment relative to the initial first coordinate system, a rotation characteristic of the first coordinate system of the ith image moment relative to the initial first coordinate system and a translation characteristic of the first coordinate system relative to the second coordinate system at the same moment.

6. An initial feature determination method according to any one of claims 1-5, wherein the pose of the first coordinate system of each of the plurality of image moments with respect to the initial first coordinate system comprises a rotation feature and a translation feature of the first coordinate system of each of the plurality of image moments with respect to the initial first coordinate system, the relative pose comprising a translation feature of the first coordinate system with respect to the second coordinate system at the same moment and a rotation feature of the first coordinate system with respect to the second coordinate system at the same moment;

determining an initial velocity feature and an initial gravity feature according to the relative pose, the position information of the target three-dimensional point in the plurality of images, the pose of the first coordinate system of each image time in the plurality of image times relative to the initial first coordinate system, the time difference between each image time and the initial time, the time difference between the first image time and the second image time, and the first acceleration integral between the first coordinate system of the first image time and the second image time, including:

according to a preset non-unit modular length normal vector, a translation feature of a first coordinate system at the ith image moment relative to the initial first coordinate system, a rotation feature of the first coordinate system at the ith image moment relative to the initial first coordinate system, a translation feature of the first coordinate system at the same moment relative to a second coordinate system, a rotation feature of the first coordinate system at the same moment relative to the second coordinate system, position information of the target three-dimensional point in the ith image, a translation feature of the first coordinate system at the jth image moment relative to the initial first coordinate system, a rotation feature of the first coordinate system at the jth image moment relative to the initial first coordinate system, position information of the target three-dimensional point in the jth image, a time difference between the ith image moment and the jth image moment, And determining the initial speed characteristic and the initial gravity characteristic by the time difference between the jth image moment and the initial moment, the time difference between the ith image moment and the initial moment and the first acceleration integral between the first coordinate system of the ith image moment and the first coordinate system of the jth image moment, wherein the jth image moment is one of the image moments, the ith moment is a moment except the jth image moment in the image moments, i is not less than 1, and j is not less than 2.

7. An initial feature determination device, comprising: the device comprises an acquisition module and a determination module;

the acquisition module is configured to acquire a relative posture and position information of a target three-dimensional point in a plurality of images, wherein the relative posture is a posture of a first coordinate system relative to a second coordinate system at the same time, the first coordinate system is a coordinate system corresponding to an image sensor included in electronic equipment, the second coordinate system is a coordinate system corresponding to an inertial sensor included in the electronic equipment, and the plurality of images are respectively acquired by the image sensor at a plurality of image moments, and each image moment is a moment when the image sensor acquires one image;

the determining module is configured to determine a pose of a first coordinate system of each of the plurality of image moments with respect to an initial first coordinate system, a time difference between each of the plurality of image moments and an initial moment, a time difference between a first image moment and a second image moment, and a first acceleration integral between a first coordinate system of the first image moment and a first coordinate system of the second image moment, the first image moment being one of the plurality of image moments, the second image moment being a moment of the plurality of image moments other than the first image moment, the initial moment being a first one of the plurality of image moments, the initial first coordinate system being the first coordinate system of the initial moment;

the determination module is further configured to determine an initial velocity feature and an initial gravity feature according to the relative pose, the position information of the target three-dimensional point in the plurality of images, the pose of the first coordinate system of each of the plurality of image moments relative to the initial first coordinate system, the time difference between each of the plurality of image moments and the initial moment, the time difference between the first image moment and the second image moment, and the first acceleration integral between the first coordinate system of the first image moment and the first coordinate system of the second image moment, wherein the initial velocity feature is a velocity feature of the electronic device at the initial moment, and the initial gravity feature is a gravity feature of the electronic device at the initial moment.

8. An electronic device, characterized in that the electronic device comprises:

a processor;

a memory configured to store the processor-executable instructions;

wherein the processor is configured to execute the instructions to implement the initial feature determination method of any one of claims 1-6.

9. A computer-readable storage medium having instructions stored thereon, wherein the instructions in the computer-readable storage medium, when executed by an electronic device, enable the electronic device to perform an initial feature determination method as claimed in any one of claims 1-6.

10. A computer program product, characterized in that the computer program product comprises computer instructions which, when run on an electronic device, cause the electronic device to perform the initial feature determination method according to any one of claims 1-6.

Images