CN117671004A

CN117671004A - Wearing equipment posture estimation method, electronic equipment, storage medium and wearing system

Info

Publication number: CN117671004A
Application number: CN202311633556.XA
Authority: CN
Inventors: 董海青; 郝冬宁; 蔡勇亮
Original assignee: Hubei Xingji Meizu Technology Co ltd
Current assignee: Hubei Xingji Meizu Technology Co ltd
Priority date: 2023-11-29
Filing date: 2023-11-29
Publication date: 2024-03-08

Abstract

The disclosure provides a wearing equipment posture estimation method, which can be applied to the technical field of intelligent wearing equipment. The method comprises the following steps: determining a first local coordinate system of a plurality of marking points on the wearing equipment in a world coordinate system and a second local coordinate system of the wearing equipment in a camera coordinate system, wherein the first local coordinate system and the second local coordinate system are coordinate systems taking corresponding marking points as origins; according to the first base vector of the first local coordinate system and the second base vector of the second local coordinate system, matching a first mark point in the world coordinate system with a second mark point in the camera coordinate system to obtain a mark point pair; and estimating the posture of the wearable equipment according to the mark point pairs matched by any two frames of the multi-frame camera pictures. The disclosure also provides a wearable device posture estimation electronic device, a storage medium, a program product and a wearable system.

Description

Wearing equipment posture estimation method, electronic equipment, storage medium and wearing system

Technical Field

The disclosure relates to the technical field of intelligent wearing equipment, in particular to a wearing equipment posture estimation method and a wearing system.

Background

With the rapid development of man-machine interaction technology, intelligent head-mounted devices are increasingly and widely applied to daily lives of people, such as watching movies, playing games, navigating road conditions and the like by using augmented reality (Augmented Reality, AR) glasses.

The mainstream 3DOF (D egrees Of Freedom, degree of freedom) pose estimation algorithm for AR (Augmented Reality, reality augmentation) glasses mainly relies on IMU (Inertial measurement unit inertial measurement unit), i.e. internal fusion of gyroscope, accelerometer, magnetometer. This estimation method may cause gradual degradation of estimation accuracy with an increase in the running time. In addition, under certain application scenes, the state estimation of the monocular AR glasses needs to be performed in 6 degrees of freedom, but on the premise of no inertial measurement unit, estimated absolute scale information is difficult to obtain, so that the application range of the AR glasses is limited.

In addition, a gesture estimation method based on feature points can be adopted, and the method mainly comprises the steps of feature extraction, feature description, abnormal point removal, gesture estimation and the like, wherein the feature point extraction and description are mainly influenced by illumination, environment textures and the like, the illumination change is obvious, and the similarity of the environment textures can reduce the accuracy and the stability of the AR glasses gesture estimation. Moreover, the platform computing force of the AR glasses is often low, and the pose estimation method based on the feature points needs high platform computing force, which is difficult to meet the requirement of real-time estimating the AR pose.

Disclosure of Invention

In view of the above, the present disclosure provides a wearable device posture estimation method, an electronic device, a storage medium, a program product, and a wearable system that improve posture estimation efficiency and accuracy, for at least partially solving the above technical problems.

According to a first aspect of the present disclosure, there is provided a wearable device posture estimation method, including: determining a first local coordinate system of a plurality of marking points on the wearing equipment in a world coordinate system and a second local coordinate system of the wearing equipment in a camera coordinate system, wherein the first local coordinate system and the second local coordinate system are coordinate systems taking corresponding marking points as origins; according to the first base vector of the first local coordinate system and the second base vector of the second local coordinate system, matching a first mark point in the world coordinate system with a second mark point in the camera coordinate system to obtain a mark point pair; and estimating the posture of the wearable equipment according to the mark point pairs matched by any two frames of the multi-frame camera pictures.

According to an embodiment of the present disclosure, determining a local coordinate system of a plurality of marker points includes: determining coordinates of a plurality of mark points; determining the relative position relation between the corresponding non-origin point and the origin point when each marking point is taken as the origin point according to the coordinates of the plurality of marking points; determining a base vector of an origin mark point according to the relative position relation; determining a local coordinate system of the origin mark point according to the base vector; the local coordinate system comprises a first local coordinate system and a second local coordinate system, and the coordinates comprise first coordinates of a plurality of mark points in a world coordinate system and second coordinates in a camera coordinate system.

According to an embodiment of the present disclosure, determining coordinates of a plurality of marker points includes: acquiring a mark point coordinate file of the wearing equipment; determining a first coordinate according to the mark point coordinate file; and performing corner detection on the camera picture to determine a second coordinate.

According to an embodiment of the present disclosure, matching a first marker point in a world coordinate system with a second marker point in a camera coordinate system according to a first base vector of a first local coordinate system and a second base vector of a second local coordinate system, obtaining a marker point pair includes: performing vector space transformation on the first base vector and the second base vector; and determining the first mark point and the second mark point as mark point pairs under the condition that the first base vector and the second base vector are collinear or perpendicular.

According to an embodiment of the present disclosure, estimating a pose of a wearable device according to a marker point pair matched to any two frames of a multi-frame camera frame includes: determining a first gesture of the wearable device in the m-th frame of camera picture according to the second mark point in the m-th frame of camera picture and the first mark point in the corresponding world coordinate system; determining a second gesture of the wearable device in the nth frame of camera picture according to the second mark point in the nth frame of camera picture and the first mark point in the corresponding world coordinate system; according to the first gesture and the second gesture, determining the rotation amount and the displacement amount of the wearing equipment from the m-th frame of camera picture to the n-th frame of camera picture; and estimating the posture of the wearing equipment according to the rotation amount and the displacement amount from the m-th frame camera picture to the n-th frame camera picture; wherein m is less than n, and the difference between m and n is less than a first preset threshold.

According to an embodiment of the present disclosure, estimating a pose of a wearable device according to a marker point pair matched to any two frames of a multi-frame camera frame includes: respectively determining a second mark point in the m-th frame of camera picture and a second mark point in the n-th frame of camera picture according to the mark point pairs; respectively acquiring a first pixel gray value of a second mark point in an mth frame of camera picture and a second pixel gray value of the second mark point in an nth frame of camera picture; translating and/or rotating the nth frame of camera frame to minimize errors of the first pixel gray value and the second pixel gray value; according to the translation amount and/or rotation amount of the nth frame of camera picture, carrying out posture estimation on the wearable equipment; wherein m < n, the difference between m and n is smaller than a first preset threshold, and the camera frame rate is larger than a second preset threshold.

According to an embodiment of the present disclosure, the mth frame of camera frames and the nth frame of camera frames are adjacent frame of camera frames.

According to an embodiment of the present disclosure, estimating a pose of a wearable device according to a marker point pair matched to any two frames of a multi-frame camera frame includes: constructing a matrix equation set according to the marked point pairs; and solving a matrix equation set by adopting a least square method to obtain the posture of the wearable device.

A second aspect of the present disclosure provides a wearing system, comprising: a wearable device, a camera, and a processor; wherein the camera is for taking a picture of the wearable device and the processor is for performing the method as in any of the embodiments of the present disclosure.

A third aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method of any of the embodiments described above.

A fourth aspect of the present disclosure also provides a computer readable storage medium having stored thereon executable instructions which, when executed by a processor, cause the processor to perform the method of any of the embodiments described above.

A fifth aspect of the present disclosure also provides a computer program product comprising a computer program which, when executed by a processor, implements the method of any of the embodiments described above.

Compared with the prior art, the wearing equipment posture estimation method, the electronic equipment, the storage medium, the program product and the wearing system have at least the following beneficial effects:

(1) According to the wearing equipment posture estimation method, the mark points with the known coordinates are added on the intelligent wearing equipment (such as the glasses frame of the AR glasses), the local coordinate system of the current mark point is constructed, and the mark points are represented by the two basis vectors of the local coordinate system, so that the influence of illumination, environmental textures and the like can be avoided, and the accuracy of feature point description is improved. And then the same mark points in different frames are matched, and the posture of the intelligent wearing equipment is estimated through the relative position change of the same mark points, so that the accuracy of the posture estimation of the intelligent wearing equipment is improved.

(2) According to the method for estimating the posture of the wearable device, the frequency of posture estimation is close to the frame rate of the camera, namely the platform calculation force required by the method is very low, and the real-time requirement on posture estimation of the intelligent wearable device can be met.

(3) According to the wearable equipment posture estimation method, measurement data of the IMU is not needed, and the application scene of equipment posture estimation is widened.

Drawings

The foregoing and other objects, features and advantages of the disclosure will be more apparent from the following description of embodiments of the disclosure with reference to the accompanying drawings, in which:

fig. 1A schematically illustrates an application scenario diagram of a wearable device pose estimation method, an electronic device, a storage medium, a program product, and a wearable system according to an embodiment of the present disclosure; fig. 1B schematically illustrates a schematic diagram of a wearable device posture estimation method according to an embodiment of the present disclosure;

fig. 2A schematically illustrates a flowchart of a wearable device pose estimation method according to an embodiment of the present disclosure; fig. 2B schematically illustrates a flowchart of a method of wearable device pose estimation according to another embodiment of the present disclosure;

FIG. 3 schematically illustrates a method flow diagram for determining a local coordinate system of a plurality of marker points, in accordance with an embodiment of the present disclosure;

FIG. 4 schematically illustrates a method flow diagram for determining coordinates of a plurality of marker points, in accordance with an embodiment of the present disclosure;

FIG. 5 schematically illustrates a method flow diagram of matching resulting marker pairs according to an embodiment of the present disclosure;

fig. 6 schematically illustrates a method flow diagram for pose estimation of a wearable device according to an embodiment of the disclosure;

fig. 7 schematically illustrates a flow chart of a method of pose estimation of a wearable device according to another embodiment of the disclosure;

fig. 8 schematically illustrates a method flow diagram for pose estimation of a wearable device according to yet another embodiment of the present disclosure;

fig. 9 schematically illustrates a block diagram of a processor in a wearable system according to an embodiment of the disclosure; and

fig. 10 schematically illustrates a block diagram of an electronic device adapted to implement a wearable device pose estimation method according to an embodiment of the disclosure.

Detailed Description

Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is only exemplary and is not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. In addition, in the following description, descriptions of well-known structures and techniques are omitted so as not to unnecessarily obscure the concepts of the present disclosure.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and/or the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.

All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It should be noted that the terms used herein should be construed to have meanings consistent with the context of the present specification and should not be construed in an idealized or overly formal manner.

Where expressions like at least one of "A, B and C, etc. are used, the expressions should generally be interpreted in accordance with the meaning as commonly understood by those skilled in the art (e.g.," a system having at least one of A, B and C "shall include, but not be limited to, a system having a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).

The wearable device posture estimation method provided by the embodiment of the disclosure can be applied to mobile phones, tablet computers, wearable devices (such as augmented reality (augmented reality, AR)/Virtual Reality (VR) devices), vehicle-mounted devices, notebook computers, ultra-mobile personal computer (UMPC), netbooks, personal digital assistants (personal digital assistant, PDA) and other terminal devices, and can also be applied to databases, servers and service response systems based on terminal artificial intelligence.

The mobile terminals (terminal devices) in the embodiments of the present disclosure include various handheld devices, vehicle mount devices, wearable devices, computing devices, or other processing devices connected to a wireless modem, such as cell phones, tablets, desktop notebooks, and smart devices that can run applications, including a central console of a smart car, etc. Specifically, it may refer to a User Equipment (UE), an access terminal, a subscriber unit, a subscriber station, a mobile station, a remote terminal, a mobile device, a user terminal, a wireless communication device, a user agent, or a user equipment. The terminal device may also be a satellite phone, a cellular phone, a smart phone, a wireless data card, a wireless modem, a machine type communication device, a cordless phone, a session initiation protocol (session initiation protocol, SIP) phone, a wireless local loop (wireless local loop, WLL) station, a personal digital assistant (personal digital assistant, PDA), a handheld device with wireless communication capabilities, a computing device or other processing device connected to a wireless modem, a vehicle mounted device or a wearable device, a Virtual Reality (VR) terminal device, an augmented reality (augmented reality, AR) terminal device, a wireless terminal in industrial control (industrial control), a wireless terminal in self-driving (self-driving), a wireless terminal in telemedicine (remote media), a wireless terminal in smart grid (smart grid), a wireless terminal in transportation security (transportation safety), a wireless terminal in smart city (smart city), a future terminal in smart home (smart home) network, a terminal in a 5G network or a communication network, etc. The mobile terminal may be powered by a battery, may also be attached to and powered by a power system of the vehicle or vessel. The power supply system of the vehicle or the ship may also charge the battery of the mobile terminal to extend the communication time of the mobile terminal.

Fig. 1A schematically illustrates an application scenario diagram of a wearable device posture estimation method, an electronic device, a storage medium, a program product, and a wearable system according to an embodiment of the present disclosure. Fig. 1B schematically illustrates a schematic diagram of a wearing device posture estimation method according to an embodiment of the present disclosure.

As shown in fig. 1A, an application scenario 100 according to this embodiment may include a smart wearable device 10, a stand 20, a camera 30, and a control device 40. Wherein, the intelligent wearing apparatus 10 is fixedly arranged on the object placing table 20. The control device 40 can control the object placing table 20 to move in a six-degree-of-freedom manner, so as to drive the intelligent wearable device 10 to rotate and/or move. The camera 30 is fixedly arranged in the scene and is used for shooting the intelligent wearable device 10 and transmitting the shot picture to the control device 40 for data processing.

As shown in fig. 1B, a plurality of marker points may be set on the smart wearable device 10, and coordinates of the plurality of marker points may be acquired from a map point file. The map point file is, for example, a coordinate information file of the recorded mark point when the industrial design is performed on the glasses. The camera 30 is adopted to shoot the camera 30, and the local coordinate system of the marked point in the current frame is constructed, and the two base vectors of the local coordinate system are used for representing the point, so that the influence of illumination, environmental textures and the like can be avoided, and the accuracy of describing the characteristic points is improved. By acquiring the relative position changes (R represents rotation, T represents displacement, T represents a transformation matrix, i.e., pose x) of a plurality of marker points on the smart wearable device 10 of different frames, the pose of the smart wearable device 10 can be accurately estimated in real time.

In embodiments of the present disclosure, the smart wearable device 10 may be smart glasses, such as AR glasses, or may be general glasses, such as myopia glasses, presbyopic glasses, sunglasses, and the like. The glasses may be conventional glasses without display capability, such as lenses being conventional optical lenses such as myopia lenses, hyperopia lenses, plano lenses, etc.; or glasses that do not have display capabilities but have audio capabilities, such as bluetooth glasses. The glasses may also be glasses with display capability, for example, the lenses are devices with display capability, such as Micro OLED display screen, micro LED display screen, LCD display screen, OLED display screen, etc.; or the lens is a device having display capability, such as a display source by Liquid Crystal On Silicon (LCOS), liquid Crystal Display (LCD), digital Micromirror Device (DMD), digital Light Processing (DLP), silicon-based OLED (OLED), micro LED, micro OLED, etc., the capability of being projected through an optical waveguide, etc., to form a display screen to the eyes of a user, such as AR glasses. The AR glasses will be described below as an example.

Although AR glasses are exemplified herein, other types of smart glasses, such as VR (Virtual Reality) glasses, MR (Mediated Reality) glasses, XR (Extended Reality) glasses, and the like, are still contemplated by embodiments of the present disclosure.

It can be appreciated that, in addition to the above-described simulated test environment, the wearable device posture estimation method of the present disclosure may also be applied to, for example, a car driving navigation scenario. When a user drives a vehicle by wearing the AR glasses, a camera is fixedly arranged at the driving position of the vehicle, and the AR glasses are shot to accurately estimate the posture of the AR glasses in real time.

It should be noted that, the execution subject of the wearable device posture estimation method may be an intelligent wearable device (for example, AR glasses), a vehicle, or an edge device in communication connection with the intelligent wearable device. The edge device may be a terminal, server, or other device having computing resources.

The wearing device posture estimation method of the disclosed embodiment will be described in detail below by fig. 2 to 8 based on the scenes described in fig. 1A and 1B.

Fig. 2A schematically illustrates a flowchart of a wearable device pose estimation method according to an embodiment of the present disclosure. Fig. 2B schematically illustrates a flowchart of a method of wearable device pose estimation according to another embodiment of the present disclosure.

As shown in fig. 2A, the wearing device posture estimation method of this embodiment includes, for example, operations S210 to S230, and the wearing device posture estimation method may be executed by a computer program on corresponding computer hardware.

In operation S210, a first local coordinate system of a plurality of mark points on the wearable device in a world coordinate system and a second local coordinate system in a camera coordinate system, each of which is a coordinate system with the corresponding mark point as an origin, are determined.

In operation S220, a first marker point in the world coordinate system and a second marker point in the camera coordinate system are matched according to the first base vector of the first local coordinate system and the second base vector of the second local coordinate system, so as to obtain a marker point pair.

In operation S230, the pose estimation is performed on the wearable device according to the marker point pairs matched to any two frames of the multi-frame camera frame.

In some embodiments, as shown in FIG. 2B, for example, the AR glasses mark point coordinate file, i.e., the map point file, is first read and saved to the global variable P _w ，P _w ＝{P ₁ ，P ₂ …P _n }. Then, a local coordinate system of map points is constructed, and two basis vectors of the local coordinate system of each point are obtained. Then the camera is used for obtaining the image of the AR glasses, the marked points are extracted, and a current point set P is constructed _c ＝{p ₁ ，p ₂ …p _n }. Then, a local coordinate system of each point of the current frame is constructed, and two basis vectors of the local coordinate system of each point are obtained. And finally, matching the base vector of each point of the current frame with the base vectors of all map points, estimating the gesture by using the matched point pairs, and outputting the final gesture. The method of the embodiment of the disclosure can relatively accurately identify the mark points before and after the gesture change when the gesture of the AR glasses is changed, and is more robust than other methods for describing the feature points, so that the stability and accuracy of the gesture estimation of the AR glasses are indirectly improved.

For example, in a car navigation scene, the positions and attitudes of some special mark points, such as special stickers or graduation marks, can be set on the frame of the AR glasses, and then the positions and attitudes of the mark points are captured by a camera.

The specific implementation steps are as follows:

the AR glasses frame is provided with a plurality of marking points, and the marking points may be distributed according to a specific rule, such as equidistance arrangement or distributed according to a certain shape. These marker points may be used as reference points for determining the pose of the AR glasses. The mark point position and pose on the AR glasses are captured by the camera. In the capturing process, it is necessary to ensure that the camera can clearly capture the details of the mark points and can accurately measure the positions and attitudes of the mark points.

Using the algorithms in the present disclosure, the captured marker point positions and poses are analyzed and processed. First, it is necessary to convert the captured image into a digital signal, and then calculate the position and posture of each marker point by an algorithm. And estimating the posture of the AR glasses according to the calculated mark point position and posture information. For example, the rotation angle or translation distance of the AR glasses can be deduced by comparing the position changes of the mark points in the two frames of images.

By the method, the gesture of the AR glasses can be accurately estimated by capturing the position and gesture information of the mark points on the AR glasses, so that more accurate and real-time navigation service is provided for a driver. For example, when the posture of the AR glasses is changed, the displayed road indication direction or the real-time road sign identification information can be timely adjusted according to the captured position and posture information of the mark points, so as to help the driver to better understand the driving environment.

In the implementation process, the following points should be noted: establishing a local coordinate system: how to select points around the feature points to build a local coordinate system is a key issue. Points directly adjacent to the feature point may be selected or points further away may be selected in some way, such as using gradient or color information of the pixel. The method selected should be determined according to the specific application scenario and image content. Matching of the mark points: it is also an important issue how to determine whether two marker points are identical between two frames of images. In addition to comparing whether the axes of the local coordinate system are parallel or perpendicular, other properties, such as color, shape, etc., may be compared.

Fig. 3 schematically illustrates a flow chart of a method of determining a local coordinate system of a plurality of marker points, according to an embodiment of the disclosure.

According to an embodiment of the present disclosure, the local coordinate system includes, for example, a first local coordinate system and a second local coordinate system, as shown in fig. 3, and the local coordinate systems of the plurality of marker points are determined, for example, through operations S311 to S314.

In operation S311, coordinates of a plurality of mark points are determined. Wherein the coordinates include a first coordinate of the plurality of marker points in a world coordinate system and a second coordinate in a camera coordinate system.

In operation S312, a relative positional relationship between the corresponding non-origin mark point and the origin mark point when each mark point is taken as the origin is determined according to the coordinates of the plurality of mark points.

In operation S313, a base vector of the origin mark point is determined according to the relative positional relationship.

In operation S314, a local coordinate system of the origin mark point is determined according to the base vector.

For example, the world coordinate system is to determine the positions of points in space according to a cartesian right-hand coordinate system, the X-axis is the horizontal direction, the Y-axis is the vertical direction, the Z-axis is the direction perpendicular to the XY-plane, and the coordinates of the origin O are (0, 0). The world coordinate system is a fixed and invariable coordinate system and is the default coordinate system.

For example, in a car navigation scene, the method of "determining a local coordinate system of a plurality of marker points" may be applied by:

first, a plurality of mark points are set on a frame of AR glasses, and coordinates of each mark point are determined. These coordinates may be local coordinates with respect to the marker point itself or may be coordinates with respect to the world coordinate system.

Then, by calculating the relative positional relationship of each marker point with other marker points, the local coordinate system of each marker point can be determined. For example, if point P is marked ₁ The coordinates of (0, 0) and the mark point P ₂ Where the coordinates of (1, 0) are then P can be determined ₂ Point relative to P ₁ The positional relationship of the dots is 1 unit distance along the positive x-axis direction.

By determining the basis vector for each marker point, the direction and size of each local coordinate system can be described. For example, two basis vectors of a local coordinate system with the current marker point as the origin can be calculated by:

first, coordinate information of all the marker points is collected, and then a matrix is constructed by taking the current marker point as an origin, wherein each row represents a position vector of one marker point relative to the origin. Subtracting the current point from all points to construct a matrix J with the current point as an origin to construct a map point P ₁ For example, a local coordinate system of:

J ₁ ＝[P ₂ -P ₁ ，P ₃ -P ₁ ，…，P _n -P ₁ ] _2*(N-1) (1)

the variance matrix of the matrix may then be calculated. The variance matrix is a matrix for measuring the variation degree of elements in the matrix, and can be used for describing the relation among the elements in the matrix. Building a variance matrix H ₁ ：

Finally, singular Value Decomposition (SVD) is carried out on the variance matrix to obtain H ₁ ＝U∑V ^T A set of pairwise orthogonal unit vector sequences U and V may be obtained, which may represent the relationship between the individual position vectors in the original matrix. Extracting two mutually perpendicular basis vectors from a unit vector sequence obtained by singular value decomposition to obtain a current marking point P ₁ Is the two basis vectors of the local coordinate system of the origin. The method can respectively obtain the world coordinate system and the local coordinate system of all the marking points under the current frame camera coordinate system.

Through the above steps, a local coordinate system of a plurality of mark points on the AR glasses can be determined. These local coordinate systems may be used to describe changes in the pose of the AR glasses and may be used to estimate the pose of the AR glasses. For example, when the AR glasses are rotated or translated, the change in the posture of the AR glasses may be inferred from the local coordinate system of the marker points, and the navigation directions or real-time road sign identification information may be adjusted according to these information.

Fig. 4 schematically illustrates a method flowchart of determining coordinates of a plurality of marker points, according to an embodiment of the present disclosure.

According to an embodiment of the present disclosure, as shown in fig. 4, coordinates of a plurality of mark points are determined, for example, through operations S4111 to S4113.

In operation S4111, a mark point coordinate file of the wearable device is acquired.

In operation S4112, the first coordinates are determined from the marker point coordinate file.

In operation S4113, corner detection is performed on the camera frame to determine the second coordinates.

For example, taking AR glasses as an example, how to apply these steps will be described in detail: in AR glasses applications, some marking points may be predefined, for example, at specific positions of the glasses frame, and then coordinate information of these marking points may be stored in a coordinate file. When the pose of the AR glasses needs to be determined, a marker point coordinate file may be opened, and the first coordinates of each marker point may be extracted. These first coordinates may be local coordinates with respect to the marker point itself or coordinates with respect to the world coordinate system.

After the first coordinates of each mark point are extracted, corner detection can be performed on the camera picture to determine the second coordinates of each mark point. Corner detection is an image processing technique that can be used to detect key points in an image, such as edge intersections, corner points, etc. A corner detection algorithm may be used to perform corner detection on the camera view and extract the second coordinates of each marker point.

In some embodiments, FAST algorithm may be used to perform corner detection on the camera view. In the FAST algorithm, a pixel point is first selected from an image as a corner point, and then the pixels around the pixel point are detected to determine whether the pixel point is the corner point.

Specifically, the FAST algorithm is processed as follows:

any one pixel point in the image can be selected as a corner point, and then surrounding pixel points are detected. For the pixel points around the corner points, the response values of the pixel points can be calculated according to the distance and direction relation between the pixel points and the corner points. The higher the response value, the closer the distance between the pixel point and the corner point is, and the more consistent the direction is. According to a certain threshold condition, a pixel point with a response value higher than the threshold value can be determined as a candidate point of the corner point. Then, the candidate points may be further screened to obtain final corner points. In addition, after loading the image file, the image may be preprocessed to remove noise and interference factors.

After the corner coordinates are obtained, the second coordinates of the marker point can be determined using these coordinate information. For example, the coordinates of the corner point of each marker point may be used as the second coordinates and then used for pose estimation together with the first coordinates. Thus, the second coordinates of the mark point can be obtained by using the FAST algorithm, and the pose of the AR glasses can be determined by using the coordinate information. The method of the present disclosure can also accurately estimate the scale information of the current AR glasses due to the known coordinates and the relative position determination.

It can be appreciated that when performing corner detection by applying the FAST algorithm, the corner detection can be implemented by using an existing open source library or writing codes by itself.

Through the above steps, the first and second coordinates of each mark point can be acquired, and the pose of the AR glasses can be determined using these coordinate information. For example, the rotation angle or translation distance of the AR glasses may be inferred by comparing the difference between the first coordinates and the second coordinates of the two mark points. Meanwhile, according to the first coordinate information and the second coordinate information of the plurality of mark points, more accurate posture estimation can be performed on the AR glasses.

Fig. 5 schematically illustrates a flow chart of a method of matching resulting marker pairs according to an embodiment of the present disclosure.

According to an embodiment of the present disclosure, as shown in fig. 5, a first marker point in the world coordinate system and a second marker point in the camera coordinate system are matched, for example, by operations S521 to S522, resulting in a marker point pair.

In operation S521, a vector space transformation is performed on the first base vector and the second base vector.

In operation S522, in the case where the first base vector and the second base vector are collinear or perpendicular, the first mark point and the second mark point are determined as a mark point pair.

In some embodiments, the marking point P is extracted with the current frame ₁ For example (other point matching processes are similar), its local coordinate system has a matrix of two basis vectors of b ₁ ＝[e ₁ ，e ₂ ] _2*2 Will b ₁ Local coordinate systems { B } respectively with map points ₁ ，B ₁ ，…，B _n }，B _n ＝[E _1n ，E _2n ] _2*2 Multiplying. Wherein e ₁ 、e ₂ Marking a point P for the current frame ₁ Two basis vectors of the local coordinate system of B _n Two basis vectors E of the local coordinate system for the nth map point _1n And E is _2n A matrix is formed. Theoretically, only b ₁ And B is connected with ₁ The result of the multiplication is 0 and 90 degrees (because the AR glasses motion belongs to rigid motion, no matter what posture the AR glasses are in, the relative posture between the mark points on the AR glasses cannot change, namely the local coordinate system is kept unchanged), b ₁ Other than other coordinate systems, because b ₁ And the matching is completed without collineation or verticality with other coordinate systems.

For example, an AR glasses, four points (i.e., marked points captured by the camera coordinate system) P have been marked on the frame of the glasses ₁ 、P ₂ 、P ₃ 、P ₄ And defines four corresponding mark points P in the world coordinate system ₁ ′、P ₂ ′、P ₃ ′、P ₄ '. The pose of the AR glasses is estimated by these marker points.

First, coordinate information of each marker point is acquired. For the mark points on the AR glasses, the coordinate position of each mark point in the camera coordinate system may be recorded. For example, suppose P ₁ The local coordinates of the point are (0, 0), P ₂ The local coordinates of the point are (1, 0), P ₃ The local coordinates of the points are (1, 1), P ₄ The local coordinates of the point are (0, 1). At the same time, it is also necessary to know the coordinate position of each marker point in the world coordinate system. For example, suppose P ₁ The world coordinates of the' point are (0, 0), P ₂ The world coordinates of the' point are (1, 0), P ₃ The world coordinates of the' point are (1, 1), P ₄ The world coordinates of the' point are (0, 1).

Next, a basis vector matrix for each local coordinate system is constructed. For the mark points on the AR glasses, a local coordinate system can be constructed with each mark point as an origin. Then, two basis vectors for each local coordinate system can be calculated and form a 2x2 matrix. For example, in P ₁ The basis vector matrix of the local coordinate system with the point as the origin can be expressed as:

wherein the first row represents a base vector in the x-axis direction and the second row represents a base vector in the y-axis direction. In this example, the elements of the basis vector matrix are either 1 or 0, with the elements of the first row and first column and the second row and second column being 1, indicating the x-axis and y-axis directions. The other position element is 0, indicating that there is no base vector in this coordinate system in this direction.

Similarly, other base vector matrices of marker points may be obtained.

Then, vector space transformation is required. Specifically, each base vector matrix is multiplied by the base vector matrix in the corresponding world coordinate system to obtain a new matrix. For example, P is ₁ Basis vector matrix of points [ [1,0 ]]，[0，1]]And P ₁ Basis vector matrix of' points [ [1,0 ]]，[0，1]]Multiplying to obtain a new matrix:

similarly, the transformation results of other marker points can be obtained.

By observing the transformed matrix, it can be found that if two base vectors of a certain mark point are parallel or perpendicular to two base vectors in the corresponding world local coordinate system, the axis of the local coordinate system is parallel or perpendicular to the corresponding world officeThe axes of the partial coordinate systems are parallel or perpendicular. For example, by observing the above transformation results, it can be found that P is ₁ The axis of the local coordinate system with the point as the origin is parallel to the axis of the coordinate system with the point as the origin ₁ The 'point is the axis of the world's local coordinate system of the origin; with P ₂ The axis of the local coordinate system with the point as the origin is perpendicular to the axis of the coordinate system with the point as the origin ₂ The 'point is the axis of the world's local coordinate system of the origin.

From the above results, a set of marker pairs can be determined. For example, P can be ₁ Point and P ₁ ' points are determined as a set of marker point pairs; will P ₂ Point and P ₂ The' points are determined as a set of marker point pairs. Similarly, other marker point pairs may also be determined.

Finally, pose estimation can be performed by these marker point pairs. For example, if the AR glasses are rotated by an angle θ about the y-axis, then P ₁ Point and P ₁ The x-coordinates of the' points should differ by θ, while P ₂ Point and P ₂ The x-coordinates of the' points should be equal. Similarly, pose estimation may also be performed based on other marker point pairs.

Through the above steps, a set of marker pairs can be determined using the marker points on the AR glasses and their corresponding marker points in the world coordinate system, and the pose estimation can be performed by these marker pairs.

Fig. 6 schematically illustrates a flow chart of a method of pose estimation for a wearable device according to an embodiment of the disclosure.

According to an embodiment of the present disclosure, as shown in fig. 6, the posture of the wearing device is estimated, for example, through operations S631 to S634.

In operation S631, the first pose of the wearable device in the mth frame of camera frame is determined according to the second mark point in the mth frame of camera frame and the first mark point in the corresponding world coordinate system.

In operation S632, a second pose of the wearable device in the nth frame of camera frame is determined according to the second mark point in the nth frame of camera frame and the first mark point in the corresponding world coordinate system.

In operation S633, the rotation amount and the displacement amount of the wearable device from the m-th frame camera screen to the n-th frame camera screen are determined according to the first pose and the second pose. And

In operation S634, the pose estimation of the wearable device is performed according to the rotation amount and the displacement amount from the mth frame camera view to the nth frame camera view. Wherein m is less than n, and the difference between m and n is less than a first preset threshold.

For example, an AR glasses, a camera is used to capture an image of the AR glasses and the marker points are identified by image processing techniques. There are multiple frames of camera frames from which the pose of the AR glasses is estimated.

According to the matched mark point pairs, the positions of the second mark points in the m-th frame of camera images and the n-th frame of camera images can be respectively determined. The world coordinate positions of these marker points can then be determined from their positions in the respective local coordinate systems. For example, perspective transformation or camera calibration techniques may be used to convert marker points in the local coordinate system into the world coordinate system.

Next, the rotation amount and displacement amount of the AR glasses from the m-th frame camera view to the n-th frame camera view may be determined according to the first pose and the second pose. Specifically, the rotation amount may be represented using a quaternion or euler angle, and the displacement amount may be derived by calculating a positional change of the marker point in the three-dimensional space.

Finally, the pose of the AR glasses may be estimated from the rotation amount and the displacement amount from the m-th frame camera view to the n-th frame camera view. For example, a Kalman filter or an extended Kalman filter may be used to predict the pose of AR glasses and use this predicted value to control the perspective or position in a virtual reality scene.

It will be appreciated that the two frames should be as close as possible, i.e. the difference between m and n should be as small as possible, to improve the accuracy of the pose estimation.

The above procedure is estimated based on the change in posture between successive frames. If the time interval between two frames is long, or there is a strong motion, inaccuracy in the pose estimation may result. Therefore, in practical application, an appropriate method and technique need to be selected according to the specific situation to perform pose estimation.

Fig. 7 schematically illustrates a flow chart of a method of pose estimation for a wearable device according to another embodiment of the disclosure.

According to an embodiment of the present disclosure, as shown in fig. 7, the posture of the wearing device is estimated, for example, through operations S731 to S734.

In operation S731, a second mark point in the m-th frame camera view and a second mark point in the n-th frame camera view are determined according to the mark point pairs, respectively.

In operation S732, the first pixel gray value of the second mark point in the m-th frame camera frame and the second pixel gray value of the second mark point in the n-th frame camera frame are acquired, respectively.

In operation S733, the nth frame of camera frame is translated and/or rotated to minimize the error of the first pixel gray value and the second pixel gray value. And

In operation S734, the pose estimation is performed on the wearable device according to the translational amount and/or the rotational amount of the nth frame of camera screen. Wherein m < n, the difference between m and n is smaller than a first preset threshold, and the camera frame rate is larger than a second preset threshold.

For example, to provide more accurate pose estimation results, a direct approach may also be employed to pose estimate the wearable device.

According to the matched mark point pairs, a second mark point in the m-th frame of camera picture and a second mark point in the n-th frame of camera picture can be respectively determined. These marker points are specific points on the wearable device whose location is known relative to its local coordinate system.

For each second marker point, its first pixel gray value in the mth frame of camera frame and its second pixel gray value in the nth frame of camera frame may be acquired. These pixel gray values are the luminance information of the marker point in the respective camera frame.

The error of the first pixel gray value and the second pixel gray value can be minimized by performing a translation and/or rotation operation on the nth frame of camera frame. This is accomplished by an optimization algorithm, such as least squares.

By means of a translation and/or rotation operation, a new position of the second mark point in the nth frame of camera picture can be obtained. Then, the change in posture of the wearable device can be calculated from this new position and the position of the second mark point in the m-th frame camera screen. For example, the relative displacement or angular change between the two marker points may be calculated to estimate the pose.

Finally, the estimated pose changes may be used to update the pose of the AR glasses. This change in pose may be applied, for example, to the control system of AR glasses or to the virtual reality scene.

The above process is also based on the change of the marker point between the successive frames to make the pose estimation. In order to obtain a reliable pose estimation result, accurate control and management of the camera frame rate and the recognition accuracy of the marker point are also required.

Preferably, the mth frame of camera frames and the nth frame of camera frames are adjacent frame of camera frames. The benefits of using adjacent frames are mainly the following:

The calculated amount is reduced: neighboring frames typically have a high similarity, and thus this feature can be used to reduce the amount of computation of pose estimation. For example, feature extraction and matching can be avoided in each frame by performing feature matching and tracking of matching points between adjacent frames.

Accuracy and stability are improved: the change of the gesture between the adjacent frames is usually continuous, so that the accuracy and stability of the gesture estimation can be improved by using this characteristic. For example, noise and fluctuations in the pose estimation can be reduced by smoothing the pose changes between adjacent frames.

Adapting to dynamic environment: in a dynamic environment, the gesture of an object changes at any time, and adjacent frames can better capture the change. Therefore, the method using the adjacent frames is more suitable for the attitude estimation of the dynamic environment.

Real-time performance: the time interval between adjacent frames is typically short, so that real-time pose estimation can be achieved using this feature. For example, online learning and model updating can be performed between adjacent frames, thereby continuously optimizing the effect of pose estimation.

In summary, the method of using adjacent frames has many advantages in pose estimation, including reducing the amount of computation, improving accuracy and stability, adapting to dynamic environment, and realizing real-time performance. Therefore, in practical application, a proper method and technology can be selected according to specific requirements to perform attitude estimation.

Fig. 8 schematically illustrates a flow chart of a method of pose estimation for a wearable device according to yet another embodiment of the disclosure.

According to an embodiment of the present disclosure, as shown in fig. 8, the posture estimation of the wearable device is performed, for example, through operations S831 to S832.

In operation S831, a matrix equation set is constructed from the labeled point pairs.

In operation S832, a least square method is adopted to solve a matrix equation set to obtain a posture of the wearable device.

For example, from the labeled point pairs, a system of matrix equations is constructed:

A ^T Ax＝A ^T b (3)

wherein, the matrix A is a matrix formed by the marking points of the camera coordinate system, b is a vector formed by the marking points of the world coordinate system, x is the gesture to be estimated, and the gesture can be obtained by solving the equation set.

For example, a system of equations may be constructed to estimate pose based on the position of the marker points in two frames. Specifically, this system of equations may be solved using a least squares method. Assuming that there are N marker points, the system of equations can be reduced to:

Ax＝b (4)

wherein A is a matrix of N×M, N is the number of marker points, and M is the number of pose parameters. x is an M-dimensional vector representing the pose parameters. b is an N-dimensional vector representing the position of the marker point in the world coordinate system. This system of equations can be solved using a least squares method to obtain an optimal solution for the pose parameter x. Specifically, algorithms such as QR decomposition, SVD decomposition, etc. may be used to solve this system of equations. The QR decomposition may decompose the matrix a into a quadrature matrix Q and an upper triangular matrix S, so that the solution of the equation set is more stable and reliable.

Finally, according to the obtained gesture parameter x, the gesture of the AR glasses can be estimated. In particular, this gesture parameter may be applied to the control system of AR glasses or to the virtual reality scene.

It should be noted that this example is only a simple demonstration, and many more factors, such as the moving speed of the camera, the recognition accuracy of the mark point, and the image quality, need to be considered in practical application. Therefore, in practical application, an appropriate method and technique need to be selected according to the specific situation to perform pose estimation.

Based on the wearing equipment posture estimation method, the disclosure further provides a wearing system. The device will be described in detail below in connection with fig. 9.

Fig. 9 schematically shows a block diagram of a processor in a wearable system according to an embodiment of the disclosure.

As shown in fig. 9, the wearing system of this embodiment includes, for example: wearing equipment, camera and processor. Wherein the camera is for taking a picture of the wearable device and the processor is for performing the method as in any of the embodiments of the present disclosure. The wearing system processor 900 includes, for example: a determination module 910, a matching module 920, and an estimation module 930.

The determining module 910 is configured to determine a first local coordinate system of the world coordinate system of the plurality of marker points on the wearable device and a second local coordinate system of the camera coordinate system, where the first local coordinate system and the second local coordinate system are both coordinate systems with the corresponding marker points as origins. In an embodiment, the determining module 910 may be configured to perform the operation S210 described above, which is not described herein.

The matching module 920 is configured to match a first marker point in the world coordinate system with a second marker point in the camera coordinate system according to the first base vector of the first local coordinate system and the second base vector of the second local coordinate system, so as to obtain a marker point pair. In an embodiment, the matching module 920 may be configured to perform the operation S220 described above, which is not described herein.

The estimation module 930 is configured to perform pose estimation on the wearable device according to the marker point pair matched by any two frames of the multi-frame camera frame. In an embodiment, the estimation module 930 may be configured to perform the operation S230 described above, which is not described herein.

Any of the determination module 910, the matching module 920, and the estimation module 930 may be combined in one module to be implemented, or any of the modules may be split into a plurality of modules according to an embodiment of the present disclosure. Alternatively, at least some of the functionality of one or more of the modules may be combined with at least some of the functionality of other modules and implemented in one module. According to embodiments of the present disclosure, at least one of the determination module 910, the matching module 920, and the estimation module 930 may be implemented at least in part as hardware circuitry, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system-on-chip, a system-on-substrate, a system-on-package, an Application Specific Integrated Circuit (ASIC), or in hardware or firmware, such as any other reasonable manner of integrating or packaging the circuitry, or in any one of or a suitable combination of any of three implementations of software, hardware, and firmware. Alternatively, at least one of the determination module 910, the matching module 920, and the estimation module 930 may be at least partially implemented as computer program modules, which when executed, may perform the corresponding functions.

As shown in fig. 10, an electronic device 1000 according to an embodiment of the present disclosure includes a processor 1001 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 1002 or a program loaded from a storage section 1008 into a Random Access Memory (RAM) 1003. The processor 1001 may include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or an associated chipset and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), or the like. The processor 1001 may also include on-board memory for caching purposes. The processor 1001 may include a single processing unit or multiple processing units for performing different actions of the method flows according to embodiments of the present disclosure.

In the RAM 1003, various programs and data necessary for the operation of the electronic apparatus 1000 are stored. The processor 1001, the ROM 1002, and the RAM 1003 are connected to each other by a bus 1004. The processor 1001 performs various operations of the method flow according to the embodiment of the present disclosure by executing programs in the ROM 1002 and/or the RAM 1003. Note that the program may be stored in one or more memories other than the ROM 1002 and the RAM 1003. The processor 1001 may also perform various operations of the method flow according to the embodiments of the present disclosure by executing programs stored in the one or more memories.

According to an embodiment of the disclosure, the electronic device 1000 may also include an input/output (I/O) interface 1005, the input/output (I/O) interface 1005 also being connected to the bus 1004. The electronic device 1000 may also include one or more of the following components connected to the I/O interface 1005: an input section 1006 including a keyboard, a mouse, and the like; an output portion 1007 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), etc., and a speaker, etc.; a storage portion 1008 including a hard disk or the like; and a communication section 1009 including a network interface card such as a LAN card, a modem, or the like. The communication section 1009 performs communication processing via a network such as the internet. The drive 1010 is also connected to the I/O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like, is installed as needed in the drive 1010, so that a computer program read out therefrom is installed as needed in the storage section 1008.

The present disclosure also provides a computer-readable storage medium that may be embodied in the apparatus/device/system described in the above embodiments; or may exist alone without being assembled into the apparatus/device/system. The computer-readable storage medium described above carries one or more programs that, when executed, implement the wearable device posture estimation method according to the embodiments of the present disclosure.

According to embodiments of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, which may include, for example, but is not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. For example, according to embodiments of the present disclosure, the computer-readable storage medium may include ROM 1002 and/or RAM 1003 and/or one or more memories other than ROM 1002 and RAM 1003 described above.

Embodiments of the present disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowcharts. When the computer program product runs in a computer system, the program code is used for enabling the computer system to realize the wearing equipment posture estimation method provided by the embodiment of the disclosure.

The above-described functions defined in the system/apparatus of the embodiments of the present disclosure are performed when the computer program is executed by the processor 1001. The systems, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the disclosure.

In one embodiment, the computer program may be based on a tangible storage medium such as an optical storage device, a magnetic storage device, or the like. In another embodiment, the computer program may also be transmitted in the form of signals on a network medium, distributed, and downloaded and installed via the communication section 1009, and/or installed from the removable medium 1011. The computer program may include program code that may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.

In such an embodiment, the computer program may be downloaded and installed from a network via the communication portion 1009, and/or installed from the removable medium 1011. The above-described functions defined in the system of the embodiments of the present disclosure are performed when the computer program is executed by the processor 1001. The systems, devices, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the disclosure.

According to embodiments of the present disclosure, program code for performing computer programs provided by embodiments of the present disclosure may be written in any combination of one or more programming languages, and in particular, such computer programs may be implemented in high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. Programming languages include, but are not limited to, such as Java, c++, python, "C" or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the Internet using an Internet service provider).

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

Those skilled in the art will appreciate that the features recited in the various embodiments of the disclosure and/or in the claims may be provided in a variety of combinations and/or combinations, even if such combinations or combinations are not explicitly recited in the disclosure. In particular, the features recited in the various embodiments of the present disclosure and/or the claims may be variously combined and/or combined without departing from the spirit and teachings of the present disclosure. All such combinations and/or combinations fall within the scope of the present disclosure.

The embodiments of the present disclosure are described above. However, these examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the embodiments cannot be used advantageously in combination. The scope of the disclosure is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be made by those skilled in the art without departing from the scope of the disclosure, and such alternatives and modifications are intended to fall within the scope of the disclosure.

Claims

1. A method for estimating a pose of a wearable device, comprising:

determining a first local coordinate system of a plurality of marking points on the wearing equipment in a world coordinate system and a second local coordinate system in a camera coordinate system, wherein the first local coordinate system and the second local coordinate system are coordinate systems taking corresponding marking points as origins;

According to the first base vector of the first local coordinate system and the second base vector of the second local coordinate system, matching a first mark point in the world coordinate system with a second mark point in the camera coordinate system to obtain a mark point pair; and

and estimating the gesture of the wearable equipment according to the mark point pairs matched by any two frames of the multi-frame camera picture.

2. The method of claim 1, wherein determining the local coordinate system of the plurality of marker points comprises:

determining coordinates of the plurality of marker points;

determining the relative position relationship between the corresponding non-origin mark point and the origin mark point when each mark point is taken as the origin according to the coordinates of the plurality of mark points;

determining a base vector of the origin mark point according to the relative position relation; and

determining a local coordinate system of the origin mark point according to the base vector;

wherein the local coordinate system includes the first local coordinate system and the second local coordinate system, the coordinates including first coordinates of the plurality of marker points in the world coordinate system and second coordinates in the camera coordinate system.

3. The method of claim 2, wherein the determining coordinates of the plurality of marker points comprises:

acquiring a mark point coordinate file of the wearable equipment;

determining the first coordinates according to the mark point coordinate file; and

and detecting the corner point of the camera picture, and determining the second coordinate.

4. The method of claim 1, wherein the matching the first marker point in the world coordinate system with the second marker point in the camera coordinate system based on the first base vector of the first local coordinate system and the second base vector of the second local coordinate system, the obtaining the marker point pair comprises:

performing vector space transformation on the first base vector and the second base vector;

and determining the first mark point and the second mark point as the mark point pair under the condition that the first base vector and the second base vector are collinear or perpendicular.

5. The method of claim 4, wherein the estimating the pose of the wearable device according to the marker point pairs matched to any two frames of a multi-frame camera frame comprises:

determining a first gesture of the wearable device in the camera picture of the m frame according to the second mark point in the camera picture of the m frame and the corresponding first mark point in the world coordinate system;

Determining a second gesture of the wearable device in the camera picture of the nth frame according to the second mark point in the camera picture of the nth frame and the corresponding first mark point in the world coordinate system;

determining the rotation amount and the displacement amount of the wearable device from the m-th frame of camera picture to the n-th frame of camera picture according to the first gesture and the second gesture; and

according to the rotation amount and the displacement amount from the m-th frame camera picture to the n-th frame camera picture, carrying out gesture estimation on the wearable equipment;

wherein m is less than n, and the difference between m and n is less than a first preset threshold.

6. The method of claim 4, wherein the estimating the pose of the wearable device according to the marker point pairs matched to any two frames of a multi-frame camera frame comprises:

respectively determining the second mark point in the m-th frame of camera picture and the second mark point in the n-th frame of camera picture according to the mark point pairs;

respectively acquiring a first pixel gray value of the second mark point in an mth frame of camera picture and a second pixel gray value of the second mark point in an nth frame of camera picture;

translating and/or rotating the nth frame of camera frame to minimize an error of the first pixel grayscale value and the second pixel grayscale value; and

According to the translation amount and/or rotation amount of the nth frame of camera picture, carrying out posture estimation on the wearable equipment;

wherein m < n, the difference between m and n is smaller than a first preset threshold, and the camera frame rate is larger than a second preset threshold.

7. The method of claim 5 or 6, wherein the mth frame of camera frames and the nth frame of camera frames are adjacent frame of camera frames.

8. The method of claim 4, wherein the estimating the pose of the wearable device according to the marker point pairs matched to any two frames of a multi-frame camera frame comprises:

constructing a matrix equation set according to the marked point pairs;

and solving the matrix equation set by adopting a least square method to obtain the gesture of the wearable device.

9. A wearable system, comprising:

a wearable device, a camera, and a processor;

wherein the camera is for taking a photograph of the wearable device and the processor is for performing the method of any of claims 1-8.

10. An electronic device, comprising:

one or more processors;

storage means for storing one or more programs,

Wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method of any of claims 1-8.

11. A computer readable storage medium having stored thereon executable instructions which, when executed by a processor, cause the processor to perform the method according to any of claims 1-8.

12. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 8.