CN114721511A - Method and device for positioning three-dimensional object - Google Patents
Method and device for positioning three-dimensional object Download PDFInfo
- Publication number
- CN114721511A CN114721511A CN202210261706.8A CN202210261706A CN114721511A CN 114721511 A CN114721511 A CN 114721511A CN 202210261706 A CN202210261706 A CN 202210261706A CN 114721511 A CN114721511 A CN 114721511A
- Authority
- CN
- China
- Prior art keywords
- dimensional object
- virtual
- posture
- virtual object
- dimensional
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/01—Indexing scheme relating to G06F3/01
- G06F2203/012—Walk-in-place systems for allowing a user to walk in a virtual environment while constraining him to a given position in the physical environment
Abstract
The invention discloses a method for positioning a three-dimensional object, which comprises the following steps: acquiring an image of a three-dimensional object by using a single camera, and identifying the posture and the position of the three-dimensional object, wherein the outer surface of the three-dimensional object is divided into a plurality of areas, and the colors of adjacent areas are different; displaying a virtual object corresponding to the three-dimensional object; and after the change of the posture and/or the position of the three-dimensional object in the three-dimensional space is detected, adjusting the virtual object according to the change quantity of the posture and/or the position. Correspondingly, the embodiment of the invention provides a device for positioning a three-dimensional object, which solves the problem that the change of a virtual object cannot be adaptively adjusted according to the dynamic change of the three-dimensional object in a single-camera scene in the prior art.
Description
Technical Field
The invention belongs to the technical field of augmented reality, and particularly relates to a method and a device for positioning a three-dimensional object.
Background
In the current augmented reality application, a three-dimensional object may be acquired through a camera, the three-dimensional object is recognized, and a corresponding virtual three-dimensional object is displayed in a virtual scene, for example, an image of a hexahedron may be acquired, and a corresponding virtual object, such as a virtual hexahedron, a virtual character, a virtual globe, and the like, may be displayed in a display screen.
In the prior art, a virtual object can be displayed in a manner of acquiring a three-dimensional object, but the change of the virtual object cannot be adaptively adjusted according to the dynamic change of the three-dimensional object in a single-camera use scene, so that the cost is high, and the position and the posture of the three-dimensional object are difficult to finely recognize.
Disclosure of Invention
The invention provides a method and a device for positioning a three-dimensional object, which solve the problem that the change of a virtual object cannot be adaptively adjusted according to the dynamic change of the three-dimensional object in a single-camera scene in the prior art.
In order to achieve the above object, the present invention provides a method for positioning a three-dimensional object, comprising:
acquiring an image of a three-dimensional object by using a single camera, and identifying the posture and the position of the three-dimensional object, wherein the outer surface of the three-dimensional object is divided into a plurality of areas, and the colors of adjacent areas are different;
displaying a virtual object corresponding to the three-dimensional object;
and after the change of the posture and/or the position of the three-dimensional object in the three-dimensional space is detected, adjusting the virtual object according to the change quantity of the posture and/or the position.
In one embodiment, the adjusting the virtual object includes:
rotating the virtual object in a virtual space, wherein the rotation angle and the angular velocity of the virtual object correspond to the posture variation of the three-dimensional object, or,
and moving the position of the virtual object in the virtual space, wherein the displacement amount of the virtual object corresponds to the position variation amount of the three-dimensional object.
In one embodiment, after the adjusting the virtual object, the method further includes:
when the camera acquires different combinations of different color surfaces of the three-dimensional object, different virtual scenes are entered according to a preset instruction, or,
displaying a first virtual scene when the angular velocity of rotation of the virtual object exceeds a first preset threshold, or,
and when the rotation angular velocity of the virtual object is lower than a second preset threshold value, displaying a second virtual scene, wherein the first preset threshold value is larger than the second preset threshold value.
In one embodiment, the adjusting the virtual object according to the amount of change in the posture and/or the position includes:
and adjusting the size of the virtual object according to the depth-of-field distance between the three-dimensional object and the camera.
In one embodiment, recognizing the pose and position of the three-dimensional object comprises:
carrying out color block segmentation on an image, and decomposing the image into a plurality of areas with different colors;
averaging the colors of each region, and traversing all adjacent color block pairs;
screening the color block pairs by using a table look-up method, and screening out an area matched with a preset model;
and calculating orientation data of the matching area, and acquiring the position and the posture of the three-dimensional object.
In one embodiment, after the calculating the orientation data of the matching area, the method further includes:
calculating a candidate solution corresponding to the matching area;
pairwise comparing the compatibility of the candidate solutions, discarding any one of the two compatible candidate solutions;
identifying edge pixels between the pair of color patches using an edge detection algorithm;
optimizing the position and pose of the three-dimensional object using an optimization formula, the optimization formula being:
where P is a position and attitude parameter of the three-dimensional object, includingPosition coordinates (x, y, z) and attitude angle (q)w，qx，qy，qz) (ii) a f is a projection function for calculating the point X on the three-dimensional object surface when the three-dimensional object is in the P postureiThe position of (a); e, calculating the difference between the projection position and the observation position by taking the cost function as an E; x is the number ofiAnd thetaiIs an edge pixel point, x, detected in the imageiIs the coordinate of the edge point in the image, θiIs the tangent angle of the edge point.
In one embodiment, after the color block segmentation is performed on the image, the method further comprises:
establishing a model of the three-dimensional object;
traversing adjacent surfaces in the model, and recording the color pairs and the surface orientations of the adjacent surfaces;
and recording the coordinate information of the boundary of all adjacent surfaces.
The embodiment of the invention also provides a method for positioning the three-dimensional object, which comprises the following steps:
acquiring an image of a three-dimensional object by using a single camera, wherein the outer surface of the three-dimensional object is divided into a plurality of areas, and the colors of the adjacent areas are different;
recording two or more adjacent surfaces in the three-dimensional object;
carrying out color block segmentation on an image, and decomposing the image into a plurality of areas with different colors;
averaging the colors of each region, and traversing all adjacent color block pairs;
screening the color block pairs by using a table look-up method, and screening out an area matched with a preset model;
calculating orientation data of the matching area, and acquiring the position and the posture of the three-dimensional object;
and displaying a virtual object corresponding to the three-dimensional object.
In one embodiment, after the calculating the orientation data of the matching area, the method further includes:
calculating a candidate solution corresponding to the matching area;
pairwise comparing the compatibility of the candidate solutions, discarding any one of the two compatible candidate solutions;
identifying edge pixels between the pair of color patches using an edge detection algorithm;
optimizing the position and pose of the three-dimensional object using an optimization formula, the optimization formula being:
where P is the position and attitude parameters of the three-dimensional object, including position coordinates (x, y, z) and attitude angle (q)w，qx，qy，qz) (ii) a f is a projection function for calculating a point X on the surface of the three-dimensional object when the three-dimensional object is in the P postureiThe position of (a); e, calculating the difference between the projection position and the observation position by taking the cost function as an E; x is the number ofiAnd thetaiIs an edge pixel point, x, detected in the imageiIs the coordinate of the edge point in the image, θiIs the tangent angle of the edge point.
Embodiments of the present invention further provide a positioning apparatus for a three-dimensional object, the apparatus including a processor and a memory for storing a computer program capable of running on the processor; wherein the processor is configured to execute the method for positioning a three-dimensional object when running the computer program.
The embodiment of the invention also provides a computer-readable storage medium, on which computer-executable instructions are stored, and the computer-executable instructions are used for executing the method for positioning the three-dimensional object.
The embodiment of the invention provides a method and a device for positioning a three-dimensional object, wherein the method identifies different color combination areas of the three-dimensional object through a single camera, determines the spatial attitude and the position of the three-dimensional object, displays a corresponding virtual object, can also determine the spatial attitude and the position of the three-dimensional object in a refined manner, and can also measure the depth of field of the three-dimensional object through the single camera, so that the method and the device are low in cost and high in precision.
Drawings
FIG. 1 is a flow chart of a method for locating a three-dimensional object in an embodiment of the invention;
FIG. 2 is a flow chart of a method for recognizing the pose and position of a three-dimensional object according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of three-dimensional object recognition in an embodiment of the present invention;
FIG. 4 is a further illustration of three-dimensional object recognition in an embodiment of the present invention;
FIG. 5 is a schematic diagram of an embodiment of the present invention for optimizing pose by using an edge of a color block of a three-dimensional object;
FIG. 6 is a schematic structural diagram of a three-dimensional object positioning device according to an embodiment of the present invention;
FIG. 7 is a schematic diagram of a three-dimensional object positioning device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
To achieve the above object, as shown in fig. 1, an embodiment of the present invention provides an application program recommendation method, including:
s101, acquiring an image of a three-dimensional object by using a single camera, and identifying the posture and the position of the three-dimensional object, wherein the outer surface of the three-dimensional object is divided into a plurality of areas, and the colors of adjacent areas are different;
in the embodiment of the present invention, the three-dimensional object positioning and the virtual object display may be implemented by a three-dimensional object positioning device, optionally, the positioning device includes a single camera, a processing unit, and a display unit, the single camera collects an image of the three-dimensional object in a detection area, the processing unit implements image processing and finally determines the posture and position of the three-dimensional object, and creates a virtual object identical to or corresponding to the three-dimensional object in a virtual scene by means of an augmented reality AR technology. The three-dimensional object may be a polyhedron, such as a sphere, cylinder, tetrahedron, hexahedron, or the like. In the embodiment of the invention, the orientation and the posture of the three-dimensional object surface facing the camera can be determined through the color combination of each surface in the rotating or moving process of the three-dimensional object. For convenience of description, the embodiments of the present invention are described by taking a hexahedron as an example, and the rest of the three-dimensional objects (with different color surface combinations) are also within the scope of the embodiments of the present invention.
As shown in fig. 2, in one embodiment, the recognizing the pose and the position of the three-dimensional object may specifically be:
s201, carrying out color block segmentation on the image, and decomposing the image into a plurality of areas with different colors;
as shown in fig. 3, the image may be a yellow color region, a green color region, a blue color region, a black color region, a white color region, and a red color region.
Furthermore, after color block segmentation, a model of the three-dimensional object may also be built; traversing adjacent surfaces in the model, and recording the color pairs and the surface orientations of the adjacent surfaces; and recording the coordinate information of the boundary of all adjacent surfaces. That is, in the embodiment of the present invention, adjacent faces in the three-dimensional object may be traversed, color pairs (such as "green-red", "red-yellow", and the like) of the adjacent faces and orientations of the faces may be recorded, and a table may be built for use in a subsequent algorithm; at the same time, the coordinate information of the boundary of all adjacent surfaces (represented by a series of discrete sampling points) is recorded for subsequent accurate measurement.
When the camera can see two or more surfaces simultaneously, the position and the posture (namely the orientation) of the mark can be preliminarily judged through color combination, and then more accurate position and posture data can be obtained through an iterative algorithm.
S202, averaging the colors of each region, and traversing all adjacent color block pairs (pair);
s203, screening the color block pairs by using a table look-up method, and screening out an area matched with a preset model;
and (3) screening color block pairs by using a table look-up method, for example, "red-white" is matched with the three-dimensional object model, but "red-blue" and "green-purple" are not matched with the model (the former is because red and blue are not adjacent in the model, and the latter is because purple is not available in the model), discarding the color block pairs which are not matched, and screening out an area (color block pair) matched with a preset model.
And S204, calculating orientation data of the matching area, and acquiring the position and the posture of the three-dimensional object.
After screening, some candidate color block pairs are left, for example, "red-white", "red-black", "black-white", and the like are obtained in fig. 4, and the orientation data of the surfaces can be found out by using a table look-up method, so that the approximate direction of the camera relative to the mark can be known; meanwhile, due to the fact that the data of the two surfaces exist simultaneously, the approximate rotation angle of the camera (the rotation angle with the connecting line of the camera and the mark as an axis) can be calculated, and further the approximate position and the posture of the three-dimensional object can be calculated.
In the embodiment of the invention, the position and the posture can be further optimized, so that the posture and the position of the three-dimensional object in the space can be obtained in a refined manner. The specific method comprises the following steps:
after calculating the orientation data of the matching area, the embodiment of the present invention further includes:
s2041, calculating a candidate solution corresponding to the matching area;
s2042, pairwise comparing the compatibility of the candidate solutions, and discarding any one of the two compatible candidate solutions;
when the positions of the markers obtained by the two candidate solutions are closer to the posture (the difference between the distance and the angle is smaller than a certain threshold), the two candidate solutions can be considered to be compatible, that is, the two candidate solutions are color block pairs on the same marker. One of the solutions may be discarded at this point.
S2043, identifying edge pixels between the color block pairs by using an edge detection algorithm;
s2044, optimizing the position and the posture of the three-dimensional object by using an optimization formula, wherein the optimization formula is as follows:
where P is the position and attitude parameters of the three-dimensional object, including position coordinates (x, y, z) and attitude angle (q)w，qx，qy，qz) (ii) a f is a projection function for calculating the point X on the three-dimensional object surface when the three-dimensional object is in the P postureiThe position of (a); e, calculating the difference between the projection position and the observation position by taking the cost function as an E; x is the number ofiAnd thetaiIs an edge pixel point, x, detected in the imageiIs the coordinate of the edge point in the image, θiIs the tangent angle of the edge point. After the above formula is subjected to iterative optimization by using a Levenberg-Marquardt algorithm, a better P can be obtained*And obtaining the optimized position and posture of the marker.
After the attitude optimization is completed through an iterative algorithm, accurate attitude data of the three-dimensional object can be obtained.
In addition, for video data, kalman filtering (kalman filter) can be used to smooth the poses of successive frames, and finally a relatively stable sequence of poses can be obtained.
S102, displaying a virtual object corresponding to the three-dimensional object;
in the embodiment of the invention, the AR technology can be used for displaying the virtual object corresponding to the three-dimensional object in the virtual scene after the posture and the position of the three-dimensional object are recognized. For example, a hexahedron having the same size and shape as the three-dimensional object is displayed, and the color of each surface is identical to that of the actual three-dimensional object.
S103, after the change of the posture and/or the position of the three-dimensional object in the three-dimensional space is detected, the virtual object is adjusted according to the change of the posture and/or the position.
In one embodiment, the adjusting the virtual object may specifically be:
rotating the virtual object in a virtual space, wherein the rotation angle and the angular velocity of the virtual object correspond to the posture variation of the three-dimensional object, or,
and moving the position of the virtual object in the virtual space, wherein the displacement amount of the virtual object corresponds to the position variation amount of the three-dimensional object.
In one embodiment, after the adjusting the virtual object, the method further includes:
when the camera acquires different combinations of different color surfaces of the three-dimensional object, different virtual scenes are entered according to a preset instruction, or,
displaying a first virtual scene when the rotation angular velocity of the virtual object exceeds a first preset threshold, or,
and when the rotation angular velocity of the virtual object is lower than a second preset threshold value, displaying a second virtual scene, wherein the first preset threshold value is larger than the second preset threshold value.
In one embodiment, the adjusting the virtual object according to the variation of the posture and/or the position includes:
and adjusting the size of the virtual object according to the depth-of-field distance between the three-dimensional object and the camera.
For example, in the field of games, the three-dimensional object may be in the shape of a polyhedron such as a magic wand, and the three-dimensional positioning method can be used to identify combinations of different face colors to determine which particular pose is currently facing the camera. When the user operates the magic wand, the virtual magic wand can be adaptively changed, for example, when the user rotates the magic wand, the virtual magic wand also synchronously rotates, and when the user moves the magic wand, the virtual magic wand also synchronously moves. Meanwhile, according to different distances, the depth of field of the actual magic stick relative to the camera can be measured through a single camera, the depth of field is different, the size of the virtual magic stick changes, and a user can use the characteristics to carry out different game experiences.
Meanwhile, a user can design different game experiences according to different characteristics of the magic wand, for example, the magic wand is rotated, the magic wand is switched to different sides before the camera is in front of the camera (namely, the color combinations collected by the camera are different), new interactive experiences can be designed, for example, when the magic wand is switched to the side A, a certain game is started, when the magic wand is switched to the side B, another game is started, when the magic wand is switched to the side B, an interactive virtual scene is generated, when the magic wand is switched to the side B, another scene is generated, meanwhile, when the rotating speed and the rotating angle are different, different games can be designed, namely, different angles and different speeds are used for rotating, the games corresponding to different sides are different, or the corresponding game playing methods are different.
The embodiment of the invention provides a method and a device for positioning a three-dimensional object, wherein the method identifies different color combination areas of the three-dimensional object through a single camera, determines the spatial attitude and the position of the three-dimensional object, displays a corresponding virtual object, can also determine the spatial attitude and the position of the three-dimensional object in a refined manner, and can also measure the depth of field of the three-dimensional object through the single camera, so that the method and the device are low in cost and high in precision.
In addition, the embodiment of the invention also provides a method for positioning the three-dimensional object, wherein the marked information is recorded in advance by a positioning algorithm, and the marked information comprises coordinates of all vertexes, parameter equations of edges and the color of a surface. The model is input with the following points:
1. traversing adjacent faces in the model, recording the color pairs (such as 'green-red', 'red-yellow' and the like) of the adjacent faces and the orientation of the faces, and establishing a table for a subsequent algorithm;
2. the coordinate information of the boundary of all adjacent surfaces (represented by a series of discrete sample points) is recorded for subsequent accurate measurement.
When the camera can see two or more surfaces simultaneously, the positions and postures (namely the orientations) of the marks can be preliminarily judged through color combination, and then more accurate position and posture data can be obtained through an iterative algorithm.
The method specifically comprises the following steps:
s501, acquiring an image of a three-dimensional object by using a single camera, wherein the outer surface of the three-dimensional object is divided into a plurality of areas, and the colors of adjacent areas are different;
s502, recording two or more adjacent surfaces in the three-dimensional object;
s503, carrying out color block segmentation on the image, and decomposing the image into a plurality of areas with different colors;
s504, averaging the colors of each region, and traversing all adjacent color block pairs;
for example, "red-white" is consistent with the model, but "red-blue" and "green-violet" are inconsistent with the model (the former because red and blue are not adjacent in the model, and the latter because there is no violet in the model).
S505, screening the color block pairs by using a table look-up method, and screening out an area matched with a preset model;
for example, "red-white", "red-black", "black-white", etc. are obtained in fig. 4, and the orientation data of these faces can be found by using a table look-up method, so as to know the approximate orientation of the camera with respect to the mark; meanwhile, due to the fact that the data of the two surfaces exist simultaneously, the approximate rotation angle of the camera (the rotation angle with the connecting line of the camera and the mark as an axis) can be calculated, and the approximate position and the posture of the marker can be further calculated.
S506, calculating orientation data of the matching area, and acquiring the position and the posture of the three-dimensional object;
when the positions of the markers obtained by the two candidate solutions are closer to the posture (the difference between the distance and the angle is smaller than a certain threshold), the two candidate solutions can be considered to be compatible, that is, the two candidate solutions are color block pairs on the same marker. One of the solutions may be discarded at this point.
And S507, displaying the virtual object corresponding to the three-dimensional object.
In one embodiment, after the calculating the orientation data of the matching area, the method further includes:
s5061, calculating a candidate solution corresponding to the matching area;
s5062, pairwise comparing the compatibility of the candidate solutions, discarding any one of the two compatible candidate solutions;
s5063, identifying edge pixels between the color lump pairs by using an edge detection algorithm;
s5064, optimizing the position and the posture of the three-dimensional object by using an optimization formula, wherein the optimization formula is:
where P is the position and attitude parameters of the three-dimensional object, including the position coordinates (x, y, z) and attitude angle (q)w，qx，qy，qz) The attitude angle here is expressed by a quaternion (quaternion); f is a projection function for calculating the point X on the marker in P attitudeiWhere on the image captured by the camera; e is a cost function, the difference between the projection position and the observation position is calculated, and the cost is larger when the difference is larger; x is the number ofiAnd thetaiIs the edge pixel point, x, detected in the imageiIs the coordinate of the edge point in the image, θiIs the tangent angle of the edge point. After the Levenberg-Marquardt algorithm is used for carrying out iterative optimization on the formula, a better P can be obtained*And obtaining the optimized position and posture of the marker.
In addition, for video data, kalman filtering (kalman filter) may be used to smooth the poses of successive frames, resulting in a relatively stable sequence of poses.
Fig. 5 is a schematic diagram illustrating a principle of posture optimization by using a three-dimensional object color block edge in an embodiment of the present invention, where a dotted line is a color block edge estimated from a current estimated posture, and a solid line is a color block edge obtained by actual shooting. And during optimization, the dotted line is closed to the solid line by adjusting the posture.
In addition, in the embodiment of the invention, the system can be combined with a remote control module to carry out human-computer interaction on the virtual object. A conventional remote controller has a modeled operation instruction set, and presses a button to generate an instruction signal and transmit the signal, so that a terminal such as a television, an air conditioner, or the like responds to the instruction. In the embodiment of the invention, a series of operation protocols can be set in a targeted manner, and the remote control command is combined with the space posture/position and the like of the three-dimensional object to form a set of new human-computer interaction protocols. For example, a series of instruction sets may be provided in the remote control module, and the resulting interactions at different spatial poses/positions may be differentiated. After a key moving to the right is pressed, if the position of the three-dimensional object is not detected to be updated, the virtual object moves to the right by one grid, and if the position of the three-dimensional object is detected to be updated, a key moving to the right is also pressed, and three grids can be moved to the right.
FIG. 6 is a schematic view of one of the three-dimensional positioning devices in an embodiment of the present invention; a user may operate an application with a device such as a cell phone, tablet, or computer. The device may include information entry devices such as cameras, microphones, application props, and/or AR devices. The apparatus may also include output devices such as a display device, speakers, etc.
The embodiment of the invention also provides a storage medium, which stores computer instructions, and the instructions are executed by a processor to realize the method for realizing the three-dimensional object positioning.
Fig. 7 is a schematic structural diagram of a system according to an embodiment of the present invention. The system 600 may include one or more Central Processing Units (CPUs) 610 (e.g., one or more processors) and memory 620, one or more storage media 630 (e.g., one or more mass storage devices) that store applications 632 or data 634. Memory 620 and storage medium 630 may be, among other things, transient or persistent storage. The program stored in the storage medium 630 may include one or more modules (not shown), each of which may include a series of instruction operations for the apparatus. Still further, the central processor 610 may be configured to communicate with the storage medium 630 to execute a series of instruction operations in the storage medium 630 on the device 600. The system 600 may further include one or more power supplies 640, one or more wired or wireless network interfaces 650, and one or more input/output interfaces 660. the steps performed by the above-described method embodiment for three-dimensional positioning may be based on the system architecture shown in fig. 7.
It should be understood that, in the various embodiments of the present application, the sequence numbers of the processes do not mean the execution sequence, and the execution sequence of the processes should be determined by the functions and the inherent logic thereof, and should not constitute any limitation to the implementation process of the embodiments of the present application.
Those of ordinary skill in the art will appreciate that the various illustrative modules and method steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application. It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the system, the apparatus and the module described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
All parts of the specification are described in a progressive mode, the same and similar parts of all embodiments can be referred to each other, and each embodiment is mainly introduced to be different from other embodiments. In particular, as to the apparatus and system embodiments, since they are substantially similar to the method embodiments, the description is relatively simple and reference may be made to the description of the method embodiments in relevant places.
Finally, it is to be noted that: the above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the scope of the application. To the extent that such modifications and variations of the present application fall within the scope of the claims and their equivalents, they are intended to be included within the scope of the present application.
Claims (7)
1. A method of three-dimensional object localization, comprising:
acquiring an image of a three-dimensional object by using a single camera, and identifying the posture and the position of the three-dimensional object, wherein the outer surface of the three-dimensional object is divided into a plurality of areas, and the colors of adjacent areas are different;
displaying a virtual object corresponding to the three-dimensional object;
after the change of the posture and/or the position of the three-dimensional object in the three-dimensional space is detected, adjusting the virtual object according to the change quantity of the posture and/or the position;
receiving an input instruction and generating an instruction code;
and adjusting the virtual object according to the instruction code and the variation of the posture and/or the position.
2. The method of claim 1, wherein the adjusting the virtual object comprises:
rotating the virtual object in a virtual space, wherein the rotation angle and the angular velocity of the virtual object correspond to the posture variation of the three-dimensional object, or,
and moving the position of the virtual object in the virtual space, wherein the displacement amount of the virtual object corresponds to the position variation amount of the three-dimensional object.
3. The method of claim 2, wherein after the adjusting the virtual object, further comprising:
when the camera acquires different combinations of different color surfaces of the three-dimensional object, different virtual scenes are entered according to a preset instruction, or,
displaying a first virtual scene when the rotation angular velocity of the virtual object exceeds a first preset threshold, or,
and when the rotation angular velocity of the virtual object is lower than a second preset threshold value, displaying a second virtual scene, wherein the first preset threshold value is larger than the second preset threshold value.
4. The method of claim 1, wherein the adjusting the virtual object according to the amount of change in the pose and/or position comprises:
and adjusting the size of the virtual object according to the depth-of-field distance between the three-dimensional object and the camera.
5. The method of claim 1, wherein identifying the pose and position of the three-dimensional object comprises:
carrying out color block segmentation on an image, and decomposing the image into a plurality of areas with different colors;
averaging the colors of each region, and traversing all adjacent color block pairs;
screening the color block pairs by using a table look-up method, and screening out an area matched with a preset model;
and calculating orientation data of the matching area, and acquiring the position and the posture of the three-dimensional object.
6. The method of claim 5, wherein after calculating orientation data for the matching region, further comprising:
calculating a candidate solution corresponding to the matching area;
pairwise comparing the compatibility of the candidate solutions, discarding any one of the two compatible candidate solutions;
identifying edge pixels between the pair of color patches using an edge detection algorithm;
optimizing the position and attitude of the three-dimensional object using an optimization formula, the optimization formula being:
where P is the position and attitude parameters of the three-dimensional object, including position coordinates (x, y, z) and attitude angle (q)w,qx,qy,qz) (ii) a f is a projection function for calculating the point X on the three-dimensional object surface when the three-dimensional object is in the P postureiThe position of (a); e, calculating the difference between the projection position and the observation position by taking the cost function as an E; x is the number ofiAnd thetaiIs an edge pixel point, x, detected in the imageiIs the coordinate of the edge point in the image, θiIs the tangent angle of the edge point.
7. A three-dimensional object positioning apparatus, characterized by an apparatus processor and a memory for storing a computer program operable on the processor; wherein the processor is adapted to perform the method of three-dimensional object localization of any of claims 1 to 6 when running the computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210261706.8A CN114721511A (en) | 2019-04-24 | 2019-04-24 | Method and device for positioning three-dimensional object |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910335679.2A CN110134234B (en) | 2019-04-24 | 2019-04-24 | Method and device for positioning three-dimensional object |
| CN202210261706.8A CN114721511A (en) | 2019-04-24 | 2019-04-24 | Method and device for positioning three-dimensional object |
Related Parent Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910335679.2A Division CN110134234B (en) | 2019-04-24 | 2019-04-24 | Method and device for positioning three-dimensional object |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114721511A true CN114721511A (en) | 2022-07-08 |
Family
ID=67570956
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210261706.8A Pending CN114721511A (en) | 2019-04-24 | 2019-04-24 | Method and device for positioning three-dimensional object |
| CN201910335679.2A Active CN110134234B (en) | 2019-04-24 | 2019-04-24 | Method and device for positioning three-dimensional object |
Family Applications After (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910335679.2A Active CN110134234B (en) | 2019-04-24 | 2019-04-24 | Method and device for positioning three-dimensional object |
Country Status (1)
| Country | Link |
|---|---|
| CN (2) | CN114721511A (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112440882B (en) * | 2019-09-04 | 2024-03-08 | 梅赛德斯-奔驰集团股份公司 | System and method for placing articles in car trunk and car comprising system |
| CN110928414A (en) * | 2019-11-22 | 2020-03-27 | 上海交通大学 | Three-dimensional virtual-real fusion experimental system |
Family Cites Families (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2477793A (en) * | 2010-02-15 | 2011-08-17 | Sony Corp | A method of creating a stereoscopic image in a client device |
| CN102074012B (en) * | 2011-01-22 | 2012-06-27 | 四川农业大学 | Method for three-dimensionally reconstructing tender shoot state of tea by combining image and computation model |
| JP2013101045A (en) * | 2011-11-08 | 2013-05-23 | Fanuc Ltd | Recognition device and recognition method of three-dimensional position posture of article |
| CN102737245B (en) * | 2012-06-06 | 2014-06-11 | 清华大学 | Three-dimensional scene object boundary detection method and device |
| US10509865B2 (en) * | 2014-09-18 | 2019-12-17 | Google Llc | Dress form for three-dimensional drawing inside virtual reality environment |
| CN104680519B (en) * | 2015-02-06 | 2017-06-23 | 四川长虹电器股份有限公司 | Seven-piece puzzle recognition methods based on profile and color |
| CN106447725B (en) * | 2016-06-29 | 2018-02-09 | 北京航空航天大学 | Spatial target posture method of estimation based on the matching of profile point composite character |
| US10055882B2 (en) * | 2016-08-15 | 2018-08-21 | Aquifi, Inc. | System and method for three-dimensional scanning and for capturing a bidirectional reflectance distribution function |
| CN107274453A (en) * | 2017-06-12 | 2017-10-20 | 哈尔滨理工大学 | Video camera three-dimensional measuring apparatus, system and method for a kind of combination demarcation with correction |
| CN108537149B (en) * | 2018-03-26 | 2020-06-02 | Oppo广东移动通信有限公司 | Image processing method, image processing device, storage medium and electronic equipment |
| CN109615655A (en) * | 2018-11-16 | 2019-04-12 | 深圳市商汤科技有限公司 | A kind of method and device, electronic equipment and the computer media of determining gestures of object |
-
2019
- 2019-04-24 CN CN202210261706.8A patent/CN114721511A/en active Pending
- 2019-04-24 CN CN201910335679.2A patent/CN110134234B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN110134234B (en) | 2022-05-10 |
| CN110134234A (en) | 2019-08-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20220003537A1 (en) | Method and apparatus for measuring geometric parameter of object, and terminal | |
| CN110517319B (en) | Method for determining camera attitude information and related device | |
| CN104956404B (en) | It is rebuild with the real-time three-dimensional that power effective depth sensor uses | |
| CN106705837B (en) | Object measuring method and device based on gestures | |
| CN110443853B (en) | Calibration method and device based on binocular camera, terminal equipment and storage medium | |
| US11308655B2 (en) | Image synthesis method and apparatus | |
| JP5778967B2 (en) | Information processing program, information processing method, information processing apparatus, and information processing system | |
| CN111935393A (en) | Shooting method, shooting device, electronic equipment and storage medium | |
| WO2021143282A1 (en) | Three-dimensional facial model generation method and apparatus, computer device and storage medium | |
| CN110825234A (en) | Projection type augmented reality tracking display method and system for industrial scene | |
| US10825249B2 (en) | Method and device for blurring a virtual object in a video | |
| CN107911621A (en) | A kind of image pickup method of panoramic picture, terminal device and storage medium | |
| CN109920004B (en) | Image processing method, device, calibration object combination, terminal equipment and calibration system | |
| WO2021136386A1 (en) | Data processing method, terminal, and server | |
| CN110134234B (en) | Method and device for positioning three-dimensional object | |
| WO2022152001A1 (en) | Gesture recognition method and apparatus, electronic device, readable storage medium, and chip | |
| CN110361005A (en) | Positioning method, positioning device, readable storage medium and electronic equipment | |
| WO2020232971A1 (en) | Fisheye camera calibration system, method and apparatus, and electronic device and storage medium | |
| JP2019197279A (en) | Image processing apparatus, method for controlling image processing apparatus, and program | |
| US8643679B2 (en) | Storage medium storing image conversion program and image conversion apparatus | |
| CN110152293B (en) | Method and device for positioning control object and method and device for positioning game object | |
| CN104243970A (en) | 3D drawn image objective quality evaluation method based on stereoscopic vision attention mechanism and structural similarity | |
| CN112912936A (en) | Mixed reality system, program, mobile terminal device, and method | |
| CN111179341B (en) | Registration method of augmented reality equipment and mobile robot | |
| JP7006810B2 (en) | 3D measuring device, mobile robot, push wheel type moving device and 3D measurement processing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |