CN114996492A - Map storage method, pose determination method and storage medium - Google Patents
Map storage method, pose determination method and storage medium Download PDFInfo
- Publication number
- CN114996492A CN114996492A CN202210581219.XA CN202210581219A CN114996492A CN 114996492 A CN114996492 A CN 114996492A CN 202210581219 A CN202210581219 A CN 202210581219A CN 114996492 A CN114996492 A CN 114996492A
- Authority
- CN
- China
- Prior art keywords
- image
- target object
- information
- position information
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/51—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/587—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using geographical or spatial information, e.g. location
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30244—Camera pose
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Library & Information Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The application provides a map storage method, a pose determination method and a storage medium, wherein the map storage method comprises the following steps: acquiring a map image to be processed, and segmenting a plurality of target objects included in the map image to be processed; determining image characteristic information and position information of a target object; and performing associated storage of the image characteristic information and the position information for each of the plurality of target objects.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a map storage method, a pose determination method, and a storage medium.
Background
Currently, electronic devices (e.g., AR (Augmented Reality) glasses, AR helmets, cameras, etc.) can capture and store map images. However, the conventional method for storing the map image has the problem of large storage data amount.
Disclosure of Invention
The application provides the following technical scheme:
one aspect of the present application provides a map storage method, including:
acquiring a map image to be processed, and segmenting a plurality of target objects included in the map image to be processed;
determining image characteristic information and position information of the target object;
and performing associated storage of the image characteristic information and the position information for each of the plurality of target objects.
The determining the position information of the target object comprises:
determining two-dimensional position information of the target object in a map image to be processed;
acquiring a first target image acquired before the map image to be processed, wherein the first target image at least comprises the target object;
determining two-dimensional position information of the target object in the first target image;
determining three-dimensional position information of the target object based on the two-dimensional position information of the target object in the map image to be processed and the two-dimensional position information of the target object in the first target image;
and determining the position information of the target object based on the three-dimensional position information and the image acquisition position information corresponding to the map image to be processed.
The method further comprises the following steps:
acquiring a plurality of second target images before the map image to be processed, wherein the second target images at least comprise the plurality of target objects;
and adjusting the three-dimensional position information of the target object based on the second target images and the map image to be processed.
The image feature information includes semantic feature information, and the determining the image feature information of the target object includes:
and inputting the target object into a neural network model to obtain the semantic feature information of the target object determined by the neural network model.
Another aspect of the present application provides a pose determination method, including:
acquiring a first image acquired by an image acquisition device, and segmenting a plurality of first objects included in the first image;
determining image characteristic information of the first object;
determining a target object associated with each of the plurality of first objects from map information obtained by performing the map storage method according to any one of claims 1 to 4, based on image characteristic information of the first object;
and determining the pose of the image acquisition device based on the position information of the target object, wherein the position information of the target object is prestored in the map information.
The determining, from map information, a target object associated with each of the plurality of first objects based on image feature information of the first object, includes:
matching a plurality of candidate target objects corresponding to each first object from the map information based on the image feature information of each first object, wherein the image feature information of each candidate target object and the image feature information of the first object meet a first approximation condition;
and determining the target object from the candidate target objects according to the similarity information of the image characteristic information of the candidate target objects and the image characteristic information of the first object.
The determining the pose of the image acquisition device based on the position information of the target object comprises:
determining two-dimensional position information of the first object in the first image;
determining a pose of the image acquisition device based on the position information of the target object and the two-dimensional position information of the first object in the first image.
The determining the pose of the image acquisition apparatus based on the position information of the target object and the two-dimensional position information of the first object in the first image comprises:
establishing geometric constraint information of the image acquisition device relative to the target object according to the position information of the target object and the two-dimensional position information of the first object in the first image;
determining a pose of the image capture device based on the geometric constraint information.
The determining the pose of the image capture device based on the position information of the target object includes:
obtaining a plurality of groups of target object combinations according to the target objects;
determining a plurality of candidate poses of the image acquisition device according to the plurality of groups of target object combinations;
determining a pose of the image capture device from difference data between the plurality of candidate poses.
A third aspect of the present application provides a storage medium storing a computer program implementing the map storage method according to any one of the above items or the pose determination method according to any one of the above items, the computer program being executed by a processor to implement the map storage method according to any one of the above items or the pose determination method according to any one of the above items.
According to the method and the device, the map image to be processed is obtained, the target objects included in the map image to be processed are segmented, the image characteristic information and the position information of the target objects are determined, the image characteristic information and the position information are stored in association with each of the target objects, data storage is achieved by taking the target objects as a unit, and the data size of the map image to be processed is reduced.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive labor.
Fig. 1 is a schematic flowchart of a map storage method provided in embodiment 1 of the present application;
FIG. 2 is a schematic view of an implementation scenario of a map storage method provided in the present application;
FIG. 3 is a schematic flow chart of a map storage method provided in embodiment 2 of the present application;
FIG. 4 is a schematic flow chart of a map storage method provided in embodiment 3 of the present application;
fig. 5 is a schematic flowchart of a pose determination method provided in embodiment 4 of the present application;
fig. 6 is a schematic view of an implementation scenario of a pose determination method provided by the present application;
fig. 7 is a schematic flowchart of a pose determination method provided in embodiment 5 of the present application;
fig. 8 is a schematic flowchart of a pose determination method provided in embodiment 6 of the present application;
FIG. 9 is a schematic diagram of a map storage device provided herein;
fig. 10 is a schematic structural diagram of a pose determination apparatus provided in the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only some embodiments of the present application, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
In order to make the aforementioned objects, features and advantages of the present application more comprehensible, the present application is described in further detail with reference to the accompanying drawings and the detailed description.
Referring to fig. 1, a schematic flowchart of a map storage method provided in embodiment 1 of the present application, the method may be applied to an electronic device, and the present application does not limit a product type of the electronic device, as shown in fig. 1, the method may include, but is not limited to, the following steps:
step S101, obtaining a map image to be processed, and segmenting a plurality of target objects included in the map image to be processed.
Optionally, the map image to be processed may be an image acquired by an image acquisition device, or may also be a map image acquired through other methods and channels, such as user uploading, network downloading, and the like.
Segmenting a plurality of target objects included in the map image to be processed, which may specifically include but is not limited to:
and S1011, inputting the map image to be processed into a pre-trained first neural network model to obtain a candidate frame of the target object contained in the map image to be processed determined by the first neural network model.
The first neural network model is obtained by utilizing a plurality of first training images and real frames of a plurality of target objects marked in each first training image in advance.
And S1012, segmenting the image in the candidate frame in the map image to be processed to obtain the target object.
Segmenting a plurality of target objects included in the map image to be processed, and specifically, but not limited to:
and S1013, inputting the map image to be processed into the first neural network model to obtain a candidate frame of the target object contained in the map image to be processed determined by the first neural network model and semantic feature information of the image in the candidate frame.
The first neural network model is obtained by training by utilizing a plurality of second training images and semantic feature information of real frames of a plurality of target objects marked in each second training image and images in the real frames.
And S1014, segmenting the image in the candidate frame in the map image to be processed to obtain the target object.
The embodiment further provides another implementation manner for segmenting a plurality of target objects included in the map image to be processed, which may specifically include:
and S1015, inputting the map image to be processed into the first neural network model, and obtaining the candidate frame of the target object contained in the map image to be processed determined by the first neural network model, the semantic feature information of the image in the candidate frame, and the two-dimensional position information of the candidate frame in the map image to be processed.
The first neural network model is obtained by training by using a plurality of second training images and the real frames of a plurality of target objects marked in each second training image, semantic feature information of images in the real frames and two-dimensional position information of the real frames in the second training images.
And S1016, segmenting the image in the candidate frame in the map image to be processed to obtain the target object.
For example, as shown in part (a) of fig. 2, the map image to be processed includes target objects such as computers, plants, and chairs, and the target objects such as computers, plants, and chairs in the map image to be processed are segmented, and the segmented computers, plants, and chairs are shown in part (b) of fig. 2.
It should be noted that fig. 2 is only an example of the map image to be processed and the segmentation, and is not meant to be a limitation of the map image to be processed and the segmentation.
In this embodiment, the first neural network model may be, but is not limited to: a YOLO neural network model or an SSD neural network model.
And step S102, determining image characteristic information and position information of the target object.
In this embodiment, the image feature information may include, but is not limited to: semantic feature information. The semantic feature information of the target object indicates the meaning of the target object.
Of course, the image characteristic information may also include, but is not limited to: any one or more of color feature information, texture feature information, and shape feature information.
Corresponding to the embodiment where the image feature information is semantic feature information and steps S1011-S1012, the image feature information of the determined target object may include, but is not limited to:
and S1021, inputting the target object into the neural network model to obtain semantic feature information of the target object determined by the neural network model.
The neural network model in this step is different from the first neural network model. The neural network model in this step is obtained by training in advance by using a plurality of training objects and semantic feature information of each training object.
Of course, the image characteristic information of the determination target object may also include, but is not limited to:
s1022, the semantic feature information of the image in the candidate frame obtained in step S1013 is acquired, and the acquired semantic feature information of the image in the candidate frame is determined as the image feature information of the target object.
The position information of the target object may be: two-dimensional position information of the target object in the map image to be processed or three-dimensional position information of the target object in the space.
The determining method of the position information of the target object in the map image to be processed may include:
s1023, acquiring the two-dimensional position information of the candidate frame in the map image to be processed obtained in step S1015, and determining the acquired two-dimensional position information as the position information of the target object in the map image to be processed.
And step S103, performing associated storage of image characteristic information and position information for each of the plurality of target objects.
The associated storage of the image feature information and the position information for each of the plurality of target objects may include, but is not limited to:
s1031, obtaining identification information of the map image to be processed;
s1032, performing associated storage of image characteristic information, position information and identification information of the map image to be processed on each of the plurality of target objects.
For example, if the map image to be processed is shown in part (a) of fig. 2, the identification information of the map image to be processed is Img ID1, the image feature information of the computer in part (b) of fig. 2 is feature1, the position information of the computer is location1, the image feature information of the plant in part (b) of fig. 2 is feature2, the position information of the plant is location2, the image feature information of the chair in part (b) of fig. 2 is feature3, the position information of the chair is location3, the computer is stored in association with feature1, location1 and Img ID1, the plants are stored in association with feature2, location2 and Img ID1, and the chair is stored in association with feature3, location3 and Img ID 1.
It is understood that the image feature information and/or the position information associated with each of the plurality of target objects and the association relationship of the image feature information and the position information of each of the plurality of target objects may be acquired from the map information.
In the embodiment, the map image to be processed is acquired, the target objects included in the map image to be processed are segmented, the image feature information and the position information of the target objects are determined, and the image feature information and the position information are stored in association with each of the target objects, so that data storage is realized by taking the target objects as a unit, and the data volume of the map image to be processed is reduced.
And when the image characteristic information is semantic characteristic information, the semantic characteristic information is relatively stable under different illumination environments, so that the semantic characteristic information can be used for coping with changes of the illumination environments, on the basis, the semantic characteristic information and the position information are stored in association with each of a plurality of target objects, so that under the condition that at least the semantic characteristic information of the target objects is used for a specific task (such as an image retrieval task), the influence caused by the changes of the illumination environments is reduced, and the accuracy of executing the specific task is ensured.
As another optional embodiment of the present application, referring to fig. 3, a schematic flowchart of an embodiment 2 of a map storage method provided by the present application is provided, and this embodiment mainly relates to a refinement scheme of the map storage method described in the foregoing embodiment 1, and the method may include, but is not limited to, the following steps:
step S201, obtaining a map image to be processed, and segmenting a plurality of target objects included in the map image to be processed.
The detailed process of step S201 can be referred to the related description of step S101 in embodiment 1, and is not described herein again.
And step S202, determining image characteristic information of the target object.
For details of this step, reference may be made to related description of step S102 in embodiment 1, and details are not described here.
And step S203, determining two-dimensional position information of the target object in the map image to be processed.
Step S204, a first target image collected before the map image to be processed is obtained, wherein the first target image at least comprises a target object.
The first target image acquired before the map image to be processed may be: a first target image acquired a moment before the acquisition moment of the map image to be processed; or the first target image is acquired n times before the acquisition time of the map image to be processed, wherein n is a number larger than 1.
And step S205, determining two-dimensional position information of the target object in the first target image.
This step may include, but is not limited to:
s2051, inputting the first target image into the first neural network model, and obtaining the candidate frame of the target object included in the first target image determined by the first neural network model, the semantic feature information of the image in the candidate frame, and the two-dimensional position information of the candidate frame in the first target image.
And S2052, segmenting the image in the candidate frame in the first target image to obtain a target object.
S2053, determine the two-dimensional position information of the candidate frame in the first target image as the two-dimensional position information of the target object in the first target image.
And step S206, determining the three-dimensional position information of the target object based on the two-dimensional position information of the target object in the map image to be processed and the two-dimensional position information of the target object in the first target image.
This step may include, but is not limited to:
s2061, determining the triangularization relation between the target object in the map image to be processed and the target object in the first target image based on the two-dimensional position information of the target object in the map image to be processed and the two-dimensional position information of the target object in the first target image.
S2062, determining and obtaining the depth information of the target object based on the triangularization relation.
S2063, obtaining the three-dimensional position information of the target object based on the two-dimensional position information of the target object in the map image to be processed and the depth information of the target object.
And step S207, determining the position information of the target object based on the three-dimensional position information and the image acquisition position information corresponding to the map image to be processed.
The image acquisition position information corresponding to the map image to be processed can be understood as: and acquiring three-dimensional position information of an image acquisition device of the map image to be processed in space.
Determining the position information of the target object based on the three-dimensional position information and the image acquisition position information corresponding to the map image to be processed, which can be understood as follows: and determining the three-dimensional position information of the target object in the space based on the three-dimensional position information and the image acquisition position information corresponding to the map image to be processed.
Steps S202 to S207 are a specific implementation of step S102 in embodiment 1.
And S208, performing associated storage of the image characteristic information and the position information for each of the plurality of target objects.
In the embodiment, data storage is realized by taking a map image to be processed as a unit through acquiring the map image to be processed, segmenting a plurality of target objects included in the map image to be processed, determining image characteristic information of the target objects, determining two-dimensional position information of the target objects in the map image to be processed, acquiring a first target image acquired before the map image to be processed by determining two-dimensional position information of the target objects in the map image to be processed, determining three-dimensional position information of the target objects based on the two-dimensional position information of the target objects in the map image to be processed and the two-dimensional position information of the target objects in the first target image, determining position information of the target objects based on the three-dimensional position information and the image acquisition position information corresponding to the map image to be processed, and performing associated storage of the image characteristic information and the position information for each of the plurality of target objects, the data amount stored in the map image to be processed is reduced.
As another alternative embodiment of the present application, referring to fig. 4, a flowchart of an embodiment 3 of a map storage method provided by the present application is shown, where this embodiment is mainly an extension of the map storage method described in the foregoing embodiment 2, and the method may include, but is not limited to, the following steps:
step S301, obtaining a map image to be processed, and segmenting a plurality of target objects included in the map image to be processed.
And step S302, determining image characteristic information of the target object.
And step S303, determining two-dimensional position information of the target object in the map image to be processed.
Step S304, acquiring a first target image collected before the map image to be processed, wherein the first target image at least comprises a target object.
Step S305, two-dimensional position information of the target object in the first target image is determined.
Step S306, determining three-dimensional position information of the target object based on the two-dimensional position information of the target object in the map image to be processed and the two-dimensional position information of the target object in the first target image.
The detailed processes of steps S301 to S306 can be referred to the related descriptions of steps S201 to S206 in embodiment 2, and are not described herein again.
Step S307, a plurality of second target images before the map image to be processed are obtained, wherein the second target images at least comprise a plurality of target objects.
In this embodiment, the manner of obtaining the plurality of second target images before the map image to be processed is not limited, for example, the plurality of target images may be randomly obtained from the target images before the map image to be processed, and the obtained plurality of target images are determined as the plurality of second target images; or acquiring a plurality of target images with continuous acquisition time from the target images before the map image to be processed, and determining the acquired plurality of target images as a plurality of second target images.
And step S308, adjusting the three-dimensional position information of the target object based on the plurality of second target images and the map image to be processed.
This step may include, but is not limited to:
s3081, projecting the target object to the second target image based on the three-dimensional position information of the target object, and determining a projection area corresponding to the target object in the second target image and two-dimensional position information of the projection area.
S3082, determining an error between the two-dimensional position information of the projection area and the two-dimensional position information of the target object in the second target image.
S3083, determining whether the sum of the errors converges.
If not, go to step S3084; if converged, step S3085 is performed.
And step S3084, adjusting the three-dimensional position information of the target object.
In the present application, the manner of adjusting the three-dimensional position information of the target object is not limited. For example, the three-dimensional position information of the target object may be specifically adjusted based on a Bundle Adjustment (Bundle Adjustment) algorithm.
Step 3085, finishing the adjustment.
S3082, determining a triangularization relation between the target object in the map image to be processed and the target object in the second target image based on the two-dimensional position information of the target object in the map image to be processed and the two-dimensional position information of the target object in the second target image.
And S3083, determining and obtaining the target depth information of the target object based on the triangularization relation determined in the step S3082.
S3084, based on the minimized reprojection error model and the target depth information of the target object, adjusting the depth information in the three-dimensional position information of the target object.
Step S309, determining the position information of the target object based on the adjusted three-dimensional position information and the image acquisition position information corresponding to the map image to be processed.
Step S310, performing associated storage of image characteristic information and position information for each of a plurality of target objects.
In the embodiment, by acquiring a map image to be processed, segmenting a plurality of target objects included in the map image to be processed, determining image feature information of the target objects, and by determining two-dimensional position information of the target objects in the map image to be processed, acquiring a first target image acquired before the map image to be processed, determining two-dimensional position information of the target objects in the first target image, determining three-dimensional position information of the target objects based on the two-dimensional position information of the target objects in the map image to be processed and the two-dimensional position information of the target objects in the first target image, and acquiring a plurality of second target images before the map image to be processed, adjusting the three-dimensional position information of the target objects based on the plurality of second target images and the map image to be processed, improving the precision of the three-dimensional position information of the target objects, based on the adjusted three-dimensional position information and the image acquisition position information corresponding to the map image to be processed, the method has the advantages that the position information of the target object is determined, the accuracy of the position information of the target object is improved, the image characteristic information and the position information are stored in a correlated mode for each of the target objects, data storage is achieved by taking the target object as a unit, the data size of the map image to be processed is reduced, and the accuracy of the stored position information is guaranteed.
Referring to fig. 5, a schematic flowchart of a pose determination method provided in embodiment 4 of the present application, where the method may be applied to an electronic device, and the present application does not limit a product type of the electronic device, as shown in fig. 5, the method may include, but is not limited to, the following steps:
step S401, a first image acquired by an image acquisition device is acquired, and a plurality of first objects included in the first image are segmented.
In this embodiment, the plurality of first objects included in the first image may be segmented, but not limited to, in a manner used for segmenting the plurality of target objects included in the map image to be processed as described in the foregoing embodiments.
Step S402, determining image characteristic information of the first object.
In this embodiment, the image characteristic information of the first object may be determined in a manner, but not limited to, that used to determine the image characteristic information of the target object as described in the foregoing embodiments.
The image characteristic information of the first object may include, but is not limited to: semantic feature information. The semantic feature information of the first object indicates the meaning of the first object.
Of course, the image characteristic information of the first object may also include, but is not limited to: any one or more of color feature information, texture feature information, and shape feature information of the first object.
In step S403, a target object associated with each of the plurality of first objects is determined from the map information based on the image feature information of the first object.
The map information is obtained by executing the map storage method as provided in any one of embodiments 1 to 3.
This step may include, but is not limited to:
s4031, matching a plurality of candidate target objects corresponding to each first object from the map information based on the image feature information of each first object, where the image feature information of each of the plurality of candidate target objects and the image feature information of the first object satisfy a first approximation condition.
The image feature information of each of the plurality of candidate target objects and the image feature information of the first object satisfy a first proximity condition, which may include, but is not limited to:
the degree of approximation between the image feature information of each of the plurality of candidate target objects and the image feature information of the first object is greater than a first set degree of approximation threshold.
S4032, determining a target object from the candidate target objects according to the similarity information between the image feature information of the candidate target object and the image feature information of the first object.
The similarity information between the image feature information of the candidate target object and the image feature information of the first object may include, but is not limited to: an approximation between the image feature information of the candidate target object and the image feature information of the first object. Accordingly, the present step may include, but is not limited to:
s40321 compares the degree of approximation between the image feature information of each of the plurality of candidate target objects and the image feature information of the first object, and determines a candidate target object having the highest degree of approximation among the plurality of candidate target objects as a target object.
Of course, the target object associated with each of the plurality of first objects is determined from the map information based on the image feature information of the first object, and may include but is not limited to:
s4033, matching a plurality of candidate target objects corresponding to each first object from the map information based on the image feature information of each first object, where the image feature information of each of the plurality of candidate target objects and the image feature information of the first object satisfy a first approximation condition.
S4034, acquiring identification information of an image to which the image feature information of the candidate target object belongs, and identification information of an image to which a candidate target object corresponding to each other first object in the plurality of first objects belongs, and determining whether the identification information of the image to which the image feature information of the candidate target object belongs is the same as identification information of the image to which the image feature information of the candidate target object belongs.
If yes, go to step S4035.
And S4035, determining the candidate target object as a target object.
The following examples are given to enhance understanding of steps S4033-S4035. For example, as shown in fig. 6, if the plurality of first objects are a computer a, a plant b and a chair c, the image feature information of the computer a is feature11, the image feature information of the plant b is feature22, and the image feature information of the chair c is feature33, a plurality of candidate target objects corresponding to the computer a are matched from the map information based on the feature11, and the candidate target objects are respectively a computer a1, a computer a2 and a computer A3; matching a plurality of candidate target objects corresponding to the plant B from the map information based on feature22, wherein the candidate target objects are respectively plant B1 and plant B2; matching a plurality of candidate target objects corresponding to the chair C from the map information based on feature33, which are C2 and C3, respectively; the image feature information of the computer a1 and the image feature information of the plant B1 belong to the same identification information, Img ID1, the image feature information of the computer a2, the image feature information of the plant B2 and the image feature information of the chair C2 belong to the same identification information, Img ID2, and the image feature information of the computer A3 and the image feature information of the chair C3 belong to the same identification information, Img ID 3.
If the identification information of the images to which the image feature information of the plant B1 and the plant B2 corresponding to the plant B belongs and the identification information of the images to which the image feature information of the chair C2 and the chair C3 corresponding to the chair C belong are determined to have the same identification information (namely, Img ID2) as the identification information of the image to which the image feature information of the computer A2 belongs, the computer A2 is determined to be the target object; or, it is determined that the identification information of the image to which the image feature information of the computer a1, the computer a2, and the computer A3 corresponding to the computer a belongs and the identification information of the image to which the image feature information of the chair C2 and the chair C3 corresponding to the chair C belong have the same identification information (i.e., Img ID2) as the identification information of the image to which the image feature information of the plant B2 belongs, and then the plant B2 is the target object; or, it is determined that the same identification information (i.e., Img ID2) as that of the image to which the image feature information of the chair C2 belongs exists in the identification information of the image to which the image feature information of the computer a1, the computer a2, and the computer A3 corresponding to the computer a and the identification information of the image to which the image feature information of the plant B1 and the plant B2 corresponding to the plant B, and then the chair C2 is the target object.
It can be understood that, if the identification information of the image to which the candidate target object belongs corresponding to each of the other first objects exists in the identification information of the image to which the image feature information of the candidate target object belongs, the image to which the image feature information of the candidate target object belongs also includes one of the candidate target objects corresponding to each of the other first objects, so as to ensure the accuracy of the target object.
It is understood that the position information of the target object is pre-stored in the map information, and after the target object is determined, the position information of the target object may be acquired from the map information. For example, as shown in fig. 6, the position information of the computer a2 is acquired from the map information; and/or acquiring the position information of the plant B2 from the map information; and/or obtaining the position information of the chair C2 from the map information.
And S404, determining the pose of the image acquisition device based on the position information of the target object.
In this embodiment, the positioning of the image capturing device is realized by acquiring a first image captured by the image capturing device, segmenting a plurality of first objects included in the first image, determining image feature information of the first objects, determining a target object associated with each of the plurality of first objects from map information having a small amount of stored data based on the image feature information of the first objects, and determining the pose of the image capturing device based on position information of the target object.
As another alternative embodiment of the present application, referring to fig. 7, a schematic flowchart of an embodiment 5 of a pose determination method provided by the present application is provided, where this embodiment mainly is a refinement of the pose determination method described in the above embodiment 4, and the method may include, but is not limited to, the following steps:
step S501, a first image acquired by an image acquisition device is acquired, and a plurality of first objects included in the first image are segmented.
And step S502, determining image characteristic information of the first object.
In step S503, a target object associated with each of the plurality of first objects is determined from the map information based on the image feature information of the first object.
The detailed processes of steps S501-S503 can refer to the related descriptions of steps S401-S403 in embodiment 4, and are not described herein again.
Step S504, two-dimensional position information of the first object in the first image is determined.
This step may include, but is not limited to:
s5041, inputting the first image into the second neural network model, and obtaining a candidate frame of the first object included in the first image and two-dimensional position information of the candidate frame in the first image, which are determined by the second neural network model.
The second neural network model is obtained by utilizing a plurality of third training images and real frames of a plurality of target objects marked in each third training image and two-dimensional position information of the real frames in the third training images.
S5042, setting the two-dimensional position information of the candidate frame in the first image as the two-dimensional position information of the first object in the first image.
And S505, determining the pose of the image acquisition device based on the position information of the target object and the two-dimensional position information of the first object in the first image.
The position information of the target object may include, but is not limited to: two-dimensional position information of the target object in the image to which the target object belongs; or, three-dimensional position information of the target object in space.
The embodiment where the position information of the corresponding target object is two-dimensional position information of the target object in the image to which the corresponding target object belongs may include, but is not limited to:
s5051, determining the pose of the image acquisition device based on the two-dimensional position information of the target object in the image to which the target object belongs and the two-dimensional position information of the first object in the first image.
The embodiment that the position information of the corresponding target object is three-dimensional position information of the target object in the space may include, but is not limited to:
s5052, determining the pose of the image acquisition device based on the three-dimensional position information of the target object in the space and the two-dimensional position information of the first object in the first image.
This step may include, but is not limited to:
s5053, according to the position information of the target object and the two-dimensional position information of the first object in the first image, geometric constraint information of the image acquisition device relative to the target object is established.
The step S5053 may include:
and establishing two-dimensional geometric constraint information of the image acquisition device relative to the target object according to the two-dimensional position information of the target object in the image to which the target object belongs and the two-dimensional position information of the first object in the first image.
The position information of the corresponding target object is three-dimensional position information of the target object in the space, and step S5053 may include:
and establishing three-dimensional geometric constraint information of the image acquisition device relative to the target object according to the three-dimensional position information of the target object in the space and the two-dimensional position information of the first object in the first image.
S5054, determining the pose of the image acquisition device based on the geometric constraint information.
Steps S504 to S505 are a specific implementation manner of step S104 in embodiment 4.
In the embodiment, the positioning of the image capturing device is realized by acquiring a first image captured by the image capturing device, segmenting a plurality of first objects included in the first image, determining image feature information of the first objects, determining a target object associated with each of the plurality of first objects from stored map information with a small data amount based on the image feature information of the first objects, determining two-dimensional position information of the first objects in the first image, and determining the pose of the image capturing device based on the position information of the target object and the two-dimensional position information of the first objects in the first image.
Furthermore, according to the position information of the target object and the two-dimensional position information of the first object in the first image, geometric constraint information of the image acquisition device relative to the target object is established, the pose of the image acquisition device is determined based on the geometric constraint information, the accuracy of the pose of the image acquisition device can be ensured, and the positioning accuracy of the image acquisition device is ensured.
As another alternative embodiment of the present application, referring to fig. 8, a schematic flowchart of an embodiment 6 of a pose determination method provided by the present application is provided, where this embodiment mainly is a refinement of the pose determination method described in the above embodiment 4, and the method may include, but is not limited to, the following steps:
step S601, a first image acquired by an image acquisition device is acquired, and a plurality of first objects included in the first image are segmented.
Step S602, image characteristic information of the first object is determined.
Step S603 determines a target object associated with each of the plurality of first objects from the map information based on the image feature information of the first object.
The detailed processes of steps S601-S603 can refer to the related descriptions of steps S401-S403 in embodiment 4, and are not described herein again.
And step S604, obtaining a plurality of groups of target object combinations according to the target objects.
This step may include, but is not limited to:
s6041 matches a plurality of target objects to be used corresponding to the target object from the map information according to the image feature information of the target object, where the image feature information of the target object to be used and the image feature information of the target object satisfy a second similarity condition.
The image feature information of the target object to be used and the image feature information of the target object satisfy the second proximity condition, which may include, but is not limited to:
the degree of approximation between the image feature information of the target object to be used and the image feature information of the target object is greater than a second set degree of approximation threshold.
S6042, dividing a plurality of target objects to be used into a plurality of groups to obtain a plurality of groups of target object combinations, wherein each group of target object combination at least comprises at least two target objects to be used, and each group of target object combination has difference.
And step S605, determining a plurality of candidate poses of the image acquisition device according to the combination of the plurality of groups of target objects.
This step may include, but is not limited to:
and respectively determining a candidate pose of the image acquisition device according to the position information of at least two target objects to be used in each group of target object combination.
And step S606, determining the pose of the image acquisition device according to the difference data among the candidate poses.
Based on the determination of the multiple candidate poses of the image capturing device in step S605, the difference data between the multiple candidate poses is determined, and the pose of the image capturing device is determined according to the difference data between the multiple candidate poses.
Determining a pose of the image capture device from difference data between the plurality of candidate poses may include:
and deleting the abnormal candidate pose from the candidate poses according to the difference data among the candidate poses, and determining the pose of the image acquisition device based on the candidate pose after the abnormal candidate pose is deleted.
The difference between the outlier candidate pose and other candidate poses of the plurality of candidate poses is greater than a set threshold.
Determining the pose of the image acquisition device based on the candidate pose after the abnormal candidate pose is deleted, which may include but is not limited to:
and taking the average value or the median of the candidate poses after the abnormal candidate poses are deleted as the poses of the image acquisition device.
In this embodiment, a first image acquired by an image acquisition device is acquired, a plurality of first objects included in the first image are segmented, image feature information of the first objects is determined, a target object associated with each of the plurality of first objects is determined from stored map information with a small data size based on the image feature information of the first objects, a plurality of sets of target object combinations are obtained according to the target objects, a plurality of candidate poses of the image acquisition device are determined according to the plurality of sets of target object combinations, a pose of the image acquisition device is determined according to difference data between the plurality of candidate poses, accuracy of the pose of the image acquisition device is improved, and accurate positioning of the image acquisition device is achieved.
Next, a description will be given of a map storage device provided in the present application, and the map storage device described below and the map storage method described above may be referred to in correspondence with each other.
Referring to fig. 9, the map storage device includes: a first segmentation module 100, a first determination module 200 and a storage module 300.
The first segmentation module 100 is configured to obtain a map image to be processed, and segment a plurality of target objects included in the map image to be processed.
A first determining module 200, configured to determine image feature information and position information of a target object.
The storage module 300 is configured to perform associated storage of image feature information and position information for each of a plurality of target objects.
The process of determining the position information of the target object by the first determining module 200 may specifically include:
determining two-dimensional position information of a target object in a map image to be processed;
acquiring a first target image acquired before a map image to be processed, wherein the first target image at least comprises a target object;
determining two-dimensional position information of a target object in a first target image;
determining three-dimensional position information of a target object based on two-dimensional position information of the target object in a map image to be processed and two-dimensional position information of the target object in a first target image;
and determining the position information of the target object based on the three-dimensional position information and the image acquisition position information corresponding to the map image to be processed.
The process of the first determination module 200 determining the position information of the target object may further include:
acquiring a plurality of second target images before a map image to be processed, wherein the second target images at least comprise a plurality of target objects;
and adjusting the three-dimensional position information of the target object based on the plurality of second target images and the map image to be processed.
In this embodiment, the image feature information includes semantic feature information, and the process of determining the image feature information of the target object by the first determining module 200 may specifically include:
and inputting the target object into the neural network model to obtain the semantic feature information of the target object determined by the neural network model.
Next, a description will be given of a posture determining apparatus provided by the present application, and the posture determining apparatus described below and the posture determining method described above may be referred to in correspondence with each other.
Referring to fig. 10, the pose determination apparatus includes: a second segmentation module 400, a second determination module 500, a third determination module 600, and a fourth determination module 700.
The second segmentation module 400 is configured to acquire the first image acquired by the image acquisition device, and segment a plurality of first objects included in the first image.
A second determining module 500 for determining image characteristic information of the first object.
A third determining module 600, configured to determine a target object associated with each of the plurality of first objects from map information based on image feature information of the first objects, where the map information is obtained by performing the map storage method as described in any one of embodiments 1 to 3.
A fourth determining module 700, configured to determine a pose of the image capturing apparatus based on position information of the target object, where the position information of the target object is pre-stored in the map information.
The third determining module 600 may specifically be configured to:
matching a plurality of candidate target objects corresponding to each first object from the map information based on the image characteristic information of each first object, wherein the image characteristic information of each candidate target object and the image characteristic information of the first object meet a first approximation condition;
and determining the target object from the candidate target objects according to the similarity information of the image characteristic information of the candidate target objects and the image characteristic information of the first object.
The fourth determining module 700 may specifically be configured to:
determining two-dimensional position information of a first object in a first image;
and determining the pose of the image acquisition device based on the position information of the target object and the two-dimensional position information of the first object in the first image.
The fourth determining module 700 may specifically determine the pose of the image capturing apparatus based on the position information of the target object and the two-dimensional position information of the first object in the first image, and may specifically include:
according to the position information of the target object and the two-dimensional position information of the first object in the first image, establishing geometric constraint information of the image acquisition device relative to the target object;
and determining the pose of the image acquisition device based on the geometric constraint information.
In this embodiment, the fourth determining module 700 may be specifically configured to:
obtaining a plurality of groups of target object combinations according to the target objects;
determining a plurality of candidate poses of the image acquisition device according to the combination of the plurality of groups of target objects;
and determining the pose of the image acquisition device according to the difference data among the candidate poses.
Corresponding to the embodiment of the map storage method or the pose determination method provided by the application, the application also provides an embodiment of electronic equipment applying the map storage method or the pose determination method.
The electronic device may include the following structure:
a memory and a processor.
A memory for storing at least one set of instructions;
and the processor is used for calling and executing the instruction set in the memory, and executing the map storage method introduced in any one of the embodiment 1-3 or the pose determination method introduced in any one of the embodiment 4-6 by executing the instruction set.
Corresponding to the embodiment of the map storage method or the pose determination method provided by the application, the application also provides an embodiment of a storage medium.
In this embodiment, a storage medium stores a computer program that implements the map storage method described in any one of method embodiments 1 to 3 or the pose determination method described in any one of embodiments 4 to 6, and the computer program is executed by a processor to implement the map storage method described in any one of method embodiments 1 to 3 or the pose determination method described in any one of embodiments 4 to 6.
It should be noted that the focus of each embodiment is different from that of other embodiments, and the same and similar parts between the embodiments may be referred to each other. For the device-like embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in the process, method, article, or apparatus that comprises the element.
For convenience of description, the above devices are described as being divided into various modules by functions, and are described separately. Of course, the functionality of the various modules may be implemented in the same one or more software and/or hardware implementations as the present application.
From the above description of the embodiments, it is clear to those skilled in the art that the present application can be implemented by software plus necessary general hardware platform. Based on such understanding, the technical solutions of the present application or portions thereof that contribute to the prior art may be embodied in the form of a software product, which may be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, or the like, and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute the method according to the embodiments or some portions of the embodiments of the present application.
The map storage method, the pose determination method and the storage medium provided by the present application are described in detail above, and specific examples are applied in the present application to explain the principle and the implementation of the present application, and the description of the above embodiments is only used to help understand the method and the core idea of the present application; meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.
Claims (10)
1. A map storage method, comprising:
acquiring a map image to be processed, and segmenting a plurality of target objects included in the map image to be processed;
determining image characteristic information and position information of the target object;
and performing associated storage of the image characteristic information and the position information for each of the plurality of target objects.
2. The method of claim 1, the determining location information for the target object, comprising:
determining two-dimensional position information of the target object in a map image to be processed;
acquiring a first target image acquired before the map image to be processed, wherein the first target image at least comprises the target object;
determining two-dimensional position information of the target object in the first target image;
determining three-dimensional position information of the target object based on the two-dimensional position information of the target object in the map image to be processed and the two-dimensional position information of the target object in the first target image;
and determining the position information of the target object based on the three-dimensional position information and the image acquisition position information corresponding to the map image to be processed.
3. The method of claim 2, further comprising:
acquiring a plurality of second target images before the map image to be processed, wherein the second target images at least comprise the plurality of target objects;
and adjusting the three-dimensional position information of the target object based on the second target images and the map image to be processed.
4. The method of claim 1, the image feature information comprising semantic feature information, the determining image feature information of the target object comprising:
and inputting the target object into a neural network model to obtain semantic feature information of the target object determined by the neural network model.
5. A pose determination method, comprising:
acquiring a first image acquired by an image acquisition device, and segmenting a plurality of first objects included in the first image;
determining image characteristic information of the first object;
determining a target object associated with each of the plurality of first objects from map information obtained by performing the map storage method according to any one of claims 1 to 4, based on image feature information of the first object;
and determining the pose of the image acquisition device based on the position information of the target object, wherein the position information of the target object is prestored in the map information.
6. The method of claim 5, the determining a target object associated with each of the plurality of first objects from map information based on image feature information of the first object, comprising:
matching a plurality of candidate target objects corresponding to each first object from the map information based on the image feature information of each first object, wherein the image feature information of each candidate target object and the image feature information of the first object meet a first approximation condition;
and determining the target object from the candidate target objects according to the similarity information of the image characteristic information of the candidate target objects and the image characteristic information of the first object.
7. The method of claim 5, the determining the pose of the image acquisition device based on the position information of the target object, comprising:
determining two-dimensional position information of the first object in the first image;
determining a pose of the image acquisition device based on the position information of the target object and the two-dimensional position information of the first object in the first image.
8. The method of claim 7, the determining the pose of the image acquisition device based on the position information of the target object and the two-dimensional position information of the first object in the first image, comprising:
establishing geometric constraint information of the image acquisition device relative to the target object according to the position information of the target object and the two-dimensional position information of the first object in the first image;
determining a pose of the image capture device based on the geometric constraint information.
9. The method of claim 5, the determining the pose of the image acquisition device based on the position information of the target object, comprising:
obtaining a plurality of groups of target object combinations according to the target objects;
determining a plurality of candidate poses of the image acquisition device according to the plurality of groups of target object combinations;
determining a pose of the image capture device from difference data between the plurality of candidate poses.
10. A storage medium storing a computer program implementing the map storage method according to any one of claims 1 to 4 or the pose determination method according to any one of claims 5 to 9, the computer program being executed by a processor to implement the map storage method according to any one of claims 1 to 4 or the pose determination method according to any one of claims 5 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210581219.XA CN114996492A (en) | 2022-05-26 | 2022-05-26 | Map storage method, pose determination method and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210581219.XA CN114996492A (en) | 2022-05-26 | 2022-05-26 | Map storage method, pose determination method and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114996492A true CN114996492A (en) | 2022-09-02 |
Family
ID=83028881
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210581219.XA Pending CN114996492A (en) | 2022-05-26 | 2022-05-26 | Map storage method, pose determination method and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114996492A (en) |
-
2022
- 2022-05-26 CN CN202210581219.XA patent/CN114996492A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111598993B (en) | Three-dimensional data reconstruction method and device based on multi-view imaging technology | |
| JP4545641B2 (en) | Similar image retrieval method, similar image retrieval system, similar image retrieval program, and recording medium | |
| JP5554984B2 (en) | Pattern recognition method and pattern recognition apparatus | |
| CN109145717B (en) | Face recognition method for online learning | |
| CN110136156B (en) | Polygonal area detection method and device | |
| CN111160288A (en) | Gesture key point detection method and device, computer equipment and storage medium | |
| CN109842811B (en) | Method and device for implanting push information into video and electronic equipment | |
| CN111008935B (en) | Face image enhancement method, device, system and storage medium | |
| CN110765882B (en) | Video tag determination method, device, server and storage medium | |
| CN110489574B (en) | Multimedia information recommendation method and device and related equipment | |
| CN105447190A (en) | Image retrieval method, device and server based on convolutional neural network | |
| CN113179421B (en) | Video cover selection method and device, computer equipment and storage medium | |
| CN109859314A (en) | Three-dimensional rebuilding method, device, electronic equipment and storage medium | |
| CN110969100A (en) | Human body key point identification method and device and electronic equipment | |
| CN112907569A (en) | Head image area segmentation method and device, electronic equipment and storage medium | |
| CN113963303A (en) | Image processing method, video recognition method, device, equipment and storage medium | |
| CN111027551B (en) | Image processing method, apparatus and medium | |
| CN110929731B (en) | Medical image processing method and device based on pathfinder intelligent search algorithm | |
| JP2016118971A (en) | Image feature amount registration device, method, and program | |
| JP2010257267A (en) | Device, method and program for detecting object area | |
| CN111639517A (en) | Face image screening method and device | |
| CN116363641A (en) | Image processing method and device and electronic equipment | |
| CN114694185B (en) | Cross-modal target re-identification method, device, equipment and medium | |
| CN110660091A (en) | Image registration processing method and device and photographing correction operation system | |
| CN114996492A (en) | Map storage method, pose determination method and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |