CN115574802A - Drawing and positioning method based on flat character feature center - Google Patents

Drawing and positioning method based on flat character feature center Download PDF

Info

Publication number
CN115574802A
CN115574802A CN202211172926.XA CN202211172926A CN115574802A CN 115574802 A CN115574802 A CN 115574802A CN 202211172926 A CN202211172926 A CN 202211172926A CN 115574802 A CN115574802 A CN 115574802A
Authority
CN
China
Prior art keywords
character
text
feature center
camera
positioning
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211172926.XA
Other languages
Chinese (zh)
Inventor
张成星
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Individual
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Priority to CN202211172926.XA priority Critical patent/CN115574802A/en
Publication of CN115574802A publication Critical patent/CN115574802A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/38—Electronic maps specially adapted for navigation; Updating thereof
    • G01C21/3804—Creation or updating of map data
    • G01C21/3807—Creation or updating of map data characterised by the type of data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/14—Image acquisition
    • G06V30/146—Aligning or centring of the image pick-up or image-field
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/14—Image acquisition
    • G06V30/148—Segmentation of character regions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30244—Camera pose

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Automation & Control Theory (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a drawing and positioning method based on a flat character feature center. Acquiring a video stream image, calculating a camera motion pose, calculating a plane parameter, extracting character information from the image, acquiring all effective pixels in the character information by utilizing a segmentation technology, determining a character feature center through all the effective pixels and the plane parameter, and acquiring a character map with character feature center point information; searching the same text information based on the established text map, acquiring the feature center and the plane normal vector of the text, and comparing the currently established text feature center and the plane normal vector thereof with the text feature center and the plane normal vector thereof established on the map to obtain the pose of the current camera in the text map so as to complete positioning. The invention can be used in both indoor and outdoor applications.

Description

Drawing and positioning method based on planar character feature center
The technical field is as follows:
the invention belongs to the technical field of visual mapping and positioning, and particularly relates to a mapping and positioning method based on a flat character feature center.
Background art:
at present, various schemes are available for indoor and outdoor positioning. The sensor can be divided into 2D Lidar, 3D Lidar, vision, WIFI, bluetooth, GPS and the like according to different sensors, wherein 2D laser, WIFI and Bluetooth are used indoors, 3D laser and GPS are used outdoors, and vision can be used indoors and outdoors. These solutions have certain limitations, which result in that they are not satisfactory in practical use.
WIFI and bluetooth are mainly used indoors, and both methods require a large amount of devices to be installed indoors in advance, so that the application range is limited.
The GPS can achieve high positioning accuracy outdoors and has no accumulated drift. But the signal thereof can be weakened in the scenes of tunnels, urban high buildings and the like or the positioning precision is reduced or even fails.
The 3D Lidar is mainly applied outdoors, the 2D Lidar is mainly applied indoors, the accuracy of the established map is high, but global positioning is difficult to realize, and the 2D Lidar is degraded in scenes such as tunnels and long corridors.
The visual application scenes are very rich, and the work can be carried out indoors and outdoors. At present, there are various visual positioning methods, such as a descriptor matching-based method and a two-dimensional code matching-based method. The method based on descriptor matching is illuminated, the change of the visual angle is greatly influenced, the data volume of the created map is very large, and the searching cost is high. The two-dimension code matching mode requires that two-dimension code features are added in advance in a scene, and the method is inconvenient to use in scenes such as a market.
It can be seen that most of the above techniques cannot be used indoors and outdoors at the same time, and some methods need to add devices or manual identifiers to implement positioning and navigation, which not only increases the cost but also limits the application scenarios, and some methods are very sensitive to environmental changes. How to develop a method which does not need to add artificial identification, has robust performance and wide application scenes is a problem to be solved.
Characters are common identification symbols in human society, the application is extremely common, and most of characters are written on a plane. The current OCR can rapidly and robustly identify the character information under different illumination and different visual angles, and provides effective technical support for the invention. And the portable equipment with cameras, such as mobile phones, flat panels and the like, provides wide application space for the invention.
The information disclosed in this background section is only for enhancement of understanding of the general background of the invention and should not be taken as an acknowledgement or any form of suggestion that this information forms the prior art already known to a person skilled in the art.
The invention content is as follows:
the invention aims to provide a drawing and positioning method based on a flat character feature center, wherein the flat character feature center is a brand-new visual SLAM feature, so that the defects in the prior art are overcome.
In order to achieve the purpose, the invention provides a drawing and positioning method based on a flat character feature center, which comprises the following steps: (1) drawing construction: acquiring a video stream image, calculating a camera motion pose, calculating plane parameters, extracting character information from the image, acquiring all effective pixels in the character information by using an image segmentation technology, acquiring plane parameters of characters according to all the effective pixels, determining feature centers of the characters according to all the effective pixels and the plane parameters, and acquiring a character map with character feature central point information;
(2) Positioning: searching the same character information based on the character map established in the step (1), acquiring the feature center and the plane normal vector of the character, and comparing the currently established character feature center and the plane normal vector thereof with the character feature center and the plane normal vector thereof established by the map to obtain the pose of the current camera in the character map so as to complete positioning.
Preferably, in the technical scheme, the text information comprises the position, range, character and color information of the text and the single character, and the range and the color are helpful for eliminating mismatching.
Preferably, in the technical solution, the image segmentation technique includes a pixel binarization technique and a deep learning technique.
Preferably, in the technical solution, the coordinates of the pixel points of the effective pixels are m, m = (u, v) T M is a homogeneous vector of
Figure BDA0003863003160000031
Each valid pixel point corresponds to a plane point P in the 3D world,
Figure BDA0003863003160000032
wherein
Figure BDA0003863003160000033
h represents the depth of P in the camera.
Preferably, in the technical scheme, the plane parameter is theta,
Figure BDA0003863003160000034
the plane equation is
Figure BDA0003863003160000035
Where n is a normal vector to the plane, then
Figure BDA0003863003160000036
By using
Figure BDA0003863003160000037
Indicating an inverse depth, then
Figure BDA0003863003160000038
Preferably, in the technical scheme, the plane parameter is obtained by directly obtaining a plane through equipment or fitting a plane by establishing a 3D point in a text area through SLAM.
Preferably, in the technical scheme, the characteristic center of the character is P center ，P center ＝∑P i In which P is i Summing the 3D point coordinates corresponding to all the effective pixels to obtain the character feature center for the 3D point coordinate corresponding to the ith effective pixel,
Figure BDA0003863003160000039
by P i Determines the 3D boundaries of the text.
Preferably, in the technical scheme, P center Is a camera coordinate system point and a camera pose is
Figure BDA00038630031600000310
(where w is the world coordinate system defined by SLAM, c is the camera coordinate system,
Figure BDA00038630031600000311
representing from camera coordinates to world coordinates), then converting the point under the camera coordinates to the world coordinate system may be represented as
Figure BDA00038630031600000312
Are points in world coordinate system.
Preferably, in the technical scheme, the SLAM is subjected to loop detection by using character features, and an accurate character feature central point and a character map with information of the feature central point of each text character are obtained through optimization.
Preferably, in the technical scheme, the camera pose is calculated through the SLAM algorithm in the step (2), and after the text in the map is matched, the pose is corrected and optimized, so that the position is accurately positioned.
Compared with the prior art, the invention has the following beneficial effects:
the most common character information of human society is used and matched with a visual positioning technology to realize a positioning mode which is low in cost and easy to expand, and the method is very suitable for positioning and navigation in public places. Compared with a 3D point cloud map, the capacity of the character map is greatly reduced, local storage is facilitated for a user, and communication with a back end in positioning can be omitted. Compared with other characteristics, the character characteristics have strong robustness. The technology is not limited to indoor or outdoor, can be used in places with characters, and is very suitable for places such as markets, parking lots, stations and the like.
Description of the drawings:
FIG. 1 is an illustration of a drawing of a text map created by the method for creating and positioning based on a flat text feature center according to the present invention;
FIG. 2 is an exemplary diagram of extracting all effective pixels by the method for drawing and positioning based on the flat text feature center.
The specific implementation mode is as follows:
the following detailed description of specific embodiments of the invention is provided, but it should be understood that the scope of the invention is not limited to the specific embodiments.
Throughout the specification and claims, unless explicitly stated otherwise, the word "comprise", or variations such as "comprises" or "comprising", will be understood to imply the inclusion of a stated element or component but not the exclusion of any other element or component.
A drawing and positioning method based on a flat character feature center comprises the following steps: (1) Acquiring video stream and plane parameters of a parking lot through an iPhone mobile phone with Lidar, and calculating the motion pose of a camera by using an SLAM technology;
(2) Using an OCR technology to obtain character information of parking space labels of a parking lot, wherein the character information comprises texts, positions, ranges, characters and colors of single characters;
(3) All effective pixels of a single character in the text are obtained by using a pixel binarization technology, as shown in fig. 2, all effective pixels of a parking space number 5330 are obtained, the effective pixels are white, and the coordinates of the pixel points of the effective pixels are m, m = (u, v) T M is a homogeneous vector of
Figure BDA0003863003160000051
(4) Each valid pixel point corresponds to a plane point P in the 3D world,
Figure BDA0003863003160000052
wherein
Figure BDA0003863003160000053
h represents the depth of P in the camera;
(5) Estimating a parameter theta of a plane where the characters are located by using a depth camera or a SLAM algorithm,
Figure BDA0003863003160000054
the plane equation is
Figure BDA0003863003160000055
Where n is a normal vector to the plane, then
Figure BDA0003863003160000056
By using
Figure BDA0003863003160000057
Indicating an inverse depth, then
Figure BDA0003863003160000058
(6) Calculating to obtain the characteristic center P of the character center ，P center ＝∑P i In which P is i Summing the 3D point coordinates corresponding to all the effective pixels to obtain the characteristic center of the character for the 3D point coordinate corresponding to the ith effective pixel,
Figure BDA0003863003160000059
by P i Determines the 3D boundaries of the text; p center Is a point of a camera coordinate system, n is a plane normal vector of the point, and the pose of the camera is
Figure BDA00038630031600000510
(where w is the world coordinate system defined by SLAM, c is the camera coordinate system,
Figure BDA00038630031600000511
representing from camera coordinates to world coordinates), then converting the point under the camera coordinates to the world coordinate system may be represented as
Figure BDA00038630031600000512
Figure BDA00038630031600000513
Points under the world coordinate system; the normal plane vector n of the point is converted into the normal plane vector under the world coordinate system
Figure BDA00038630031600000514
(7) Obtaining the geometric position of the character by multi-frame observation and combined with SLAM optimization on the basis of the step (6), performing loop detection and re-optimization on the SLAM by using character features to obtain an accurate character feature central point and a character map with the feature central point information of each text character, and obtaining the character map containing the parking lot parking space label information as shown in figure 1;
(8) Searching the same text information based on the text map established in the step (7), and if the same text in the text map is successfully searched, acquiring the central point of each character map in the coordinate system
Figure BDA00038630031600000515
Normal vector of plane
Figure BDA0003863003160000061
Wherein
Figure BDA0003863003160000062
Representing a map coordinate system;
(9) Obtaining the lower central point of each character world coordinate system in the step (6)
Figure BDA0003863003160000063
Normal vector n of plane w To connect these
Figure BDA0003863003160000064
n w With the product obtained in step (8)
Figure BDA0003863003160000065
One-to-one correspondence, wherein the 3D points correspond to:
Figure BDA0003863003160000066
Figure BDA0003863003160000067
in order to rotate the matrix of the matrix,
Figure BDA0003863003160000068
is a translation vector; the correspondence of the plane normal vectors is:
Figure BDA0003863003160000069
increasing direction constraint and direction weight, calculating by referring ICP algorithm
Figure BDA00038630031600000610
Then calculate out
Figure BDA00038630031600000611
Obtaining the pose of the current camera in a map coordinate system
Figure BDA00038630031600000612
Completing positioning;
(10) And (5) repeating the steps (1) - (9), calculating the pose of the camera through an SLAM algorithm, and performing pose correction optimization after matching the text in the map to achieve accurate positioning.
The foregoing description of specific exemplary embodiments of the invention has been presented for the purposes of illustration and description. It is not intended to limit the invention to the precise form disclosed, and obviously many modifications and variations are possible in light of the above teaching. The exemplary embodiments were chosen and described in order to explain certain principles of the invention and its practical application to enable one skilled in the art to make and use various exemplary embodiments of the invention and various alternatives and modifications as are suited to the particular use contemplated. It is intended that the scope of the invention be defined by the claims and their equivalents.

Claims (7)

1. A drawing and positioning method based on a flat character feature center is characterized in that: (1) establishing a diagram: acquiring a video stream, calculating a camera motion pose, calculating a plane parameter, extracting character information from an image, acquiring all effective pixels in the character information by using an image segmentation technology, determining a character feature center through all the effective pixels and the plane parameter, and acquiring a character map with character feature center point information;
(2) Positioning: searching the same text information based on the text map established in the step (1), acquiring the feature center and the plane normal vector of the text, and comparing the currently established text feature center and the plane normal vector thereof with the text feature center and the plane normal vector thereof established on the map to obtain the pose of the current camera in the text map so as to complete positioning.
2. The method for drawing and positioning based on the flat character feature center according to claim 1, wherein: the character information includes text, position of single character, range, character and color information.
3. The method for drawing and positioning based on the planar character feature center according to claim 2, wherein: the pixel point coordinate of the effective pixel is m, m = (u, v) T M is a homogeneous vector of
Figure FDA0003863003150000011
Figure FDA0003863003150000012
Each valid pixel point corresponds to a planar point P in the 3D world,
Figure FDA0003863003150000013
wherein
Figure FDA0003863003150000014
h represents the depth of P in the camera.
4. The method for drawing and positioning based on the flat character feature center according to claim 3, wherein: the center of the character is P center ，P center ＝∑P i In which P is i For the 3D point coordinate corresponding to the ith effective pixel, summing the 3D point coordinates corresponding to all the effective pixels to obtain the character feature center
Figure FDA0003863003150000015
The inverse depth is represented by the inverse of the depth,
Figure FDA0003863003150000016
where θ is a plane parameter, passing through P i Determines the 3D boundaries of the text.
5. The planar character feature center-based computer program product of claim 4The mapping and positioning method is characterized in that: p is center Is a coordinate system point of the camera, and the pose of the camera is
Figure FDA0003863003150000017
Figure FDA0003863003150000025
Where w is the world coordinate system defined by SLAM, c is the camera coordinate system,
Figure FDA0003863003150000021
representing from camera coordinates to world coordinates, the point in camera coordinates is converted to the world coordinate system as
Figure FDA0003863003150000022
Figure FDA0003863003150000023
Figure FDA0003863003150000024
Are points in world coordinate system.
6. The method for drawing and positioning based on the planar character feature center according to claim 5, wherein: and performing loop detection on the SLAM by using the character features, and optimizing to obtain an accurate character feature central point.
7. The method for drawing and positioning based on the flat character feature center according to claim 1, wherein: and (3) calculating the pose of the camera through an SLAM algorithm in the step (2), and performing pose correction optimization and accurate positioning after matching the text in the map.
CN202211172926.XA 2022-09-26 2022-09-26 Drawing and positioning method based on flat character feature center Pending CN115574802A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211172926.XA CN115574802A (en) 2022-09-26 2022-09-26 Drawing and positioning method based on flat character feature center

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211172926.XA CN115574802A (en) 2022-09-26 2022-09-26 Drawing and positioning method based on flat character feature center

Publications (1)

Publication Number Publication Date
CN115574802A true CN115574802A (en) 2023-01-06

Family

ID=84583675

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211172926.XA Pending CN115574802A (en) 2022-09-26 2022-09-26 Drawing and positioning method based on flat character feature center

Country Status (1)

Country Link
CN (1) CN115574802A (en)

Similar Documents

Publication Publication Date Title
US10664708B2 (en) Image location through large object detection
CN109520500B (en) Accurate positioning and street view library acquisition method based on terminal shooting image matching
JP2019109245A (en) Visual reinforcement navigation
CN103632626A (en) Intelligent tour guide realizing method and intelligent tour guide device based on mobile network and mobile client
CN104376118A (en) Panorama-based outdoor movement augmented reality method for accurately marking POI
CN112435338B (en) Method and device for acquiring position of interest point of electronic map and electronic equipment
CN109974733A (en) POI display methods, device, terminal and medium for AR navigation
CN113989450A (en) Image processing method, image processing apparatus, electronic device, and medium
CN104281840A (en) Method and device for positioning and identifying building based on intelligent terminal
CN109753901B (en) Indoor pedestrian tracing method and device based on pedestrian recognition, computer equipment and storage medium
CN112884841B (en) Binocular vision positioning method based on semantic target
US11915478B2 (en) Bayesian methodology for geospatial object/characteristic detection
CN113393515B (en) Visual positioning method and system combining scene annotation information
CN111767854B (en) SLAM loop detection method combined with scene text semantic information
CN114332232B (en) Smart phone indoor positioning method based on space point, line and surface feature hybrid modeling
CN115205382A (en) Target positioning method and device
CN111626971B (en) Smart city CIM real-time imaging method with image semantic perception
CN115574802A (en) Drawing and positioning method based on flat character feature center
CN115468568A (en) Indoor navigation method, device and system, server equipment and storage medium
CN112818866B (en) Vehicle positioning method and device and electronic equipment
CN111784822A (en) Smart city CIM real-time imaging method with image semantic perception
KR102555668B1 (en) Method of generating map and visual localization using the map
CN110826568A (en) Indoor positioning method and device based on road signs and electronic equipment
Zhu et al. Skyline Matching: A robust registration method between Video and GIS
Li et al. Research on Semantic Map Generation and Location Intelligent Recognition Method for Scenic SPOT Space Perception

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination