CN113607158A - Visual identification matching positioning method and system for flat light source based on visible light communication - Google Patents
Visual identification matching positioning method and system for flat light source based on visible light communication Download PDFInfo
- Publication number
- CN113607158A CN113607158A CN202110895779.8A CN202110895779A CN113607158A CN 113607158 A CN113607158 A CN 113607158A CN 202110895779 A CN202110895779 A CN 202110895779A CN 113607158 A CN113607158 A CN 113607158A
- Authority
- CN
- China
- Prior art keywords
- light source
- matching
- sequence
- rectangular
- positioning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000000007 visual effect Effects 0.000 title claims abstract description 67
- 238000000034 method Methods 0.000 title claims abstract description 59
- 238000004891 communication Methods 0.000 title claims abstract description 41
- 238000003384 imaging method Methods 0.000 claims abstract description 39
- 238000004364 calculation method Methods 0.000 claims description 18
- 238000005516 engineering process Methods 0.000 claims description 15
- 238000000605 extraction Methods 0.000 claims description 8
- 238000012545 processing Methods 0.000 claims description 8
- 238000010606 normalization Methods 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 5
- 238000003672 processing method Methods 0.000 abstract description 4
- 238000005096 rolling process Methods 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 3
- 239000000284 extract Substances 0.000 description 2
- 230000000737 periodic effect Effects 0.000 description 2
- 206010063385 Intellectualisation Diseases 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000000969 carrier Substances 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/10—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
- G01C21/12—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
- G01C21/16—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
- G01C21/165—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments
- G01C21/1656—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments with passive imaging devices, e.g. cameras
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/005—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 with correlation of navigation data from several sources, e.g. map or contour matching
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/20—Instruments for performing navigational calculations
- G01C21/206—Instruments for performing navigational calculations specially adapted for indoor navigation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04B—TRANSMISSION
- H04B10/00—Transmission systems employing electromagnetic waves other than radio-waves, e.g. infrared, visible or ultraviolet light, or employing corpuscular radiation, e.g. quantum communication
- H04B10/11—Arrangements specific to free-space transmission, i.e. transmission through air or vacuum
- H04B10/114—Indoor or close-range type systems
- H04B10/116—Visible light communication
Abstract
The invention relates to the technical field of visible light communication indoor positioning, and provides a visible light communication-based flat light source visual identification matching positioning method and system, which are used for solving the technical problems of insufficient precision, high cost and poor real-time performance of the existing indoor positioning method. The invention images the rectangular flat LED light source of visible light communication through the rolling shutter effect of the CMOS image sensor, acquires the outline of the rectangular area of the light source and the light and shade stripe information of the image by using a digital image processing method, realizes light source matching identification by defining the autocorrelation sequence and the proximity of the image signal of the light source, thereby acquiring the ID and rectangular corner coordinate information of the flat light source, realizes the matching of the image coordinates of four corners of the rectangular light source and the world coordinate relation on the basis of the IMU attitude sensor auxiliary imaging, realizes the high-precision visual positioning based on the visible light communication flat light source, realizes the centimeter-level mobile positioning, and can meet the application requirements of the indoor robot mobile positioning navigation.
Description
Technical Field
The invention relates to the technical field of indoor visible light communication visual positioning, in particular to a visual identification, matching and positioning method and system for a flat-panel light source based on visible light communication.
Background
Outdoor satellite navigation signals are difficult to receive in indoor and other underground space environments, and indoor positioning technology is needed when some indoor carriers move indoors autonomously. Currently, indoor positioning technology is rapidly developing, wherein indoor positioning technology based on visible light communication can unify location information service and lighting, and is one of the main development directions of indoor positioning technology in the future. With the upgrading of indoor lighting systems, LED lighting systems are widely adopted indoors in large buildings such as underground parking lots, shopping malls and automatic factories, and the areas are also key places for future application of automatic driving and robot intellectualization. For example, the flat-panel LED light source of a parking lot in a certain underground has a length dimension exceeding 0.5m and a width generally more than one fourth of the length, has a prominent rectangular visual characteristic, and thus can serve as an identification of a building location information service while satisfying the illumination.
Currently, the main problems of the existing indoor positioning schemes in the market are as follows:
(1) the positioning method based on radio signals has insufficient accuracy. The positioning precision of the indoor positioning method using WIFI/Bluetooth/ZigBee is generally 1-10m, which is difficult to meet the centimeter-level positioning precision requirement of the indoor robot.
(2) The cost is too high and the system is complex. The positioning scheme based on UWB (ultra wide band) has the test precision of 10cm, but needs to deploy intensive positioning base stations, the system has high cost and complex deployment, is easy to be interfered by electromagnetic waves, and has unstable positioning precision. Excessive base station deployment may interfere with the indoor electromagnetic environment.
(3) The positioning delay is large, the real-time performance is insufficient, the scheme based on the visual image monitoring can monitor and position the moving target, and the system feeds back the position information to the moving carrier after completing the positioning, so that the system has a certain delay and is not suitable for the autonomous positioning of the moving target.
(4) At present, most visible light communication visual positioning uses a circular down lamp as a positioning identifier, the size of the circular down lamp is limited, and only one positioning reference point and scale information can be provided, so that more than two circular down lamps are needed to provide effective visual positioning reference.
(5) The visual imaging decoding technology based on visible light communication has high requirements on imaging quality, high image error rate and difficulty in real-time decoding.
Disclosure of Invention
Aiming at the technical problems that the existing indoor positioning method is insufficient in precision, high in cost and poor in real-time performance and cannot meet the positioning and navigation requirements of indoor mobile robots, the invention provides a visible light communication-based flat light source visual identification matching positioning method and system.
In order to achieve the purpose, the technical scheme of the invention is realized as follows: a visual identification matching positioning method of a flat light source based on visible light communication is characterized in that an ID signal is loaded on a rectangular LED light source by using the visible light communication technology, a CMOS image sensor is used for carrying out visual imaging on the rectangular LED light source, and an IMU attitude sensor is used for recording an inclined attitude angle and a course angle during imaging; the method comprises the steps of extracting a rectangular light source image by using a digital image processing technology, obtaining a light and shade stripe gray image and a rectangular outline of a rectangular light source area, identifying ID information of a matched light source by using a signal autocorrelation method, calculating four corner points of the rectangular outline of the light source, and realizing high-precision visual positioning according to a photogrammetry principle.
Further, the CMOS image sensor adjusts an ISO value and exposure to perform visual imaging on the rectangular LED light source, and a flat-panel light source image is obtained, wherein the flat-panel light source image comprises continuous stripe information of the light source.
Further, the method for acquiring the rectangular outline of the rectangular light source region comprises the following steps: performing Gaussian operation on the flat light source image, performing binarization through an OPENCV library, and searching a contour line of a rectangular area; and extracting contour sequence points of the rectangular region by using an edge extraction method, respectively fitting four contour straight lines according to the contour sequence points, wherein the connecting point of adjacent contour straight lines is the corner point of the rectangular light source region.
Further, the method for identifying ID information of a matching light source using a signal autocorrelation method is: converting the ID information matching degree problem of the two light sources into the matching degree problem of the autocorrelation sequence of the fringe signals of the fringe gray scale images of the two light sources: calculating the period of gray stripes in the gray-scale image of the light and dark stripes based on an autocorrelation method of an auto signal, calculating an autocorrelation sequence of one period, establishing a light source matching database by using the autocorrelation sequence as matching characteristics to be associated with ID information modulated by a signal, and calculating the minimum proximity K to perform light source matching in the light source matching database.
Further, a table in the light source matching database comprises light source numbers, light source matching templates and coordinates of four corner points of the light sources; and the light sources are matched to obtain light source numbers, and world coordinates of four corner points of the light sources corresponding to the light source numbers are obtained.
Further, the method for establishing the light source matching database comprises the following steps: carrying out average calculation on the obtained light and shade stripe gray level images of the rectangular light source area line by line to generate a light and shade stripe sequence V ═ V1,v2,v3,···,vi},v1,v2,v3,···,vi∈[0,255]I is the line number of the gray light and shade fringe image; and performing signal autocorrelation calculation on the bright and dark stripe sequence V to obtain a signal autocorrelation sequence N ═ N1,n2,n3,···,nTV, and calculating the period T of the autocorrelation sequence N; extracting first T sequence values of the signal autocorrelation sequence N to carry out normalization processing, and obtaining a normalized sequence X ═ { X ═ X1,x2,x3,···,xT},x1,x2,x3,···,xT∈[0,1](ii) a And (4) associating the autocorrelation sequence N serving as the matching characteristic with the ID information of the signal modulation to establish a light source matching database.
Further, the method for calculating the proximity K comprises: defining two light source matching degree sequence DT＝X1T-X2T，X1T、X2TRespectively, a signal autocorrelation sequence normalized by one period of two light sources, and DT＝{d1,d2,d3,···,dT},d1,d2,d3,···,dT∈(-1,1)；
The degree of matching of the ID information of the two light sources, i.e., the degree of proximity K ═ a × H + b × M + c × S + d × W;
wherein a, b, c and D are coefficients, H is a sequence DTM is the difference between the maximum and minimum of (D), M is the sequence DTS is the sequence DTW is the sequence DTMiddle excess sequence DTMean 2-fold standardAn average of the sums of absolute values of the sequence of differences; a smaller value of the proximity K indicates a higher degree of matching.
Further, when the CMOS type is the same, the period T is the same, and the proximity K is the minimum, the ID information of the light source which is most matched with the rectangular flat panel light source to be positioned in the light source matching database is obtained, and the world coordinates of the four corner points of the light source corresponding to the ID information are used as the coordinates { P1, P2, P3, P4} of the corner points of the light source.
Further, when the corresponding relation between the world coordinate of the corner point of the rectangular LED light source and the image point coordinate of the rectangular LED light source is determined, the pose of the CMOS image sensor is supposed to be calculated based on a homography direct method by using one corresponding relation, the difference value between the calculated heading angle and the geomagnetic direction angle of the IMU attitude sensor is compared, if the difference value is smaller than a preset threshold value, the correct relation is determined, and if not, the calculation is supposed to be continued; the sequence corresponding relation of 4 angular points is 4 in total, and the corresponding relation can be determined by calculating for 4 times at most; and performing imaging positioning calculation by using an IMU attitude sensor assisted iteration method according to the correct corresponding relation.
Further, the preset threshold is that the deviation angle C of the heading angle and the geomagnetic azimuth angle is less than 45 degrees.
Furthermore, the photogrammetry principle uses a visual plane four-point positioning method to calculate the position information of the camera of the CMOS image sensor relative to the rectangular flat light source, so as to realize visual positioning.
A visual identification matching positioning system of a flat light source based on visible light communication comprises a rectangular flat light source and a positioning device, wherein the rectangular flat light source is a visible light communication flat LED light source; the positioning device comprises a COMS imaging sensor and an IMU attitude sensor, and the COMS imaging sensor and the IMU attitude sensor are both connected with the microprocessor.
Furthermore, the microprocessor is connected with the battery, the microprocessor and the battery are both arranged in the shell, the COMS imaging sensor is fixedly arranged on the shell, and the IMU attitude sensor is arranged in the shell; and the microprocessor processes the data acquired by the IMU attitude sensor and the COMS imaging sensor to realize visual identification matching positioning.
The invention has the beneficial effects that: the method comprises the steps of utilizing a rectangular flat light source as a visual positioning beacon, modulating and positioning ID information on the rectangular flat light source by using a visible light communication technology, imaging the flat light source through a CMOS image sensor, adjusting an ISO value to obtain continuous stripe information, calculating a stripe signal period by calculating an autocorrelation sequence of a stripe image, establishing a light source matching database by using a periodic signal autocorrelation sequence as a matching feature to be associated with the ID information, wherein the light source matching database contains a light source matching signal autocorrelation sequence and spatial three-dimensional coordinate information corresponding to 4 corner points of the light source, and providing a proximity K calculation method to realize light source visual identification matching; and the rectangular corner points of the rectangular flat light source are extracted by utilizing a digital image processing technology while the light source is identified and matched, and the visual positioning can be realized by only using one rectangular flat light source, so that the visual positioning is simpler and more robust. The positioning method based on the fusion of visible light communication and vision measurement can realize centimeter-level mobile positioning by utilizing an indoor rectangular flat-plate LED light source, and can meet the application requirements of indoor robot mobile positioning navigation.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a schematic flow chart of the present invention.
Fig. 2 is a schematic structural diagram of the present invention.
FIG. 3 is a diagram of the visual extraction effect of the rectangular flat panel light source according to the present invention.
FIG. 4 is a flow chart of visual matching of a light source according to the present invention.
FIG. 5 is a diagram illustrating the visual matching effect of a plurality of light sources according to the present invention.
FIG. 6 is a flow chart of the visual positioning of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without inventive effort based on the embodiments of the present invention, are within the scope of the present invention.
the method comprises the steps of firstly, performing signal modulation on a rectangular flat light source by utilizing a visible light communication technology, performing visual imaging on the rectangular flat light source by utilizing a CMOS image sensor, and simultaneously recording an inclined attitude angle and a course angle during the visual imaging by utilizing an IMU attitude sensor.
Aiming at the visible light communication flat LED light source, a flat light source matching algorithm and a positioning reference point extraction method are provided, and an indoor visual positioning system based on the flat light source is realized. And the visible light communication signal modulation of the rectangular flat-plate light source is realized by utilizing the visible light communication technology, and the ID information is modulated. In the visible light communication technology, signal modulation is carried out based on an LED-ID technology, the modulation frequency is 16kHz, Manchester coding is adopted, and 3 hexadecimal digits are formed.
The invention uses a common CMOS image sensor, namely a CMOS rolling shutter door opening camera to adjust the ISO value and the exposure to perform visual imaging on a light source, and a flat light source image obtained as shown in figure 3 comprises two light source images, a rectangular light source and a square light source; the flat light source image comprises continuous stripe information of the rectangular flat light source. The ISO value is a parameter for adjusting the photosensitive strength of the CMOS image sensor, and is used for matching with the exposure to enable the light source image to be more easily distinguished to form a bright and dark stripe image. The vision imaging is used for subsequent vision positioning, the vision imaging positioning precision is high, and meanwhile, the light source imaging identification can be realized. The positioning device is internally provided with an IMU attitude sensor which is integrated with the CMOS image sensor into an imaging positioning module device, the IMU attitude sensor is used for assisting imaging, and the course angle measured by the IMU is used for judging the corresponding relation between the four corner points of the light source image and the coordinates in the world coordinate system. A light and shade stripe sequence obtained by controlling a photosensitive ISO value of the CMOS image sensor has continuity, and a light source rectangular contour line is extracted by a contour edge extraction method of an OPENCV library of digital image processing. The invention uses the rectangular LED light source, and extracts the stripe information and the rectangular outline characteristics.
And step two, extracting a rectangular area of the rectangular flat plate light source in the visual imaging image by using a computer vision extraction algorithm, respectively fitting 4 straight lines on four outline sides of the rectangular area, intersecting the 4 straight lines to obtain 4 angular point coordinates of the light source, and performing reverse-time needle sequencing on the 4 angular point image coordinates.
The realization method of the computer vision extraction algorithm comprises the following steps: the method comprises the steps of obtaining an image of a rectangular light source acquired by visual imaging of a CMOS image sensor, firstly carrying out Gaussian operation on the image, then carrying out binarization through an OPENCV library and searching for a rectangular outline. And extracting contour image points of the light source region by using a digital image edge extraction method, fitting four contour straight lines according to the light source contour sequence points, wherein the connecting points of the contour straight lines are rectangular corner points of the flat light source region. And obtaining four corner point coordinates of the rectangular light source through light source contour point fitting. The rectangular contour points are sequences, four straight lines can be fitted firstly, and then adjacent straight lines are intersected to obtain four corner points for subsequent visual positioning reference.
And step three, calculating the period of the light source image gray scale stripe based on a signal autocorrelation method, calculating an autocorrelation sequence of one period, and establishing a light source matching database by using the autocorrelation sequence as a matching characteristic to be associated with the ID information modulated by the signal in the step one.
The present application differs from the invention patent with application number 201710150722.9 in that: the method has the advantages of identifying robustness, high image requirement of decoding, easy decoding error or no identification, and capability of completing light source matching identification by using a common camera.
Signal autocorrelation is a common digital signal processing method. The light source is matched to obtain a light source number, the light source number corresponds to world coordinates of four corner points of the light source, and the world coordinates are measured in advance and are input in advance in database information.
Carrying out mean value calculation on the extracted light source gray stripe image line by line to generate a light and shade stripe sequence V ═ { V ═ V }1,v2,v3,···,vi},v∈[0,255]And i is the line number of the gray stripe image, corresponding to the line of the image. And performing signal autocorrelation calculation on the bright and dark stripe sequence V to obtain a signal autocorrelation sequence N ═ N1,n2,n3,···,nTAnd calculating the period T of the autocorrelation sequence N, wherein the periodic function has a maximum value at the multiple of the period T.
Extracting first T sequence values of the signal autocorrelation sequence N to carry out normalization processing, and obtaining a normalized sequence X ═ { X ═ X1,x2,x3,···,xT},x∈[0,1]Normalized range. And (4) establishing a light source matching database by using the autocorrelation sequence as matching characteristics and associating the autocorrelation sequence with the ID information modulated by the signal in the step one, wherein tables in the light source matching database comprise light source numbers, light source matching templates and coordinates of four corner points of the light source.
And fourthly, matching a light source matching database based on a minimum proximity K (nearest neighbor calculation method) calculation method, acquiring four corner point reference coordinates of the rectangular flat light source, and realizing visual identification and matching of the light source.
The light source stripes can obtain a normalized numerical sequence X of one period, the sequence X has a T dimension, the sequence is a matching template, and the calculation method of the proximity K is a calculation method of the matching degree of the two sequences. The ID information matching degree problem of the two light sources of one period standardization sequence X is converted into the matching degree problem of the autocorrelation sequence of the fringe signals of the gray fringe images of the two light sources. Defining two light source matching degree sequence DT＝X1T-X2T，X1T、X2TRespectively, a signal autocorrelation sequence normalized by one period of two light sources, and DT＝{d1,d2,d3,···,dT},d∈(-1,1)。
The ID information matching degree of the two light sources is defined as: k ═ a × H + b × M + c × S + d × W;
wherein a, b, c and D are coefficients, H is a sequence DTM is the difference between the maximum and minimum of (D), M is the sequence DTS is the sequence DTW is the sequence DTMiddle excess sequence DTMean 2 times the mean of the sum of the absolute values of the sequences of standard deviations. As a specific example, when the coefficient a is 0.25; b is 0.25, c is 4, d is 0.02; a smaller value of the proximity K indicates a higher degree of matching.
Specifically, the light source matching process is as shown in fig. 4, for obtaining a gray stripe image of a rectangular flat panel LED light source, obtaining the type of a CMOS image sensor, extracting a light and dark stripe sequence V of the gray stripe image, calculating and extracting a period autocorrelation sequence and normalizing the period autocorrelation sequence into a normalized sequence X, calculating the proximity K to match with a light source information (matching) database, and when the CMOS type is the same, the period T is the same, and the minimum proximity K is the best matching condition, obtaining ID information of a light source in the light source information database that is the best matched with the rectangular flat panel light source to be positioned (i.e., the imaging to be matched), and using the ID information as coordinates { P1, P2, P3, P4} of the light source corner point. According to the light source matching process, the visual matching effect of the plurality of light sources is shown in fig. 5, and as can be seen from fig. 5, the light source matching method can solve the problem of multi-light source identification and can extract four corner points of the light sources.
And fifthly, calculating the position information of the camera of the CMOS image sensor relative to the rectangular flat light source by using a visual plane four-point positioning method to realize visual positioning. And (4) converting the four corner point reference coordinates of the rectangular flat light source obtained in the step four into world coordinates by using a visual plane four-point positioning method, so as to realize the positioning of the rectangular flat light source.
Acquiring four corner coordinates of a rectangular flat plate light source corresponding to ID information according to light source visual matching, when determining the corresponding relation between the corner world coordinates of the rectangular light source and the image point coordinates thereof, firstly, assuming that the position and attitude of the CMOS image sensor are calculated based on a homography direct method by using a corresponding relation, then comparing the calculated heading angle with the geomagnetic direction difference value of the IMU attitude sensor, if the difference value is less than a certain threshold value, determining a correct relation, otherwise, continuing to assume calculation, and the flow is shown in FIG. 6. The sequence corresponding relations of the 4 angular points are 4 in total, so the corresponding relations can be determined by calculating 4 times at most, then the coordinates of the positioning device are calculated by utilizing a common visual positioning algorithm according to the correct corresponding relations, and the imaging positioning calculation is carried out by utilizing an IMU attitude sensor auxiliary iteration method.
The digital image processing method of the opencv library obtains coordinates { C1, C2, C3 and C4} of an image anticlockwise sequence, the homography direct method is that a corresponding relation between a rectangular corner point sequence world coordinate and an image coordinate sequence is assumed, a heading angle H of a CMOS image sensor is calculated according to a visual positioning algorithm, a three-dimensional position and a three-dimensional attitude are obtained through imaging positioning, one attitude is the heading angle H, a geomagnetic azimuth angle M of an IMU sensor is obtained simultaneously, a deviation angle C between the heading angle H and the geomagnetic azimuth angle M is calculated, and if the C is smaller than 45 degrees, a correct coordinate corresponding relation is output.
The visible light communication rectangular flat-panel light source is imaged by the rolling shutter effect of the CMOS imaging sensor, the outline of a rectangular area of the light source and light and shade stripe information of an image are obtained by a digital image processing method, light source matching identification is realized by defining a light source image signal autocorrelation sequence and a matching degree K, so that the ID of the light source and rectangular corner point coordinate information of the flat-panel light source are obtained, matching of the relation between four corner point image coordinates of the rectangular light source and world coordinates is realized on the basis of IMU attitude sensor auxiliary imaging, and high-precision visual positioning based on the visible light communication flat-panel light source is further realized.
Other structures and principles are the same as those of embodiment 1.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (13)
1. A visual identification matching positioning method of a flat light source based on visible light communication is characterized in that an ID signal is loaded on a rectangular LED light source by using the visible light communication technology, a CMOS image sensor is used for carrying out visual imaging on the rectangular LED light source, and an IMU attitude sensor is used for recording an inclination attitude angle and a course angle during imaging; the method comprises the steps of extracting a rectangular light source image by using a digital image processing technology, obtaining a light and shade stripe gray image and a rectangular outline of a rectangular light source area, identifying ID information of a matched light source by using a signal autocorrelation method, calculating four corner points of the rectangular outline of the light source, and realizing high-precision visual positioning according to a photogrammetry principle.
2. The visual identification, matching and positioning method for flat-panel light sources based on visible light communication as claimed in claim 1, wherein the CMOS image sensor adjusts ISO values and exposure levels to perform visual imaging on the rectangular LED light sources to obtain flat-panel light source images, and the flat-panel light source images include continuous stripe information of the light sources.
3. The visual identification, matching and positioning method for the flat-panel light source based on visible light communication of claim 2, wherein the rectangular outline of the rectangular light source area is obtained by: performing Gaussian operation on the flat light source image, performing binarization through an OPENCV library, and searching a contour line of a rectangular area; and extracting contour sequence points of the rectangular region by using an edge extraction method, respectively fitting four contour straight lines according to the contour sequence points, wherein the connecting point of adjacent contour straight lines is the corner point of the rectangular light source region.
4. The visual identification, matching and positioning method for the flat panel light source based on visible light communication as claimed in claim 1 or 3, wherein the method for identifying the ID information of the matching light source by using the signal autocorrelation method is as follows: converting the ID information matching degree problem of the two light sources into the matching degree problem of the autocorrelation sequence of the fringe signals of the fringe gray scale images of the two light sources: calculating the period of gray stripes in the gray-scale image of the light and dark stripes based on an autocorrelation method of an auto signal, calculating an autocorrelation sequence of one period, establishing a light source matching database by using the autocorrelation sequence as matching characteristics to be associated with ID information modulated by a signal, and calculating the minimum proximity K to perform light source matching in the light source matching database.
5. The visual identification, matching and positioning method for the flat-panel light source based on visible light communication as claimed in claim 4, wherein the table in the light source matching database comprises light source numbers, light source matching templates and coordinates of four corner points of the light source; and the light sources are matched to obtain light source numbers, and world coordinates of four corner points of the light sources corresponding to the light source numbers are obtained.
6. The visual identification, matching and positioning method for the flat-panel light source based on visible light communication as claimed in claim 4, wherein the method for establishing the light source matching database is: carrying out average calculation on the obtained light and shade stripe gray level images of the rectangular light source area line by line to generate a light and shade stripe sequence V ═ V1,v2,v3,···,vi},v1,v2,v3,···,vi∈[0,255]I is the line number of the gray light and shade fringe image; and performing signal autocorrelation calculation on the bright and dark stripe sequence V to obtain a signal autocorrelation sequence N ═ N1,n2,n3,···,nTV., and calculating the autocorrelationA period T of the sequence N; extracting first T sequence values of the signal autocorrelation sequence N to carry out normalization processing, and obtaining a normalized sequence X ═ { X ═ X1,x2,x3,···,xT},x1,x2,x3,···,xT∈[0,1](ii) a And (4) associating the autocorrelation sequence N serving as the matching characteristic with the ID information of the signal modulation to establish a light source matching database.
7. The visual identification, matching and positioning method for the flat panel light source based on visible light communication as claimed in claim 6, wherein the proximity K is calculated by: defining two light source matching degree sequence DT＝X1T-X2T，X1T、X2TRespectively, a signal autocorrelation sequence normalized by one period of two light sources, and DT＝{d1,d2,d3,···,dT},d1,d2,d3,···,dT∈(-1,1)；
The degree of matching of the ID information of the two light sources, i.e., the degree of proximity K ═ a × H + b × M + c × S + d × W;
wherein a, b, c and D are coefficients, H is a sequence DTM is the difference between the maximum and minimum of (D), M is the sequence DTS is the sequence DTW is the sequence DTMiddle excess sequence DTAverage of the sum of the absolute values of the sequences with an average of 2 times the standard deviation; a smaller value of the proximity K indicates a higher degree of matching.
8. The visual identification, matching and positioning method for flat-panel light sources based on visible light communication as claimed in claim 7, wherein the optimal matching condition is obtained when the proximity K is the minimum and the period T is the same as the CMOS type, and the ID information of the light source which is the most matched with the rectangular flat-panel light source to be positioned in the light source matching database is obtained, and the world coordinates of the four corner points of the light source corresponding to the ID information are used as the coordinates { P1, P2, P3, P4} of the corner points of the light source.
9. The visual identification, matching and positioning method for the flat-panel light source based on visible light communication as claimed in claim 8, wherein when determining the corresponding relationship between the world coordinates of the corner points of the rectangular LED light source and the coordinates of the image points thereof, it is assumed that the pose of the CMOS image sensor is calculated based on a homography direct method with one corresponding relationship, the difference between the calculated heading angle and the geomagnetic direction angle of the IMU pose sensor is compared, if the difference is smaller than a preset threshold, it is correct, otherwise, it is assumed that the calculation is continued; the sequence corresponding relation of 4 angular points is 4 in total, and the corresponding relation can be determined by calculating for 4 times at most; and performing imaging positioning calculation by using an IMU attitude sensor assisted iteration method according to the correct corresponding relation.
10. The visual identification, matching and positioning method for flat panel light sources based on visible light communication as claimed in claim 9, wherein the preset threshold is a deviation angle C of a heading angle and a geomagnetic azimuth angle is less than 45 °.
11. The visual identification, matching and positioning method for flat panel light sources based on visible light communication as claimed in claim 9, wherein the photogrammetry principle uses a visual plane four-point positioning method to calculate the position information of a camera of a CMOS image sensor relative to a rectangular flat panel light source, so as to realize visual positioning.
12. The visual identification matching positioning system of the flat panel light source based on visible light communication according to any one of claims 1-3 and 2-11, characterized by comprising a rectangular flat panel light source (1) and a positioning device, wherein the rectangular flat panel light source (1) is a visible light communication flat panel LED light source; the positioning device comprises a COMS imaging sensor (2) and an IMU attitude sensor (3), and the COMS imaging sensor (2) and the IMU attitude sensor (3) are both connected with a microprocessor (4).
13. The positioning system according to claim 12, wherein the microprocessor (4) is connected with a battery (5), the microprocessor (4) and the battery (5) are both arranged in the housing (6), the COMS imaging sensor (2) is fixedly arranged on the housing (6), and the IMU attitude sensor (3) is arranged in the housing (6); and the microprocessor (4) processes data acquired by the IMU attitude sensor (3) and the COMS imaging sensor (2) to realize visual identification matching positioning.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110895779.8A CN113607158A (en) | 2021-08-05 | 2021-08-05 | Visual identification matching positioning method and system for flat light source based on visible light communication |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110895779.8A CN113607158A (en) | 2021-08-05 | 2021-08-05 | Visual identification matching positioning method and system for flat light source based on visible light communication |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113607158A true CN113607158A (en) | 2021-11-05 |
Family
ID=78307053
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110895779.8A Pending CN113607158A (en) | 2021-08-05 | 2021-08-05 | Visual identification matching positioning method and system for flat light source based on visible light communication |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113607158A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114124218A (en) * | 2021-11-22 | 2022-03-01 | 华南理工大学 | Geomagnetic sensor-assisted visible light single lamp positioning method and system |
| CN114157357A (en) * | 2022-01-07 | 2022-03-08 | 吉林大学 | Multi-amplitude visible light signal imaging communication demodulation method supporting terminal rotation translation |
| CN115065409A (en) * | 2022-04-28 | 2022-09-16 | 浙江大学湖州研究院 | Visible light indoor communication and positioning integrated system based on wavelength division multiplexing |
| CN117475359A (en) * | 2023-12-27 | 2024-01-30 | 湖北经济学院 | LED lamp identification method and device, electronic equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103918008A (en) * | 2011-11-08 | 2014-07-09 | 皇家飞利浦有限公司 | System and method for interactive image annotation |
| CN103940419A (en) * | 2013-04-09 | 2014-07-23 | 珠海横琴华策光通信科技有限公司 | Indoor navigation method, device and system |
| CN205896711U (en) * | 2016-07-15 | 2017-01-18 | 常熟理工学院 | Full gloss register for easy reference plant light filling lamp of photovoltaic power supply |
| WO2019214643A1 (en) * | 2018-05-09 | 2019-11-14 | 北京外号信息技术有限公司 | Method for guiding autonomously movable machine by means of optical communication device |
| CN110471402A (en) * | 2018-05-09 | 2019-11-19 | 陕西外号信息技术有限公司 | The system and method that the machine for capableing of autonomous is guided |
| WO2020155615A1 (en) * | 2019-01-28 | 2020-08-06 | 速感科技（北京）有限公司 | Vslam method, controller, and mobile device |
-
2021
- 2021-08-05 CN CN202110895779.8A patent/CN113607158A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103918008A (en) * | 2011-11-08 | 2014-07-09 | 皇家飞利浦有限公司 | System and method for interactive image annotation |
| CN103940419A (en) * | 2013-04-09 | 2014-07-23 | 珠海横琴华策光通信科技有限公司 | Indoor navigation method, device and system |
| CN205896711U (en) * | 2016-07-15 | 2017-01-18 | 常熟理工学院 | Full gloss register for easy reference plant light filling lamp of photovoltaic power supply |
| WO2019214643A1 (en) * | 2018-05-09 | 2019-11-14 | 北京外号信息技术有限公司 | Method for guiding autonomously movable machine by means of optical communication device |
| CN110471402A (en) * | 2018-05-09 | 2019-11-19 | 陕西外号信息技术有限公司 | The system and method that the machine for capableing of autonomous is guided |
| WO2020155615A1 (en) * | 2019-01-28 | 2020-08-06 | 速感科技（北京）有限公司 | Vslam method, controller, and mobile device |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114124218A (en) * | 2021-11-22 | 2022-03-01 | 华南理工大学 | Geomagnetic sensor-assisted visible light single lamp positioning method and system |
| CN114157357A (en) * | 2022-01-07 | 2022-03-08 | 吉林大学 | Multi-amplitude visible light signal imaging communication demodulation method supporting terminal rotation translation |
| CN114157357B (en) * | 2022-01-07 | 2023-08-22 | 吉林大学 | Multi-amplitude visible light signal imaging communication demodulation method supporting terminal rotation translation |
| CN115065409A (en) * | 2022-04-28 | 2022-09-16 | 浙江大学湖州研究院 | Visible light indoor communication and positioning integrated system based on wavelength division multiplexing |
| CN115065409B (en) * | 2022-04-28 | 2023-08-22 | 浙江大学湖州研究院 | Visible light indoor communication and positioning integrated system based on wavelength division multiplexing |
| CN117475359A (en) * | 2023-12-27 | 2024-01-30 | 湖北经济学院 | LED lamp identification method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113607158A (en) | Visual identification matching positioning method and system for flat light source based on visible light communication | |
| US11003972B2 (en) | Localizing a vehicle's charging or fueling port—methods and apparatuses | |
| CN103162682B (en) | Based on the indoor path navigation method of mixed reality | |
| CN108495251B (en) | Combined positioning method based on Bluetooth and LED optical positioning | |
| CN109724603A (en) | A kind of Indoor Robot air navigation aid based on environmental characteristic detection | |
| CN110470295A (en) | A kind of indoor walking navigation and method based on AR positioning | |
| CN112184765B (en) | Autonomous tracking method for underwater vehicle | |
| CN111780769A (en) | Single-lamp visible light positioning method | |
| CN114721001A (en) | Mobile robot positioning method based on multi-sensor fusion | |
| CN110133592A (en) | A kind of indoor two-point locating method based on visible light communication | |
| CN103759724A (en) | Indoor navigation method based on decorative lighting characteristic and system | |
| Huang et al. | VWR-SLAM: Tightly coupled SLAM system based on visible light positioning landmark, wheel odometer, and RGB-D camera | |
| Li et al. | High-accuracy and real-time indoor positioning system based on visible light communication and mobile robot | |
| CN111090074A (en) | Indoor visible light positioning method and equipment based on machine learning | |
| CN108111223A (en) | LED positioners, indoor locating system and indoor orientation method | |
| Shu et al. | 3d point cloud-based indoor mobile robot in 6-dof pose localization using a wi-fi-aided localization system | |
| Karakaya et al. | Low Cost easy-to-install indoor positioning system | |
| CN113469901A (en) | Positioning device based on passive infrared tag | |
| CN107330939B (en) | Visible light positioning system and method based on mine roadway lamp | |
| CN107196704A (en) | Method of locating terminal, device and electronic equipment | |
| CN109099891B (en) | Spatial positioning method, device and system based on image recognition | |
| CN114758364B (en) | Industrial Internet of things scene fusion positioning method and system based on deep learning | |
| CN108107406A (en) | LED positioners, indoor locating system and indoor orientation method | |
| CN114370871A (en) | Close coupling optimization method for visible light positioning and laser radar inertial odometer | |
| CN115014338A (en) | Mobile robot positioning system and method based on two-dimensional code vision and laser SLAM |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |