CN110648313B - Laser stripe center line fitting method based on FPGA - Google Patents

Laser stripe center line fitting method based on FPGA Download PDF

Info

Publication number
CN110648313B
CN110648313B CN201910836578.3A CN201910836578A CN110648313B CN 110648313 B CN110648313 B CN 110648313B CN 201910836578 A CN201910836578 A CN 201910836578A CN 110648313 B CN110648313 B CN 110648313B
Authority
CN
China
Prior art keywords
matrix
output
computation element
adder
multiplier
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910836578.3A
Other languages
Chinese (zh)
Other versions
CN110648313A (en
Inventor
程杨
黄踔
刘渊
霍舒豪
张德兆
王肖
李晓飞
张放
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Idriverplus Technologies Co Ltd
Original Assignee
Beijing Idriverplus Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Idriverplus Technologies Co Ltd filed Critical Beijing Idriverplus Technologies Co Ltd
Priority to CN201910836578.3A priority Critical patent/CN110648313B/en
Publication of CN110648313A publication Critical patent/CN110648313A/en
Application granted granted Critical
Publication of CN110648313B publication Critical patent/CN110648313B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Mathematical Analysis (AREA)
  • Computational Mathematics (AREA)
  • Mathematical Optimization (AREA)
  • Pure & Applied Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Computing Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Quality & Reliability (AREA)
  • Algebra (AREA)
  • Databases & Information Systems (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Image Processing (AREA)

Abstract

The invention provides a laser stripe center line fitting method based on an FPGA (field programmable gate array), which comprises the following steps of: collecting a laser line image shot by a camera through an FPGA; acquiring the abscissa of a first number of pixel points in the laser line image in a camera coordinate system to generate a first matrix, acquiring the brightness of the first number of pixel points to generate a second matrix; the first number is a positive number not less than 2; calculating a third matrix according to the first matrix; calculating a adjoint matrix of the third matrix according to the third matrix; calculating a flag bit according to the first column of the third matrix and the first row of the adjoint matrix of the third matrix; when the zone bit is not 0, calculating a fourth matrix according to the second matrix, the third matrix and the adjoint matrix of the third matrix; calculating a first coordinate of the center point of the laser line of the current row according to the fourth matrix; and calculating the central line of the laser stripe according to the first coordinate of the central point of the laser lines of all the rows. Therefore, the calculation is good in real-time performance and high in accuracy.

Description

Laser stripe center line fitting method based on FPGA
Technical Field
The invention relates to the Field of computer vision, in particular to a laser stripe center line fitting method based on a Field-Programmable Gate Array (FPGA).
Background
Laser line scanning of a workpiece to obtain a three-dimensional profile of the workpiece is popular in the current computer vision field. The laser line scans the workpiece on the conveyor belt, and the camera acquires a laser image. The laser line has the characteristic that the middle brightness is high and the laser line gradually becomes dark towards two sides, and as shown in fig. 1, the central line of the laser stripe needs to be found out for calculating the three-dimensional point cloud.
At present, Labview, Matlab and opencv are extracted from the central line of the laser stripe to skeletonize the image by calling a library function method.
Labview and Matlab cannot realize real-time image processing. The Opencv has longer calculation time, is not beneficial to high-frame-rate laser line image processing, and has non-ideal detection effect on thicker laser lines.
Disclosure of Invention
The embodiment of the invention aims to provide a laser stripe center line fitting method based on an FPGA (field programmable gate array) so as to solve the problems that the image real-time processing cannot be realized or the processing effect is not ideal in the prior art.
In order to solve the above problem, in a first aspect, the present invention provides a method for fitting a laser stripe centerline based on an FPGA, where the method includes:
collecting a laser line image shot by a camera through an FPGA;
acquiring the abscissa of a first number of pixel points in the laser line image in a camera coordinate system to generate a first matrix, acquiring the brightness of the first number of pixel points to generate a second matrix; the first number is a positive number not less than 2;
calculating a third matrix according to the first matrix;
calculating a adjoint matrix of the third matrix according to the third matrix;
calculating a flag bit according to the first column of the third matrix and the first row of the adjoint matrix of the third matrix;
when the zone bit is not 0, calculating a fourth matrix according to the second matrix, the third matrix and the adjoint matrix of the third matrix;
calculating a first coordinate of the center point of the laser line of the current row according to the fourth matrix;
and calculating the central line of the laser stripe according to the first coordinate of the central point of the laser lines of all the rows.
In a possible implementation manner, the acquiring a first number of pixel points in the laser line image before the abscissa of the camera coordinate system further includes:
and when the gray value of the pixel point in the laser line image is greater than a preset threshold value, determining that the laser line is detected, and taking the pixel point as one pixel point in a first number of pixel points.
In one possible implementation, when the first number is 7, the generated first matrix is:
Figure BDA0002192360880000021
wherein x is0-x6The abscissa of 7 pixels with pixels larger than a preset threshold in a camera coordinate system.
In one possible implementation, when the first number is 7, the generated second matrix is:
Figure BDA0002192360880000022
wherein l0-l6In sequence with x0-x6Correspondingly, the brightness value is the brightness value of the first number of pixel points.
In a possible implementation manner, the calculating a third matrix according to the first matrix specifically includes:
using the formula A-XXTCalculating a third matrix;
wherein A is the third matrix and X is the first matrix.
In one possible implementation, the FPGA includes a plurality of matrix computing units, and when the first number is 7, the using formula a ═ XX is set forthTAnd calculating a third matrix, specifically comprising:
at the first clock, the multiplier output of the first matrix computation element PE1
Figure BDA0002192360880000031
Input device
Figure BDA0002192360880000032
And
Figure BDA0002192360880000033
multiplier input of the second matrix computation element PE2
Figure BDA0002192360880000034
And x0；
At the second clock, the multiplier output of the first matrix computation element PE1
Figure BDA0002192360880000035
Input device
Figure BDA0002192360880000036
And
Figure BDA0002192360880000037
adder output
Figure BDA0002192360880000038
Multiplier output of the second matrix computation element PE2
Figure BDA0002192360880000039
Input device
Figure BDA00021923608800000310
And x1(ii) a Multiplier input of the third matrix computation element PE3
Figure BDA00021923608800000311
And 1;
at the third clock, the first matrix computation element PE1 outputs a multiplier
Figure BDA00021923608800000312
Input device
Figure BDA00021923608800000313
And
Figure BDA00021923608800000314
adder output
Figure BDA00021923608800000315
Second matrix computation element PE2 timesOutput of the law
Figure BDA00021923608800000316
Input device
Figure BDA00021923608800000317
And x2The adder output
Figure BDA00021923608800000318
Multiplier output of the third matrix computation element PE3
Figure BDA00021923608800000319
Input device
Figure BDA00021923608800000320
And 1; multiplier input x of the sixth matrix computation element PE60And 1;
at the fourth clock, the multiplier output of the first matrix computation element PE1
Figure BDA00021923608800000321
Input the method
Figure BDA00021923608800000322
And
Figure BDA00021923608800000323
adder output
Figure BDA00021923608800000324
Second matrix computation element PE2 multiplier output
Figure BDA00021923608800000325
Input device
Figure BDA00021923608800000326
And x3The adder output
Figure BDA00021923608800000327
Multiplier output of the third matrix computation element PE3
Figure BDA00021923608800000328
Input device
Figure BDA00021923608800000329
And 1, adder output
Figure BDA00021923608800000330
Multiplier output x of sixth matrix computation element PE60Inputting x1And 1;
at the fifth clock, the multiplier output of the first matrix computation element PE1
Figure BDA00021923608800000331
Input device
Figure BDA00021923608800000332
And
Figure BDA00021923608800000333
adder output
Figure BDA00021923608800000334
Second matrix computing element PE2 multiplier output
Figure BDA00021923608800000335
Input device
Figure BDA00021923608800000336
And x4The adder output
Figure BDA00021923608800000337
Multiplier output of the third matrix computation element PE3
Figure BDA00021923608800000338
Input device
Figure BDA00021923608800000339
And 1, adder output
Figure BDA00021923608800000340
The multiplier output x of the sixth matrix computation element PE61Inputting x2And 1, adder output x0；
At the sixth clock, the multiplier output of the first matrix computation element PE1
Figure BDA00021923608800000341
Input device
Figure BDA00021923608800000342
And
Figure BDA00021923608800000343
adder output
Figure BDA00021923608800000344
Second matrix computation element PE2 multiplier output
Figure BDA00021923608800000345
Input the method
Figure BDA00021923608800000346
And x5The adder output
Figure BDA0002192360880000041
Multiplier output of the third matrix computation element PE3
Figure BDA0002192360880000042
Input device
Figure BDA0002192360880000043
And 1, adder output
Figure BDA0002192360880000044
Multiplier output x of sixth matrix computation element PE62Inputting x3And 1, adder output x0+x1；
At the seventh clock, the multiplier output of the first matrix computation element PE1
Figure BDA0002192360880000045
Adder output
Figure BDA0002192360880000046
Second matrix computation element PE2 multiplier output
Figure BDA0002192360880000047
Input device
Figure BDA0002192360880000048
And x6The adder output
Figure BDA0002192360880000049
Multiplier output of the third matrix computation element PE3
Figure BDA00021923608800000410
Input device
Figure BDA00021923608800000411
And 1, adder output
Figure BDA00021923608800000412
Multiplier output x of sixth matrix computation element PE63Inputting x4And 1, adder output x0+x1+x2；
At the eighth clock, the first matrix computation element PE1 adder outputs
Figure BDA00021923608800000413
Second matrix computation element PE2 multiplier output
Figure BDA00021923608800000414
Adder output
Figure BDA00021923608800000415
Multiplier output of the third matrix computation element PE3
Figure BDA00021923608800000416
Input device
Figure BDA00021923608800000417
And 1, adder output
Figure BDA00021923608800000418
Multiplier output x of sixth matrix computation element PE64Inputting x5And 1, adder output x0+x1+x2+x3；
At the ninth clock, the first matrix computation element PE1 adder outputs
Figure BDA00021923608800000419
Second matrix computation element PE2 adder output
Figure BDA00021923608800000420
Figure BDA00021923608800000421
Multiplier output of the third matrix computation element PE3
Figure BDA00021923608800000422
Adder output
Figure BDA00021923608800000423
Figure BDA00021923608800000424
The multiplier output x of the sixth matrix computation element PE65Inputting x6And 1, adder output x0 +x1+x2+x3+x4；
At the tenth clock, the first matrix computation element PE1 adder outputs
Figure BDA00021923608800000425
Second matrix computation element PE2 adder output
Figure BDA00021923608800000426
Figure BDA00021923608800000427
Third matrix computation element PE3 adder output
Figure BDA00021923608800000428
Multiplier output x of sixth matrix computation element PE66Adder output x0+x1+x2+x3+x4+x5；
At the eleventh clock, the first matrix computation element PE1 adder outputs
Figure BDA0002192360880000051
Second matrix computation element PE2 adder output
Figure BDA0002192360880000052
Figure BDA0002192360880000053
Third matrix computation element PE3 adder output
Figure BDA0002192360880000054
Sixth matrix computation element PE6 adder output x0+x1+x2+x3+x4+x5+x6。
In a possible implementation manner, each matrix calculation unit includes a multiplier, an adder, and a register, an output of the multiplier is connected to a first input terminal of the adder, an output of the adder is connected to an input terminal of the register, and an output of the register is connected to a second input terminal of the adder;
the multiplier performs multiplication operation on the input variable, and the output result is cached through the register and is used as the input variable of the next addition operation.
In a possible implementation manner, the calculating a flag bit according to the first column of the third matrix and the first row of the companion matrix of the third matrix specifically includes:
using formulas
Figure BDA0002192360880000055
Calculating a flag bit;
wherein (A)11 A21 A31) Is the first column of the third matrix and,
Figure BDA0002192360880000056
the first row of the companion matrix of the third matrix.
In a possible implementation manner, when the flag bit is not 0, calculating a fourth matrix according to the second matrix, the third matrix, and a companion matrix of the third matrix, specifically includes:
using formulas
Figure BDA0002192360880000057
Calculating a fourth matrix;
wherein L is the second matrix, A is the third matrix, and B is the adjoint matrix of the third matrix.
In a possible implementation manner, the calculating, according to the fourth matrix, a first coordinate of a center point of a current line laser line specifically includes:
by using
Figure BDA0002192360880000058
Calculating a first coordinate of the center point of the current line laser line;
wherein, C0And C1The first and second entries in the fourth matrix.
In a second aspect, the invention provides an apparatus comprising a memory for storing a program and a processor for performing the method of any of the first aspects.
In a third aspect, the present invention provides a computer program product comprising instructions which, when run on a computer, cause the computer to perform the method according to any one of the first aspect.
In a fourth aspect, the invention provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the method of any of the first aspects.
By applying the FPGA-based laser stripe center line fitting method provided by the embodiment of the invention, the FPGA is used for acquiring the camera image, after preprocessing, the coordinate calculation of the laser stripe center line is simultaneously carried out by adopting a pipeline structure, the real-time performance is good, and the accuracy is high by adopting a parabolic symmetry axis fitting method.
Drawings
Fig. 1 is a schematic flow chart of a method for fitting a laser stripe center line based on an FPGA according to an embodiment of the present invention;
FIG. 2 is a schematic view of a laser line image according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a first number of pixel points according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a matrix calculation unit according to an embodiment of the present invention;
fig. 5 is a schematic diagram of a third matrix calculation according to an embodiment of the present invention;
fig. 6 is a schematic diagram of calculating a adjoint matrix of a third matrix according to an embodiment of the present invention.
Detailed Description
The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be further noted that, for the convenience of description, only the portions related to the related invention are shown in the drawings.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
Fig. 1 is a schematic flow chart of a method for fitting a laser stripe center line based on an FPGA according to an embodiment of the present invention, where an execution subject of the method is a terminal, a server, and other devices having a calculation function, and an application scenario of the method is an unmanned vehicle or a robot with a camera and a lidar. As shown in fig. 1, the method comprises the steps of:
and 101, acquiring a laser line image shot by a camera through the FPGA.
Specifically, by way of example and not limitation, in one scenario, a laser line on a laser radar scans a workpiece on a conveyor belt, and a camera acquires an image of the laser line during the scanning process. As shown in fig. 2, the laser line image has the characteristics of large middle brightness and gradually darkening towards two sides, and the center line of the laser stripe needs to be found for calculating the three-dimensional laser point cloud. According to the method and the device, the FPGA can be utilized to acquire the laser line image shot by the camera, the advantage of parallel processing of data by the FPGA can be utilized subsequently, and the data processing speed is increased.
By way of example and not limitation, a black-and-white camera can be directly used for shooting, so that the image is not required to be subjected to gray processing subsequently, the amount of calculation is reduced, and the processing speed of the FPGA is increased. 102, acquiring the abscissa of a first number of pixel points in the laser line image in a camera coordinate system to generate a first matrix, acquiring the brightness of the first number of pixel points, and generating a second matrix. The first number is a positive number not less than 2.
Wherein, before step 102, the method further comprises:
when the gray value of a pixel point in the laser line image is larger than a preset threshold value, the laser line is determined to be detected, and the pixel point is used as one pixel point in a first number of pixel points.
In one example, for a laser line image acquired by the FPGA, there are many rows of the laser line image due to the resolution of the laser line image, e.g., 640 rows, 480 columns for a 640 x 480 resolution image. Referring to fig. 3, for the current line, the first number is 7, and 7 pixel coordinates (x) on the laser line can be taken0,l0),(x1,l1),(x2,l2),(x3,l3),(x4,l4),(x5,l5), (x6,l6) Wherein, in the step (A),x0-x6is the abscissa, l, of 7 pixels with pixels greater than a preset threshold in a camera coordinate system0-l6In sequence with x0-x6Correspondingly, the brightness values of the seven pixel points are obtained. The first matrix X and the second matrix L are respectively:
Figure BDA0002192360880000081
and 103, calculating a third matrix according to the first matrix.
Wherein, formula A is XXTAnd calculating a third matrix, wherein A is the third matrix.
Specifically, matrix multiplication can be rapidly completed through a matrix calculation unit in the FPGA and the advantage of parallel data processing of the FPGA, so that a third matrix is obtained.
Referring to fig. 4, the first number 7 continues to be illustrated. PE1-PE9 are all matrix calculation units, each comprising a multiplier MUX, an adder ADD and a register REG, the output of the multiplier being connected to a first input of the adder, the output of the adder being connected to an input of the register, the output of the register being connected to a second input of the adder. The multiplier multiplies the two input variables, and the output result is cached through a register and is used as the input variable of the next addition operation.
Because A is a symmetric matrix, only the upper triangular element is needed to calculate the matrix A. A. the33The result was always 7, A02And A11The calculation results are the same, so the PEs to be instantiated in the FPGA matrix a include PE1, PE2, PE3 and PE 6. PE1 calculates the result first
Figure BDA0002192360880000082
Multiplication by
Figure BDA0002192360880000083
One cycle is needed, 7 cycles are needed for further calculation of x0-x6, one cycle is needed for addition, and 8 cycles are needed for calculation of PE1. The result calculated by PE6 takes the longest time, and it takes 11 cycles to add x0 and x1 to PE6 on the basis of 8 cycles.
Referring to fig. 5, when calculating the third matrix a, the work flow of the matrix calculating unit is as follows:
at the first clock, the multiplier output of the first matrix computation element PE1
Figure BDA0002192360880000084
Input device
Figure BDA0002192360880000085
And
Figure BDA0002192360880000086
multiplier input of the second matrix computation element PE2
Figure BDA0002192360880000087
And x0；
At the second clock, the multiplier output of the first matrix computation element PE1
Figure BDA0002192360880000088
Input device
Figure BDA0002192360880000089
And
Figure BDA00021923608800000810
adder output
Figure BDA0002192360880000091
Multiplier output of the second matrix computation element PE2
Figure BDA0002192360880000092
Input device
Figure BDA0002192360880000093
And x1(ii) a Multiplier input of the third matrix computation element PE3
Figure BDA0002192360880000094
And 1;
at the third clock, the first matrix computation element PE1 multiplier outputs
Figure BDA0002192360880000095
Input the method
Figure BDA0002192360880000096
And
Figure BDA0002192360880000097
adder output
Figure BDA0002192360880000098
Second matrix computation element PE2 multiplier output
Figure BDA0002192360880000099
Input device
Figure BDA00021923608800000910
And x2The adder output
Figure BDA00021923608800000911
Multiplier output of the third matrix computation element PE3
Figure BDA00021923608800000912
Input device
Figure BDA00021923608800000913
And 1; multiplier input x of the sixth matrix computation element PE60And 1;
at the fourth clock, the multiplier output of the first matrix computation element PE1
Figure BDA00021923608800000914
Input device
Figure BDA00021923608800000915
And
Figure BDA00021923608800000916
adder output
Figure BDA00021923608800000917
Second matrix computation element PE2 multiplier output
Figure BDA00021923608800000918
Input device
Figure BDA00021923608800000919
And x3The adder output
Figure BDA00021923608800000920
Multiplier output of the third matrix computation element PE3
Figure BDA00021923608800000921
Input device
Figure BDA00021923608800000922
And 1, adder output
Figure BDA00021923608800000923
Multiplier output x of sixth matrix computation element PE60Inputting x1And 1;
at the fifth clock, the multiplier output of the first matrix computation element PE1
Figure BDA00021923608800000924
Input device
Figure BDA00021923608800000925
And
Figure BDA00021923608800000926
adder output
Figure BDA00021923608800000927
Second matrix computation element PE2 multiplier output
Figure BDA00021923608800000928
Input device
Figure BDA00021923608800000929
And x4The adder output
Figure BDA00021923608800000930
Multiplier output of the third matrix computation element PE3
Figure BDA00021923608800000931
Input the method
Figure BDA00021923608800000932
And 1, adder output
Figure BDA00021923608800000933
Multiplier output x of sixth matrix computation element PE61Inputting x2And 1, adder output x0；
At the sixth clock, the multiplier output of the first matrix computation element PE1
Figure BDA00021923608800000934
Input the method
Figure BDA00021923608800000935
And
Figure BDA00021923608800000936
adder output
Figure BDA00021923608800000937
Second matrix computation element PE2 multiplier output
Figure BDA00021923608800000938
Input device
Figure BDA00021923608800000939
And x5The adder output
Figure BDA00021923608800000940
Multiplier output of the third matrix computation element PE3
Figure BDA00021923608800000941
Input device
Figure BDA00021923608800000942
And 1, adder output
Figure BDA00021923608800000943
Multiplier output x of sixth matrix computation element PE62Inputting x3And 1, adder output x0+x1；
At the seventh clock, the multiplier output of the first matrix computation element PE1
Figure BDA00021923608800000944
Adder output
Figure BDA00021923608800000945
Second matrix computation element PE2 multiplier output
Figure BDA00021923608800000946
Input device
Figure BDA00021923608800000947
And x6The adder output
Figure BDA00021923608800000948
Multiplier output of the third matrix computation element PE3
Figure BDA00021923608800000949
Input device
Figure BDA00021923608800000950
And 1, adder output
Figure BDA0002192360880000101
Multiplier output x of sixth matrix computation element PE63Inputting x4And 1, adder output x0+x1+x2；
At the eighth clock, the first matrix computation element PE1 adder outputs
Figure BDA0002192360880000102
Second matrix computation element PE2 multiplier output
Figure BDA0002192360880000103
Adder output
Figure BDA0002192360880000104
Multiplier output of the third matrix computation element PE3
Figure BDA0002192360880000105
Input device
Figure BDA0002192360880000106
And 1, adder output
Figure BDA0002192360880000107
Multiplier output x of sixth matrix computation element PE64Inputting x5And 1, adder output x0+x1+x2+x3；
At the ninth clock, the first matrix computation element PE1 adder outputs
Figure BDA0002192360880000108
Second matrix computation element PE2 adder output
Figure BDA0002192360880000109
Figure BDA00021923608800001010
Multiplier output of the third matrix computation element PE3
Figure BDA00021923608800001011
Adder output
Figure BDA00021923608800001012
Figure BDA00021923608800001013
Multiplier output x of sixth matrix computation element PE65Inputting x6And 1, adder output x0 +x1+x2+x3+x4；
At the tenth clock, the first matrix computation element PE1 adder outputs
Figure BDA00021923608800001014
Second matrix computation element PE2 adder output
Figure BDA00021923608800001015
Figure BDA00021923608800001016
Third matrix computation element PE3 adder output
Figure BDA00021923608800001017
Multiplier output x of sixth matrix computation element PE66Adder output x0+x1+x2+x3+x4+x5；
At the eleventh clock, the first matrix computation element PE1 adder outputs
Figure BDA00021923608800001018
Second matrix computation element PE2 adder output
Figure BDA00021923608800001019
Figure BDA00021923608800001020
Third matrix computation element PE3 adder output
Figure BDA00021923608800001021
Sixth matrix computation element PE6 adder output x0+x1+x2+x3+x4+x5+x6；
Data flow from top to bottom and from left to right simultaneously according to the above mode, and the FPGA can complete the calculation of the matrix A only by 11 clock cycles.
And 104, calculating the adjoint matrix of the third matrix according to the third matrix.
Wherein, from the third matrix a, a companion matrix B of the third matrix a is calculated, i.e. B ═ a*。
The calculation process of the adjoint matrix B of the matrix a is shown in fig. 6, the upper triangular element and the lower triangular element are equal or opposite to each other, so that the upper triangular element is only required to be calculated, PE1, PE2, PE3, PE5, PE6 and PE9 are calculated simultaneously, taking PE1 as an example, a cycle is required for multiplying a11 by a12, a cycle is required for multiplying a12 by a12, and a cycle is required for subtracting the previous result, which is three cycles in total. Therefore, when calculating the matrix B, 6 matrix calculation units are required, and 3 clock cycles are required for calculation.
In calculating the adjoint matrix B of the third matrix, taking PE1 calculation as an example:
at the first clock, the multiplier outputs A11And A22Product of, input A12And A12；
At the second clock, the multiplier outputs A12And A12The product of (a); adder output A11xA22；
At the third clock, the multiplier outputs A12And A12The product of (a); adder output A11xA22-A12x A12；
PE2, PE3, PE5, PE6, and PE9 were calculated in the same manner.
Step 105, calculating a flag bit according to the first column of the third matrix and the first row of the adjoint matrix of the third matrix.
Specifically, the first column of the third matrix a and the first row of the matrix B are multiplied to obtain a flag, that is:
Figure BDA0002192360880000111
and 106, when the zone bit is not 0, calculating a fourth matrix according to the second matrix, the third matrix and the adjoint matrix of the third matrix.
Specifically, if the flag bit flag is 0, the current line is invalid, and the data calculation of the next line is continued. If the flag is not 0, calculating a fourth matrix C, and calculating the fourth matrix C by using the adjoint matrix B, the second matrix L and the third matrix A of the third matrix A, wherein the calculation formula is as follows:
Figure BDA0002192360880000112
and 107, calculating the first coordinate of the center point of the current row laser line according to the fourth matrix.
In particular, by
Figure BDA0002192360880000121
Calculating a first coordinate of a center point of a current line of laser lines, wherein,
Figure BDA0002192360880000122
the abscissa of the center point of the laser line of the current line, i.e. the symmetry axis of the parabola shown in fig. 2, and the ordinate of the center point of the laser line of the current line is the line number of the current line.
And step 108, calculating the central line of the laser stripe according to the first coordinates of the central points of the laser lines of all the rows.
Specifically, step 102 and step 107 are continuously executed until the first coordinates of the laser line center points of all the lines are calculated, and then the calculated coordinates of the laser line center points are fitted according to the first coordinates of the laser line center points of each line and the corresponding line numbers, so as to obtain the center line of the laser stripe. Wherein, all lines are effective lines, and the line number of each line is the second coordinate, i.e. the ordinate, of the laser line center point of the line.
It will be appreciated that various processing is also required in performing the fitting to ensure that the fitted centerline is a straight line. The specific steps of the processing performed herein are not described in detail in this application.
By applying the FPGA-based laser stripe center line fitting method provided by the embodiment of the invention, the FPGA is used for acquiring the camera image, after the preprocessing, the advantage that the center line of the laser stripe is calculated by the FPGA for processing data in parallel is realized, the pipeline structure is adopted for simultaneous processing, the real-time performance is good, and the parabolic symmetry axis fitting method is adopted, so that the accuracy is high.
The second embodiment of the invention provides equipment which comprises a memory and a processor, wherein the memory is used for storing programs, and the memory can be connected with the processor through a bus. The memory may be a non-volatile memory, such as a hard disk drive and a flash memory, in which a software program and a device driver are stored. The software program is capable of performing various functions of the above-described methods provided by embodiments of the present invention; the device drivers may be network and interface drivers. The processor is used for executing a software program, and the software program can realize the method provided by the first embodiment of the invention when being executed.
A third embodiment of the present invention provides a computer program product including instructions, which, when the computer program product runs on a computer, causes the computer to execute the method provided in the first embodiment of the present invention.
The fourth embodiment of the present invention provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the method provided in the first embodiment of the present invention is implemented.
Those of skill would further appreciate that the various illustrative components and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied in hardware, a software module executed by a processor, or a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The above embodiments are provided to further explain the objects, technical solutions and advantages of the present invention in detail, it should be understood that the above embodiments are merely exemplary embodiments of the present invention and are not intended to limit the scope of the present invention, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.

Claims (9)

1. A laser stripe center line fitting method based on a Field Programmable Gate Array (FPGA) is characterized by comprising the following steps of:
collecting a laser line image shot by a camera through an FPGA;
acquiring the abscissa of a first number of pixel points in the laser line image in a camera coordinate system to generate a first matrix, acquiring the brightness of the first number of pixel points to generate a second matrix; the first number is a positive number not less than 2;
calculating a third matrix according to the transposed matrix of the first matrix and the cross multiplication result of the first matrix;
calculating a adjoint matrix of the third matrix according to the third matrix;
performing dot multiplication according to the first column of the third matrix and the first row of the adjoint matrix of the third matrix to calculate a flag bit; the flag bit comprises a non-0;
when the zone bit is not 0, calculating a fourth matrix according to the second matrix, the third matrix and the adjoint matrix of the third matrix;
calculating the abscissa of the first coordinate of the center point of the current line laser line according to the first item and the second item in the fourth matrix, wherein the ordinate of the center point of the current line laser line is the line number of the current line;
calculating the central line of the laser stripe according to the first coordinate of the central point of the laser lines of all the rows;
matrix multiplication is carried out through matrix calculation units in the FPGA, each matrix calculation unit comprises a multiplier, an adder and a register, the output end of the multiplier is connected with the first input end of the adder, the output end of the adder is connected with the input end of the register, and the output end of the register is connected with the second input end of the adder;
the multiplier performs multiplication operation on the input variable, and the output result is cached through the register and is used as the input variable of the next addition operation.
2. The method of claim 1, wherein said obtaining a first number of pixel points in the laser line image precedes an abscissa of a camera coordinate system, further comprising:
and when the gray value of the pixel point in the laser line image is greater than a preset threshold value, determining that the laser line is detected, and taking the pixel point as one pixel point in a first number of pixel points.
3. The method of claim 1, wherein when the first number is 7, the first matrix is generated as:
Figure FDA0003534438060000021
wherein x is0-x6The abscissa of 7 pixels with pixels larger than a preset threshold in a camera coordinate system.
4. The method of claim 3, wherein when the first number is 7, the second matrix is generated as:
Figure FDA0003534438060000022
wherein l0-l6In sequence with x0-x6Correspondingly, the brightness value is the brightness value of the first number of pixel points.
5. The method according to claim 1, wherein the calculating a third matrix according to the transposed matrix of the first matrix and the cross-product result of the first matrix specifically comprises:
using the formula A-XXTCalculating a third matrix;
wherein A is the third matrix, X is the first matrix, XTIs the transpose of the first matrix.
6. The method of claim 5, wherein the FPGA comprises a plurality of matrix computing units, and wherein the first number is 7, the using formula A is XXTAnd calculating a third matrix, specifically comprising:
at the first clock, the multiplier output of the first matrix computation element PE1
Figure FDA0003534438060000023
Input device
Figure FDA0003534438060000024
And
Figure FDA0003534438060000025
multiplier input of the second matrix computation element PE2
Figure FDA0003534438060000026
And x0；
At the second clock, the multiplier output of the first matrix computation element PE1
Figure FDA0003534438060000027
Input device
Figure FDA0003534438060000028
And
Figure FDA0003534438060000029
adder output
Figure FDA00035344380600000210
Multiplier output of the second matrix computation element PE2
Figure FDA00035344380600000211
Input device
Figure FDA00035344380600000212
And x1(ii) a Multiplier input of the third matrix computation element PE3
Figure FDA0003534438060000031
And 1;
at the third clock, the first matrix computation element PE1 multiplier outputs
Figure FDA0003534438060000032
Input device
Figure FDA0003534438060000033
And
Figure FDA0003534438060000034
adder output
Figure FDA0003534438060000035
Second matrix computing element PE2 multiplier output
Figure FDA0003534438060000036
Input the method
Figure FDA0003534438060000037
And x2The adder output
Figure FDA0003534438060000038
Multiplier output of the third matrix computation element PE3
Figure FDA0003534438060000039
Input device
Figure FDA00035344380600000310
And 1; multiplier input x of the sixth matrix computation element PE60And 1;
at the fourth clock, the multiplier output of the first matrix computation element PE1
Figure FDA00035344380600000311
Input device
Figure FDA00035344380600000312
And
Figure FDA00035344380600000313
adder output
Figure FDA00035344380600000314
Second matrix computation element PE2 multiplier output
Figure FDA00035344380600000315
Input device
Figure FDA00035344380600000316
And x3The adder output
Figure FDA00035344380600000317
Multiplier output of the third matrix computation element PE3
Figure FDA00035344380600000318
Input device
Figure FDA00035344380600000319
And 1, adder output
Figure FDA00035344380600000320
Multiplier output x of sixth matrix computation element PE60Inputting x1And 1;
at the fifth clock, the multiplier output of the first matrix computation element PE1
Figure FDA00035344380600000321
Input device
Figure FDA00035344380600000322
And
Figure FDA00035344380600000323
adder output
Figure FDA00035344380600000324
Second matrix computation element PE2 multiplier output
Figure FDA00035344380600000325
Input device
Figure FDA00035344380600000326
And x4The adder output
Figure FDA00035344380600000327
Multiplier output of the third matrix computation element PE3
Figure FDA00035344380600000328
Input device
Figure FDA00035344380600000329
And 1, adder output
Figure FDA00035344380600000348
Multiplier output x of sixth matrix computation element PE61Inputting x2And 1, adder output x0；
At the sixth clock, the multiplier output of the first matrix computation element PE1
Figure FDA00035344380600000330
Input device
Figure FDA00035344380600000331
And
Figure FDA00035344380600000332
adder output
Figure FDA00035344380600000333
Second matrix computation element PE2 multiplier output
Figure FDA00035344380600000334
Input device
Figure FDA00035344380600000335
And x5The adder output
Figure FDA00035344380600000336
Multiplier output of the third matrix computation element PE3
Figure FDA00035344380600000337
Input device
Figure FDA00035344380600000338
And 1, adder output
Figure FDA00035344380600000339
Sixth matrix calculation element PE6Is output x of the multiplier2Inputting x3And 1, adder output x0+x1；
At the seventh clock, the multiplier output of the first matrix computation element PE1
Figure FDA00035344380600000340
Adder output
Figure FDA00035344380600000341
Second matrix computing element PE2 multiplier output
Figure FDA00035344380600000342
Input device
Figure FDA00035344380600000343
And x6The adder output
Figure FDA00035344380600000344
Multiplier output of the third matrix computation element PE3
Figure FDA00035344380600000345
Input device
Figure FDA00035344380600000346
And 1, adder output
Figure FDA00035344380600000347
Multiplier output x of sixth matrix computation element PE63Inputting x4And 1, adder output x0+x1+x2；
At the eighth clock, the first matrix computation element PE1 adder outputs
Figure FDA0003534438060000041
Second matrix computation element PE2 multiplier output
Figure FDA0003534438060000042
Adder output
Figure FDA0003534438060000043
Multiplier output of the third matrix computation element PE3
Figure FDA0003534438060000044
Input device
Figure FDA0003534438060000045
And 1, adder output
Figure FDA0003534438060000046
Multiplier output x of sixth matrix computation element PE64Inputting x5And 1, adder output x0+x1+x2+x3；
At the ninth clock, the first matrix computation element PE1 adder outputs
Figure FDA0003534438060000047
Second matrix computation element PE2 adder output
Figure FDA0003534438060000048
Figure FDA0003534438060000049
Multiplier output of the third matrix computation element PE3
Figure FDA00035344380600000410
Adder output
Figure FDA00035344380600000411
Figure FDA00035344380600000412
Multiplier output x of sixth matrix computation element PE65Inputting x6And 1, adder output x0+x1+x2+x3+x4；
At the tenth clock, the first matrix computation element PE1 adder outputs
Figure FDA00035344380600000421
Second matrix computation element PE2 adder output
Figure FDA00035344380600000413
Figure FDA00035344380600000414
Third matrix computation element PE3 adder output
Figure FDA00035344380600000415
Multiplier output x of sixth matrix computation element PE66Adder output x0+x1+x2+x3+x4+x5；
At the eleventh clock, the first matrix computation element PE1 adder outputs
Figure FDA00035344380600000416
Second matrix computation element PE2 adder output
Figure FDA00035344380600000417
Figure FDA00035344380600000418
Third matrix computation element PE3 adder output
Figure FDA00035344380600000419
Sixth matrix computation element PE6 adder output x0+x1+x2+x3+x4+x5+x6。
7. The method according to claim 1, wherein the calculating a flag bit according to the first column of the third matrix and the first row of the companion matrix of the third matrix specifically comprises:
using a formula
Figure FDA00035344380600000420
Calculating a flag bit;
wherein (A)11 A21 A31) Is the first column of the third matrix and,
Figure FDA0003534438060000051
the first row of the companion matrix of the third matrix.
8. The method according to claim 1, wherein when the flag bit is not 0, calculating a fourth matrix according to the second matrix, the third matrix, and a companion matrix of the third matrix, specifically includes:
using formulas
Figure FDA0003534438060000052
Calculating a fourth matrix;
wherein L is the second matrix, A is the third matrix, and B is the adjoint matrix of the third matrix.
9. The method of claim 1, wherein calculating the abscissa of the first coordinate of the center point of the current line laser line according to the first term and the second term in the fourth matrix comprises:
by using
Figure FDA0003534438060000053
Calculating the abscissa of a first coordinate of the center point of the laser line of the current line;
wherein, C0And C1The first and second entries in the fourth matrix.
CN201910836578.3A 2019-09-05 2019-09-05 Laser stripe center line fitting method based on FPGA Active CN110648313B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910836578.3A CN110648313B (en) 2019-09-05 2019-09-05 Laser stripe center line fitting method based on FPGA

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910836578.3A CN110648313B (en) 2019-09-05 2019-09-05 Laser stripe center line fitting method based on FPGA

Publications (2)

Publication Number Publication Date
CN110648313A CN110648313A (en) 2020-01-03
CN110648313B true CN110648313B (en) 2022-05-24

Family

ID=69010359

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910836578.3A Active CN110648313B (en) 2019-09-05 2019-09-05 Laser stripe center line fitting method based on FPGA

Country Status (1)

Country Link
CN (1) CN110648313B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112800582B (en) * 2020-12-30 2022-05-24 华南理工大学 Method for generating simulated laser line of structured light vision sensor

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109284475A (en) * 2018-09-20 2019-01-29 郑州云海信息技术有限公司 A kind of matrix convolution computing module and matrix convolution calculation method

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105589677A (en) * 2014-11-17 2016-05-18 沈阳高精数控智能技术股份有限公司 Systolic structure matrix multiplier based on FPGA (Field Programmable Gate Array) and implementation method thereof
US10488483B2 (en) * 2016-06-22 2019-11-26 Comsats Institute Of Information Technology Parameterized FPGA implementation of a real-time sensitivity encoding reconstruction

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109284475A (en) * 2018-09-20 2019-01-29 郑州云海信息技术有限公司 A kind of matrix convolution computing module and matrix convolution calculation method

Also Published As

Publication number Publication date
CN110648313A (en) 2020-01-03

Similar Documents

Publication Publication Date Title
JP7346510B2 (en) Vector calculation unit in neural network processor
US20230305808A1 (en) Accelerated mathematical engine
US20210357735A1 (en) Split accumulator for convolutional neural network accelerator
WO2019152451A1 (en) Improving performance of neural network arrays
Liu et al. Towards an efficient accelerator for DNN-based remote sensing image segmentation on FPGAs
Amaricai et al. An FPGA sliding window-based architecture Harris corner detector
CN109313663B (en) Artificial intelligence calculation auxiliary processing device, method, storage medium and terminal
CN115994561B (en) Convolutional neural network acceleration method, system, storage medium, device and equipment
CN114092336B (en) Image scaling method, device, equipment and medium based on bilinear interpolation algorithm
US9058541B2 (en) Object detection method, object detector and object detection computer program
CN110648313B (en) Laser stripe center line fitting method based on FPGA
Villalpando et al. FPGA implementation of stereo disparity with high throughput for mobility applications
Bai et al. Pointnet on fpga for real-time lidar point cloud processing
CN113313252A (en) Depth separable convolution implementation method based on pulse array
CN109800793B (en) Target detection method and system based on deep learning
Ustukov et al. Modification of Retinex algorithm and its stream implementation on FPGA
Sada et al. A dataflow pipelining architecture for tile segmentation with a sparse MobileNet on an FPGA
CN111008697B (en) Convolutional neural network accelerator implementation architecture
CN104317352B (en) A kind of adaptive optics control system quickly goes tilt component processing method
Devendran et al. Optimization of the Convolution Operation to Accelerate Deep Neural Networks in FPGA.
CN110930290A (en) Data processing method and device
Mazza et al. A comparison of hardware/software techniques in the speedup of color image processing algorithms
Di Carlo et al. Safe: a self adaptive frame enhancer fpga-based ip-core for real-time space applications
Kai et al. A Winograd-based CNN Accelerator for Cloud Detection on FPGA
CN114022689A (en) Similarity calculation method and system based on FPGA

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CP01 Change in the name or title of a patent holder

Address after: B4-006, maker Plaza, 338 East Street, Huilongguan town, Changping District, Beijing 100096

Patentee after: Beijing Idriverplus Technology Co.,Ltd.

Address before: B4-006, maker Plaza, 338 East Street, Huilongguan town, Changping District, Beijing 100096

Patentee before: Beijing Idriverplus Technology Co.,Ltd.

CP01 Change in the name or title of a patent holder