CN114049323B - Real-time deformation measuring method for van vehicle based on binocular vision - Google Patents
Real-time deformation measuring method for van vehicle based on binocular vision Download PDFInfo
- Publication number
- CN114049323B CN114049323B CN202111344166.1A CN202111344166A CN114049323B CN 114049323 B CN114049323 B CN 114049323B CN 202111344166 A CN202111344166 A CN 202111344166A CN 114049323 B CN114049323 B CN 114049323B
- Authority
- CN
- China
- Prior art keywords
- deformation
- carriage
- van
- image
- images
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims description 28
- 238000012545 processing Methods 0.000 claims abstract description 35
- 238000004364 calculation method Methods 0.000 claims abstract description 13
- 238000005259 measurement Methods 0.000 claims abstract description 13
- 238000000691 measurement method Methods 0.000 claims abstract description 8
- 230000003287 optical effect Effects 0.000 claims description 18
- 238000013507 mapping Methods 0.000 claims description 12
- 230000009466 transformation Effects 0.000 claims description 9
- 239000011159 matrix material Substances 0.000 claims description 6
- 230000008569 process Effects 0.000 claims description 6
- 230000008859 change Effects 0.000 claims description 4
- 238000012937 correction Methods 0.000 claims description 3
- 238000009419 refurbishment Methods 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 claims description 2
- 238000012216 screening Methods 0.000 claims description 2
- 238000001514 detection method Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 6
- 206010039203 Road traffic accident Diseases 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06T7/85—Stereo camera calibration
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T90/00—Enabling technologies or technologies with a potential or indirect contribution to GHG emissions mitigation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Quality & Reliability (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Length Measuring Devices By Optical Means (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a binocular vision-based box type vehicle deformation real-time measurement method, which is characterized in that the deformation degree of a carriage is represented in a numerical form, and whether the carriage needs to be refurbished or not is judged. A binocular vision-based box-type vehicle deformation real-time measurement system comprises: the device comprises an image acquisition module, an image processing module and a deformation calculation module. The image acquisition module consists of a camera array and is used for acquiring box-type car images; the image processing module consists of data processing equipment and is used for receiving the image acquired by the camera and processing the image; the deformation calculation module is used for comparing and calculating whether the carriage of the box-type vehicle deforms and the deformation degree according to the three extracted images of the carriage and the images before deformation, and judging whether the vehicle needs to be refurbished. The invention solves the problem that whether the carriage is deformed or not and the deformation degree detection are difficult to judge, is convenient to detect the carriage at any time, and has the advantages in the aspects of cost, operability, accuracy and the like.
Description
Technical Field
The invention relates to machine vision measurement of deformation of a van vehicle, in particular to a real-time measurement method and a real-time measurement system of deformation of the van vehicle based on binocular vision, and belongs to the fields of machine vision image processing and van vehicle safety.
Background
With the development of the current society, natural disasters, artificial accidents and the like frequently occur, and fire rescue becomes an indispensable part of life, particularly rescue, such as transporting people and objects to safe places by fire trucks on road sections such as landslides and the like; after a major traffic accident occurs, the deformed vehicle needs to be detached by a fire truck so as to be convenient for the personnel in the vehicle to safely separate; when a fire disaster occurs in a high-rise building, a fire truck is required to extinguish the fire when traveling. But long-term emergent travel, uneven and tortuous driving pavement and the like can lead to unrecoverable deformation of the fire truck body, and if the fire truck is not inspected and trimmed in time, the service life of the fire truck can be shortened, the stability of the fire truck is poor, the operability of the fire truck is reduced, the travel speed of the fire truck is influenced, and even traffic accidents can be caused, so that the driving risk is greatly increased.
The carriage is an important component part of the fire engine, and rescue equipment is arranged in the carriage. The structure of the carriage is normal, so that the fire truck can work normally, if the carriage is deformed greatly, the gravity center is deviated, the car body is inclined, the stability in the running process can be influenced finally, and the driving safety is endangered. Therefore, the research on the deformation and the deformation degree of the carriage of the fire truck has important significance for the safety of the fire truck.
Disclosure of Invention
The invention aims to provide a real-time deformation measuring method and system for a van vehicle based on binocular vision, which solve the problem that whether the deformation of a carriage and the detection of deformation degree are difficult to judge, can represent the deformation degree of the carriage in a numerical form, and judge whether the carriage needs to be trimmed. The real-time deformation measuring method and system for the van vehicle based on binocular vision are convenient to detect the van vehicle at any time and have the advantages of being low in cost, simple to operate, high in accuracy and the like.
In order to achieve the above purpose, the technical scheme adopted by the invention is as follows: the real-time van deformation measuring method based on binocular vision is characterized by comprising the following steps of:
Simultaneously acquiring images of the van vehicle from different positions through an image acquisition module, and transmitting the images to an image processing module; the image acquisition module consists of a camera array, the camera array consists of four cameras, the four cameras are respectively positioned on four vertexes of a rectangular area on the same horizontal plane above the parking space, the size of the rectangular area depends on the volume of the van, the acquired information is ensured to be enough to recover the three-dimensional perspective view of the van, and every two cameras positioned on the adjacent vertexes form a pair of binocular cameras;
The method comprises the steps of receiving images acquired by a camera through an image processing module, processing the images to obtain three-dimensional images of the van, extracting left view, right view and top view of a van carriage from the three-dimensional images, and recovering images of three views of the van carriage before deformation of the van carriage; the method specifically comprises the following steps: receiving an image acquired by a camera, processing the image by adopting a stereo matching algorithm based on characteristics, and obtaining coordinates of all points on the surface of a space object by a parallax principle calculation method so as to reconstruct the surface shape of the three-dimensional object and the position of the object in space; recovering three-dimensional images of the van vehicle through four groups of binocular cameras, and extracting left view, right view and top view of the van vehicle through an edge recognition algorithm; the relative positions of the four vertexes of the carriage after deformation basically remain unchanged, so that the images of the three views of the carriage before deformation of the van can be recovered according to the extracted three views of the carriage;
Calculating whether the carriage of the van vehicle deforms and the deformation degree according to the three extracted images of the carriage and the three images before deformation, and judging whether the vehicle needs to be refurbished; the method specifically comprises the following steps: obtaining deformation of each side by superposition comparison, calculating the maximum deformation rate of each side, and further calculating the deformation rate of the car; and judging whether the van vehicle needs to be refurbished according to whether the van deformation rate is larger than the allowable maximum deformation rate.
Further, the method for processing the image by adopting the stereo matching algorithm based on the characteristics comprises the following steps: aiming at each group of binocular cameras, a stereo matching algorithm based on characteristics is adopted to process the images, characteristic points in the two images, including corners, straight lines, circular arcs, free curves with obvious gray level changes, zero crossings of the images and image edge characteristics, are extracted, and then the characteristic points are screened and the matching corresponding relation of the characteristics between the two images is established.
Further, the specific steps of the feature-based stereo matching algorithm are as follows:
(1) Preprocessing the image to extract characteristic points or lines;
(2) For a certain feature in the left image, calculating the similarity between each feature near the feature scanning line in the right image and the feature by using a similarity measurement formula;
(3) Selecting the feature with the greatest similarity as the matching feature of the feature in the left image;
(4) Repeating the steps until all the features in the left image are matched.
Further, for each group of double-sided cameras, coordinates of all points on the surface of the space object are obtained through the inverse process of the parallax principle, so that the surface shape of the three-dimensional object and the position of the object in space are reconstructed, and the specific steps are as follows;
Obtaining an internal parameter of a camera and an external parameter between binocular cameras through camera calibration, wherein parallax change only exists in the horizontal direction and parallax change does not exist in the vertical direction when images subjected to binocular correction are matched due to limit constraint of stereo matching; taking the left camera coordinate system as the world coordinate system, assuming that the parameters of the left camera and the right camera are the same and the mapping point of the space point P (X W,YW,ZW) opposite to the left image and the right image is P l(xl,yl),Pr(xr,yr), wherein X l-xr＝d,yl＝yr is known as the distance between the focal length f of the camera and the optical center of the binocular camera, namely the base line length B, and taking the optical center of the left camera as the origin of the world coordinate system, the following relationship is obtained:
Wherein (x o,yo) represents the mapping point of the optical center of the left camera on the image plane, and the formula (1) is normalized to obtain the mapping relation between the two-dimensional plane coordinates and the three-dimensional space point coordinates:
Wherein the transformation matrix Q is the transformation under a linear binocular camera model:
In practical situations, factors affecting three-dimensional reconstruction are mainly the deviation of the horizontal coordinates of the mapping coordinate points of the optical centers of the left and right cameras in the image, so that the parallax d needs to be compensated by the deviation of the optical centers of the two cameras, and the compensated transformation matrix Q:
Wherein x' o is the abscissa value of the right camera optical center map on the image;
Obtaining coordinates of each pixel point in an actual space through the obtained binocular stereo matching dense parallax map to form a three-dimensional reconstruction point cloud; these actual spatial points are smoothly connected to form a three-dimensional reconstruction model.
Further, calculating the deformation of the carriage according to the data of the three-dimensional reconstruction model, and judging whether the carriage needs to be refurbished or not, specifically comprising:
The upper, lower, left and right sides of the left view are respectively marked as 1 st, 2 nd, 3 rd and 4 th sides, the upper, lower, left and right sides of the right view are respectively marked as 5 th, 6 th, 7 th and 8 th sides, the upper, lower, left and right sides of the right view are respectively marked as9 th, 10 th, 11 th and 12 th sides, the three views of the extracted carriage and the view of the carriage before deformation are subjected to superposition comparison to obtain deformation amounts of the respective sides of the carriage, the deformation amount of the i th side is marked as Deltax i, wherein i=1, 2..12, and the maximum deformation rate of the respective sides of the carriage is calculated as follows:
wherein, L i is the length of the ith side before deformation, so the deformation rate of the carriage is:
Δκ＝max{Δα1,Δα2,...Δα12}，
if the carriage deformation rate is greater than the allowable maximum deformation rate ψ, namely delta kappa > ψ, the van vehicle needs to be refurbished; otherwise, the van does not require refurbishment.
The invention also provides a real-time deformation measuring system of the van vehicle based on binocular vision, which is characterized by comprising the following steps:
The image acquisition module is used for simultaneously acquiring images of the van vehicle from different positions and transmitting the images to the image processing module; the image acquisition module consists of a camera array, the camera array consists of four cameras, the four cameras are respectively positioned on four vertexes of a rectangular area on the same horizontal plane above the parking space, the size of the rectangular area depends on the volume of the van, the acquired information is ensured to be enough to recover the three-dimensional perspective view of the van, and every two cameras positioned on the adjacent vertexes form a pair of binocular cameras;
The image processing module receives the image acquired by the camera, processes the image to obtain a three-dimensional stereoscopic image of the van, extracts a left view, a right view and a top view of a carriage of the van, and restores the images of the three views of the carriage before deformation of the van; the method specifically comprises the following steps: receiving an image acquired by a camera, processing the image by adopting a stereo matching algorithm based on characteristics, and obtaining coordinates of all points on the surface of a space object by a parallax principle calculation method so as to reconstruct the surface shape of the three-dimensional object and the position of the object in space; recovering three-dimensional images of the van vehicle through four groups of binocular cameras, and extracting left view, right view and top view of the van vehicle through an edge recognition algorithm; the relative positions of the four vertexes of the carriage after deformation basically remain unchanged, so that the images of the three views of the carriage before deformation of the van can be recovered according to the extracted three views of the carriage;
The deformation calculation module calculates whether the carriage of the van vehicle deforms and the deformation degree according to the three extracted images of the carriage and the three images before deformation, and judges whether the vehicle needs to be refurbished; the method specifically comprises the following steps: obtaining deformation of each side by superposition comparison, calculating the maximum deformation rate of each side, and further calculating the deformation rate of the car; and judging whether the van vehicle needs to be refurbished according to whether the van deformation rate is larger than the allowable maximum deformation rate.
The real-time deformation measuring method and system for the van vehicle based on binocular vision, disclosed by the invention, have the following advantages:
(1) The deformation degree of the carriage of the van is calculated, and the deformation is converted into an intuitive numerical value of the maximum deformation rate of the side length of the carriage;
(2) The equipment required by measurement is simple, and the requirement on the precision of the equipment is not high, so that the cost is reduced;
(3) The invention solves the problem of measuring the deformation of the van vehicle and has the advantages of practicality, accuracy, convenience and the like.
Drawings
The invention will be described in further detail with reference to the drawings and the detailed description.
Fig. 1 is a flow chart of a real-time deformation measuring method of a van based on binocular vision.
Fig. 2 is a schematic overall layout of the real-time deformation measuring method of the van based on binocular vision.
Fig. 3 is a block diagram of a feature-based stereo matching algorithm of the present invention.
Fig. 4 is a schematic view of the parallax principle coordinate system of the present invention.
Fig. 5 is a block diagram of the real-time deformation measuring system of the van based on binocular vision.
Fig. 6 is a schematic diagram of a camera array of a real-time measurement method for deformation of a van based on binocular vision according to an embodiment of the present invention.
Fig. 7 is a left view deformation comparison chart of a real-time van vehicle deformation measurement method based on binocular vision according to an embodiment of the invention.
Fig. 8 is a right view deformation comparison chart of the real-time van vehicle deformation measurement method based on binocular vision according to the embodiment of the invention.
Fig. 9 is a top view deformation comparison chart of a real-time van vehicle deformation measurement method based on binocular vision according to an embodiment of the invention.
Detailed Description
Other advantages and effects of the present invention will become apparent to those skilled in the art from the following disclosure, which describes the embodiments of the present invention with reference to specific examples. The invention may be practiced or carried out in other embodiments that depart from the specific details, and the details of the present description may be modified or varied from the spirit and scope of the present invention.
The invention discloses a binocular vision-based real-time deformation measuring method and system for a van vehicle. In order to make the technical scheme of the invention clearer, the invention is further described in detail below with reference to the attached drawings.
The utility model provides a real-time measurement method and system of van deformation based on binocular vision for measure the deformation of van carriage, the real-time measurement system of van deformation based on binocular vision includes: the device comprises an image acquisition module, an image processing module and a deformation calculation module.
The image acquisition module consists of a camera array and is used for acquiring images of the van vehicle from different positions and transmitting the images to the image processing module.
The camera array is composed of four cameras and is respectively positioned on four vertexes of a rectangular area of the same horizontal plane above the parking space, the size of the rectangular area depends on the volume of the van, collected information is ensured to be enough to recover a three-dimensional perspective view of the van, and every two cameras positioned on adjacent vertexes form a pair of binocular cameras.
The image processing module is composed of data processing equipment and is used for receiving images acquired by the camera and processing the images to obtain three-dimensional images of the van, extracting left view, right view and top view of the van carriage from the three-dimensional images, and recovering images of three views of the van carriage before deformation of the van carriage.
The deformation calculation module is used for calculating whether the carriage of the van vehicle deforms and the deformation degree according to the three extracted images of the carriage and the three images before deformation, and judging whether the vehicle needs to be trimmed.
The method comprises the following specific steps:
Step 1, image acquisition: the four cameras shoot pictures of the van vehicle to be detected from the upper side at the same time, the cameras obtain the internal and external parameters of each camera through self-positioning, and the collected images are transmitted to the data processing equipment.
And 2, processing the image. And processing the acquired images, recovering a three-dimensional stereoscopic image of the van vehicle through four groups of binocular cameras, and extracting a left view, a right view and a top view of the van vehicle through edge recognition. Since the relative positions of the four vertexes of the carriage after deformation are basically unchanged, the three views of the original carriage before deformation can be restored according to the three extracted views of the carriage.
And processing the carriage view by adopting a stereo matching algorithm based on characteristics aiming at each group of double-sided cameras. Firstly, extracting features in two images, such as corners, straight lines, circular arcs, free curves, zero crossings of the images, edge features of the images and the like with obvious gray level changes, screening feature points and establishing a matching corresponding relation of the features between the two images. The specific implementation steps are as shown in fig. 3:
The feature-based matching algorithm comprises the following basic steps:
(1) The image is preprocessed to extract feature points or lines.
(2) For a certain feature in the left image, a similarity measurement formula is used to calculate the similarity between each feature near the feature scan line in the right image and the feature scan line.
(3) And selecting the feature with the greatest similarity as the matching feature of the feature in the left image.
(4) Repeating the steps until all the features in the left image are matched.
The feature-based matching algorithm does not directly depend on gray information, and has the advantages of good robustness, high calculation speed, small calculation amount and the like. In addition, since a region where parallax is discontinuous tends to appear in an edge region, the problem of parallax discontinuity in feature-based matching processing matching is relatively easy.
The coordinates of all points on the surface of the space object are obtained by the following calculation method for each group of double-sided cameras, so that the surface shape of the three-dimensional object and the position of the object in the space are reconstructed. The reconstruction of the three-dimensional space point is the most basic step in the three-dimensional reconstruction, and the coordinate of the space point can be obtained by the inverse process of the parallax principle.
The internal parameters of the camera and the external parameters between the binocular cameras can be obtained through camera calibration. Due to the limit constraint of stereo matching, when images after binocular correction are matched, parallax changes exist only in the horizontal direction, and parallax changes do not exist in the vertical direction. As shown in fig. 4, with the left camera coordinate system as the world coordinate system, assuming that the parameters of the left and right cameras are the same and the mapping point of the spatial point P (X W,YW,ZW) opposite to the left and right figures is P l(xl,yl),Pr(xr,yr), X l-xr＝d,yl＝yr is known, and the distance (base line length) B between the focal length f of the camera and the optical center of the binocular camera. The following relationship can be obtained with the left camera optical center as the origin of the world coordinate system:
Wherein (x o,yo) represents the mapping point of the optical center of the left camera on the image plane, and the formula (1) is normalized to obtain the mapping relation between the two-dimensional plane coordinates and the three-dimensional space point coordinates:
Wherein the transformation matrix Q is the transformation under a linear binocular camera model:
In practical situations, factors influencing three-dimensional reconstruction are mainly the deviation of the horizontal coordinates of the mapping coordinate points of the optical centers of the left and right cameras in the image. Therefore, the parallax d needs to be compensated by the deviation of the optical centers of the two cameras, and the compensated transformation matrix Q:
where x' o is the abscissa value of the right camera optical center map on the image.
And (3) obtaining coordinates of each pixel point in an actual space through the obtained binocular stereo matching dense disparity map to form a three-dimensional reconstruction point cloud. These actual spatial points are smoothly connected to form a preliminary three-dimensional reconstruction model. And further calculating the deformation of the carriage according to the data of the three-dimensional reconstruction model.
Step 3, calculating the deformation of the carriage:
The upper, lower, left and right sides of the left view are respectively marked as 1 st, 2 nd, 3 rd and 4 th sides, the upper, lower, left and right sides of the right view are respectively marked as 5 th, 6 th, 7 th and 8 th sides, and the upper, lower, left and right sides of the right view are respectively marked as 9 th, 10 th, 11 th and 12 th sides. The deformation amount of each side of the carriage can be obtained by comparing the three extracted views of the carriage with the views of the carriage before deformation, the deformation amount of the ith side is recorded as delta x i (i=1, 2,..12), and the maximum deformation rate of each side of the carriage is calculated
Wherein L i is the length of the ith side before deformation. Deformation rate of the cabin
Δκ＝max{Δα1,Δα2,...Δα12}，
If the carriage deformation rate is greater than the allowable maximum deformation rate ψ, namely delta kappa > ψ, the van vehicle needs to be refurbished; otherwise, the van does not require refurbishment.
Fig. 5 is a block diagram of a real-time deformation measuring system for a van based on binocular vision, which is used for measuring deformation of a van and judging whether the van needs to be trimmed. Specifically, this van deformation real-time measurement system based on binocular vision includes: the device comprises an image acquisition module, an image processing module and a deformation calculation module. The image acquisition module consists of a camera array, is used for acquiring images of the van vehicle from different positions and transmitting the images to the image processing module, and the camera array consists of four cameras; the image processing module consists of data processing equipment and is used for receiving the image acquired by the camera and processing the image; the deformation calculation module calculates whether the carriage of the van vehicle deforms and the deformation degree and judges whether the vehicle needs to be trimmed,
Fig. 6 and fig. 2 are a schematic diagram of a camera array and a schematic diagram of an overall real-time measurement system for deformation of a van based on binocular vision in the present invention, which are used for acquiring images of the van from different positions and transmitting the images to an image processing module. Specifically, the camera array is composed of four cameras, the four cameras are respectively positioned on four vertexes of a rectangular area of 3600mm multiplied by 7200mm at the top of the garage, the collected images can be ensured to recover a complete three-dimensional stereo view of the van, and every two cameras positioned on adjacent vertexes form a pair of binocular cameras.
The upper, lower, left and right sides of the left view are respectively marked as 1 st, 2 nd, 3 rd and 4 th sides, the upper, lower, left and right sides of the right view are respectively marked as 5 th, 6 th, 7 th and 8 th sides, and the upper, lower, left and right sides of the top view are respectively marked as 9 th, 10 th, 11 th and 12 th sides.
Fig. 7 is a left view deformation comparison chart of the real-time deformation measurement system of the van vehicle based on binocular vision, the superposition comparison between the left view of the extracted van and the left view of the van before deformation can obtain maximum deformation amounts of the 1 st, 2 nd, 3 rd and 4 th sides of the van as Δx 1max＝0mm,Δx2max＝298.2mm,Δx3max＝0mm,Δx4max =0 mm, and the maximum deformation rate Δα 1＝0％,Δα2＝6.13％,Δα3＝0％,Δα4 =0% of each side is calculated.
Fig. 8 is a right view deformation comparison chart of the binocular vision-based real-time deformation measurement system of the van vehicle, and the maximum deformation amounts of the 5 th, 6 th, 7 th and 8 th sides of the van vehicle are respectively deltax 5max＝0mm,Δx6max＝0mm,Δx7max＝274.9mm,Δx8max =0 mm by extracting the coincidence comparison between the right view of the van vehicle and the right view of the van vehicle before deformation, and the maximum deformation rate deltaalpha 5＝0％,Δα6＝0％,Δα7＝8.89％,Δα8 =0% of each side is calculated.
Fig. 9 is a right view deformation comparison chart of the binocular vision-based real-time deformation measurement system of the van vehicle, and the maximum deformation amounts of the 9 th, 10 th, 11 th and 12 th sides of the van vehicle are respectively Δx 9max＝557.4mm,Δx10max＝0mm,Δx11max＝0mm,Δx12max =0 mm by means of coincidence comparison between the top view of the van vehicle and the top view of the van vehicle before deformation, and the maximum deformation rate Δα 9＝0.54％,Δα10＝0％,Δα11＝0％,Δα12 =0% of each side is calculated. The deformation rate Δκ=max { Δα 1,Δα2,...Δα12 } =8.89% of the cabin, and since the deformation rate of the cabin is greater than the maximum permissible deformation rate ψ=4.00% (the value of ψ=can be preset as the case may be), the cabin vehicle needs to be refurbished.
The above embodiments are merely illustrative of the principles of the present invention and its effectiveness, and are not intended to limit the invention. Modifications and variations may be made to the above-described embodiments by those skilled in the art without departing from the spirit and scope of the invention. Accordingly, it is intended that all equivalent modifications and variations of the invention be covered by the claims, which are within the ordinary skill of the art, be within the spirit and scope of the present disclosure.
Claims (2)
1. The real-time van deformation measuring method based on binocular vision is characterized by comprising the following steps of:
Simultaneously acquiring images of the van vehicle from different positions through an image acquisition module, and transmitting the images to an image processing module; the image acquisition module consists of a camera array, the camera array consists of four cameras, the four cameras are respectively positioned on four vertexes of a rectangular area on the same horizontal plane above the parking space, the size of the rectangular area depends on the volume of the van, the acquired information is ensured to be enough to recover the three-dimensional perspective view of the van, and every two cameras positioned on the adjacent vertexes form a pair of binocular cameras;
The method comprises the steps of receiving images acquired by a camera through an image processing module, processing the images to obtain three-dimensional images of the van, extracting left view, right view and top view of a van carriage from the three-dimensional images, and recovering images of three views of the van carriage before deformation of the van carriage; the method specifically comprises the following steps: receiving an image acquired by a camera, processing the image by adopting a stereo matching algorithm based on characteristics, and obtaining coordinates of all points on the surface of a space object by a parallax principle calculation method so as to reconstruct the surface shape of the three-dimensional object and the position of the object in space; recovering three-dimensional images of the van vehicle through four groups of binocular cameras, and extracting left view, right view and top view of the van vehicle through an edge recognition algorithm; the relative positions of the four vertexes of the carriage after deformation basically remain unchanged, so that the images of the three views of the carriage before deformation of the van can be recovered according to the extracted three views of the carriage;
Calculating whether the carriage of the van vehicle deforms and the deformation degree according to the three extracted images of the carriage and the three images before deformation, and judging whether the van vehicle needs to be refurbished; the method specifically comprises the following steps: obtaining deformation of each side by superposition comparison, calculating the maximum deformation rate of each side, and further calculating the deformation rate of the car; judging whether the van vehicle needs to be refurbished according to whether the van deformation rate is larger than the allowable maximum deformation rate;
The method for processing the image by adopting the stereo matching algorithm based on the characteristics comprises the following steps: aiming at each group of binocular cameras, processing the images by adopting a three-dimensional matching algorithm based on characteristics, firstly extracting characteristic points in two images, including corners, straight lines, circular arcs, free curves with obvious gray level changes, zero crossings of the images and image edge characteristics, screening the characteristic points and establishing a matching corresponding relation of the characteristics between the two images;
for each group of double-sided cameras, coordinates of all points on the surface of the space object are obtained through the inverse process of the parallax principle, so that the surface shape of the three-dimensional object and the position of the object in the space are reconstructed, and the specific steps are as follows;
Obtaining an internal parameter of a camera and an external parameter between binocular cameras through camera calibration, wherein parallax change only exists in the horizontal direction and parallax change does not exist in the vertical direction when images subjected to binocular correction are matched due to limit constraint of stereo matching; taking the left camera coordinate system as the world coordinate system, assuming that the parameters of the left camera and the right camera are the same and the mapping point of the space point P (X W,YW,ZW) opposite to the left image and the right image is P l(xl,yl),Pr(xr,yr), wherein X l-xr＝d,yl＝yr is known as the distance between the focal length f of the camera and the optical center of the binocular camera, namely the base line length B, and taking the optical center of the left camera as the origin of the world coordinate system, the following relationship is obtained:
Wherein (x o,yo) represents the mapping point of the optical center of the left camera on the image plane, and the formula (1) is normalized to obtain the mapping relation between the two-dimensional plane coordinates and the three-dimensional space point coordinates:
Wherein the transformation matrix Q is the transformation under a linear binocular camera model:
In practical situations, factors affecting three-dimensional reconstruction are mainly the deviation of the horizontal coordinates of the mapping coordinate points of the optical centers of the left and right cameras in the image, so that the parallax d needs to be compensated by the deviation of the optical centers of the two cameras, and the compensated transformation matrix Q:
Wherein x' o is the abscissa value of the right camera optical center map on the image;
Obtaining coordinates of each pixel point in an actual space through the obtained binocular stereo matching dense parallax map to form a three-dimensional reconstruction point cloud; smoothly connecting the actual space points to form a three-dimensional reconstruction model;
Calculating the deformation of the carriage according to the data of the three-dimensional reconstruction model, and judging whether the carriage needs to be refurbished or not, wherein the method specifically comprises the following steps of:
The upper, lower, left and right sides of the left view are respectively marked as 1 st, 2 nd, 3 rd and 4 th sides, the upper, lower, left and right sides of the right view are respectively marked as 5 th, 6 th, 7 th and 8 th sides, the upper, lower, left and right sides of the right view are respectively marked as9 th, 10 th, 11 th and 12 th sides, the three views of the extracted carriage and the view of the carriage before deformation are subjected to superposition comparison to obtain deformation amounts of the respective sides of the carriage, the deformation amount of the i th side is marked as Deltax i, wherein i=1, 2..12, and the maximum deformation rate of the respective sides of the carriage is calculated as follows:
wherein, L i is the length of the ith side before deformation, so the deformation rate of the carriage is:
Δκ＝max{Δα1,Δα2,...Δα12}，
if the carriage deformation rate is greater than the allowable maximum deformation rate ψ, namely delta kappa > ψ, the van vehicle needs to be refurbished; otherwise, the van does not require refurbishment.
2. The real-time measurement method of deformation of van vehicle based on binocular vision according to claim 1, wherein the specific steps of the feature-based stereo matching algorithm are as follows:
(1) Preprocessing the image to extract characteristic points or lines;
(2) For a certain feature in the left image, calculating the similarity between each feature near the feature scanning line in the right image and the feature by using a similarity measurement formula;
(3) Selecting the feature with the greatest similarity as the matching feature of the feature in the left image;
(4) Repeating the steps until all the features in the left image are matched.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111344166.1A CN114049323B (en) | 2021-11-15 | 2021-11-15 | Real-time deformation measuring method for van vehicle based on binocular vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111344166.1A CN114049323B (en) | 2021-11-15 | 2021-11-15 | Real-time deformation measuring method for van vehicle based on binocular vision |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114049323A CN114049323A (en) | 2022-02-15 |
| CN114049323B true CN114049323B (en) | 2024-04-30 |
Family
ID=80208867
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111344166.1A Active CN114049323B (en) | 2021-11-15 | 2021-11-15 | Real-time deformation measuring method for van vehicle based on binocular vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114049323B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2005010820A2 (en) * | 2003-07-29 | 2005-02-03 | Holding Bev Sa | Automated method and device for perception associated with determination and characterisation of borders and boundaries of an object of a space, contouring and applications |
| CN103196371A (en) * | 2013-03-29 | 2013-07-10 | 南京工业大学 | Method and system for conducting initiative real-time three-dimensional measurement on large compartment |
| CN103217100A (en) * | 2013-03-29 | 2013-07-24 | 南京工业大学 | Online binocular vision measurement device for carriage of large bus |
| WO2015082293A1 (en) * | 2013-12-02 | 2015-06-11 | Commissariat A L'energie Atomique Et Aux Energies Alternatives | Identification of shapes by image correction |
| CN106600581A (en) * | 2016-12-02 | 2017-04-26 | 北京航空航天大学 | Train operation fault automatic detection system and method based on binocular stereoscopic vision |
| CN108458668A (en) * | 2018-01-05 | 2018-08-28 | 燕山大学 | Slab edge and Head and Tail Shape automatic checkout system based on binocular vision and method |
| CN110619329A (en) * | 2019-09-03 | 2019-12-27 | 中国矿业大学 | Carriage number and loading state identification method of railway freight open wagon based on airborne vision |
| CN111046795A (en) * | 2019-12-12 | 2020-04-21 | 扬州后潮科技有限公司 | Binocular vision-based real-time vehicle line pressing behavior detection method |
| CN111723704A (en) * | 2020-06-09 | 2020-09-29 | 杭州古德微机器人有限公司 | Raspberry pie-based van body door opening monitoring method |
| CN113393448A (en) * | 2021-06-25 | 2021-09-14 | 腾讯云计算（北京）有限责任公司 | Deformation detection method, device and equipment and computer readable storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8837774B2 (en) * | 2010-05-04 | 2014-09-16 | Bae Systems Information Solutions Inc. | Inverse stereo image matching for change detection |
-
2021
- 2021-11-15 CN CN202111344166.1A patent/CN114049323B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2005010820A2 (en) * | 2003-07-29 | 2005-02-03 | Holding Bev Sa | Automated method and device for perception associated with determination and characterisation of borders and boundaries of an object of a space, contouring and applications |
| CN103196371A (en) * | 2013-03-29 | 2013-07-10 | 南京工业大学 | Method and system for conducting initiative real-time three-dimensional measurement on large compartment |
| CN103217100A (en) * | 2013-03-29 | 2013-07-24 | 南京工业大学 | Online binocular vision measurement device for carriage of large bus |
| WO2015082293A1 (en) * | 2013-12-02 | 2015-06-11 | Commissariat A L'energie Atomique Et Aux Energies Alternatives | Identification of shapes by image correction |
| CN106600581A (en) * | 2016-12-02 | 2017-04-26 | 北京航空航天大学 | Train operation fault automatic detection system and method based on binocular stereoscopic vision |
| CN108458668A (en) * | 2018-01-05 | 2018-08-28 | 燕山大学 | Slab edge and Head and Tail Shape automatic checkout system based on binocular vision and method |
| CN110619329A (en) * | 2019-09-03 | 2019-12-27 | 中国矿业大学 | Carriage number and loading state identification method of railway freight open wagon based on airborne vision |
| CN111046795A (en) * | 2019-12-12 | 2020-04-21 | 扬州后潮科技有限公司 | Binocular vision-based real-time vehicle line pressing behavior detection method |
| CN111723704A (en) * | 2020-06-09 | 2020-09-29 | 杭州古德微机器人有限公司 | Raspberry pie-based van body door opening monitoring method |
| CN113393448A (en) * | 2021-06-25 | 2021-09-14 | 腾讯云计算（北京）有限责任公司 | Deformation detection method, device and equipment and computer readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114049323A (en) | 2022-02-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108444390B (en) | Unmanned automobile obstacle identification method and device | |
| CN104916163B (en) | Parking space detection method | |
| CN112418103B (en) | Bridge crane hoisting safety anti-collision system and method based on dynamic binocular vision | |
| Labayrade et al. | In-vehicle obstacles detection and characterization by stereovision | |
| JP3895238B2 (en) | Obstacle detection apparatus and method | |
| CN103630122B (en) | Monocular vision lane line detection method and distance measurement method thereof | |
| CN107187464B (en) | Track slab detection vehicle, system and method | |
| WO2015024407A1 (en) | Power robot based binocular vision navigation system and method based on | |
| CN111485475B (en) | Pavement pit recognition method and device | |
| CN111272139B (en) | Monocular vision-based vehicle length measuring method | |
| US11783507B2 (en) | Camera calibration apparatus and operating method | |
| CN110619674B (en) | Three-dimensional augmented reality equipment and method for accident and alarm scene restoration | |
| CN109827516B (en) | Method for measuring distance through wheel | |
| CN111694011A (en) | Road edge detection method based on data fusion of camera and three-dimensional laser radar | |
| Lu et al. | Automated visual surveying of vehicle heights to help measure the risk of overheight collisions using deep learning and view geometry | |
| CN114119729A (en) | Obstacle identification method and device | |
| CN104156946B (en) | Obstacle detection method for space tether robot in capturing area | |
| Hautière et al. | Road scene analysis by stereovision: a robust and quasi-dense approach | |
| CN112161685B (en) | Vehicle load measuring method based on surface characteristics | |
| CN114049323B (en) | Real-time deformation measuring method for van vehicle based on binocular vision | |
| Wang et al. | Geometry constraints-based visual rail track extraction | |
| CN217113327U (en) | Van deformation real-time measurement system based on binocular vision | |
| JP2014059646A (en) | Object detection device and object detection method | |
| CN115810054A (en) | Multi-camera calibration method in vehicle tire pattern depth detection equipment | |
| Nishigaki et al. | Moving obstacle detection using cameras for driver assistance system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |