WO2023066231A1

WO2023066231A1 - Vehicle point cloud recognition imaging method, system, computer device, and storage medium

Info

Publication number: WO2023066231A1
Application number: PCT/CN2022/125860
Authority: WO
Inventors: 鄂文轩; 周子策
Original assignee: 北京魔鬼鱼科技有限公司
Priority date: 2021-10-18
Filing date: 2022-10-18
Publication date: 2023-04-27
Also published as: CN114071112B; CN114071112A

Abstract

The present application discloses a vehicle point cloud recognition imaging method. The method comprises: establishing a unified three-dimensional coordinate system according to an operating environment of a vehicle; scanning, by means of an imaging device, a vehicle parked in a parking area to obtain point cloud data of the vehicle shape at different time points and presenting the point cloud data in the three-dimensional coordinate system; and optimizing the point cloud data.

Description

Vehicle point cloud recognition imaging method, system, computer equipment and storage medium

Cross References to Related Applications

This application claims the priority of a Chinese patent application filed with the China Patent Office on October 18, 2021 with the application number CN202111209381.0 and the application title "Method and System for Vehicle Point Cloud Recognition and Imaging", the entire contents of which are incorporated by reference in In this application.

technical field

The present application belongs to the technical field of image recognition, and in particular relates to a vehicle point cloud recognition imaging method, system, computer equipment and storage medium.

Background technique

With the development of urban modernization, automatic car washing services have been widely demanded by the society.

In the field of automatic car washing, it is necessary to identify and image the shape of the vehicle to form subsequent path planning. High-quality vehicle imaging is a key prerequisite for completing the automatic car washing process. Most of the current 3D imaging methods have problems such as poor imaging data quality. For example, the vehicle shape data obtained by the imaging sensor often has problems such as missing and abnormalities, which will also lead to poor imaging effects, especially in the dark mirror and car glass, the imaging quality is prone to defects, which limits the automatic car washing service The automation and intelligent development of technology.

Contents of the invention

In a first aspect, an embodiment of the present application provides a vehicle point cloud recognition imaging method, the method comprising the following steps:

Establish a unified three-dimensional coordinate system according to the operating environment of the vehicle;

The imaging equipment scans the vehicles parked in the parking area, and obtains the point cloud data of the vehicle shape at different times and presents them uniformly in the three-dimensional coordinate system. The imaging equipment includes fixed-position sensors and mobile-position sensors. sensors; and

Optimizing processing of point cloud data.

In a second aspect, the embodiment of the present application provides a vehicle point cloud recognition imaging system, including:

The three-dimensional coordinate system establishment module is used to establish a unified three-dimensional coordinate system according to the operating environment where the vehicle is located;

The imaging module is used to scan the vehicles parked in the parking area through the imaging device, obtain the point cloud data of the vehicle shape at different times and present them uniformly in the three-dimensional coordinate system, wherein the imaging device includes a fixed-position sensor and mobile-mounted sensors; and

The point cloud data optimization module is used to optimize the point cloud data.

In a third aspect, an embodiment of the present application provides a computer device, including a memory and one or more processors, where computer-readable instructions are stored in the memory, and when the computer-readable instructions are executed by the one or more processors, the One or more processors perform the following steps:

Obtain the point cloud data of the vehicle shape at different times obtained by scanning the vehicles parked in the parking area by the imaging device; wherein, the imaging device includes a fixed-position sensor and a mobile-position sensor;

Unified presentation of point cloud data in a three-dimensional coordinate system; and

Optimizing processing of point cloud data.

In a fourth aspect, the embodiment of the present application provides one or more non-volatile computer-readable storage media storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, one or more A processor performs the following steps:

Optimizing processing of point cloud data.

The details of one or more embodiments of the application are set forth in the accompanying drawings and the description below. Other features and advantages of the application will be apparent from the description, drawings, and claims.

Description of drawings

Fig. 1 is a schematic flowchart of a vehicle point cloud recognition and imaging method according to one or more embodiments.

Fig. 2 is an architecture diagram of a vehicle point cloud recognition imaging system according to one or more embodiments.

FIG. 3 is a schematic diagram of an arrangement of an imaging device according to one or more embodiments.

Figure 4 is an internal block diagram of a computer device in accordance with one or more embodiments.

Fig. 5 is a schematic diagram of an application environment of a vehicle point cloud recognition and imaging method according to one or more embodiments.

Detailed ways

The following examples are only used to illustrate the technical solutions of the present application more clearly, but not to limit the protection scope of the present application. Certain terms are used, for example, in the description and claims to refer to particular components. Those skilled in the art should understand that hardware or software manufacturers may use different terms to refer to the same component. The specification and claims do not use the difference in name as the way to distinguish components, but use the difference in function of the components as the criterion for distinguishing. The subsequent description of the specification is a preferred implementation mode for implementing the application, but the description is for the purpose of illustrating the general principle of the application, and is not intended to limit the scope of the application. The scope of protection of the present application should be defined by the appended claims.

The present application will be described below in conjunction with the accompanying drawings and specific embodiments.

An embodiment of the present application provides a vehicle point cloud recognition and imaging method, which is used to acquire vehicle shape features during self-service car washing, wherein the self-service car washing can be carried out in indoor or outdoor car wash rooms, or in other specific environmental scenarios, in There are rails installed and fixed on the periphery of the parking area for vehicle cleaning. The rails can be elliptical rails around the vehicle, or one or more linear rails arranged parallel or non-parallel along the length of the vehicle or in other directions. A mobile terminal that slides along the track. The mobile terminal can be a robot or a module with simple functions such as imaging. The mobile terminal comes with multiple and different types of sensors to scan the vehicle's appearance features in an all-round way.

Referring to FIG. 1 , the vehicle point cloud recognition imaging method may include step S101 , step S102 and step S103 .

Step S101: Establish a unified three-dimensional coordinate system according to the working environment where the vehicle is located.

After the equipment is installed, a unified three-dimensional coordinate system needs to be established, which specifically includes the following methods: according to the structural layout of the working environment where the vehicle is located, determine a certain point in the structural layout as the coordinate origin, and at the same time determine the type and direction of the coordinate system. Taking the car wash as an example, according to the size of the car wash, the spatial structure layout, the parking area and other factors, select a point in the space as the coordinate origin, and then determine the coordinate axis direction and coordinate type to establish a three-dimensional coordinate system, such as Choose a three-dimensional Cartesian coordinate system. After establishing a unified coordinate system, the position data of all subsequent scanned image points will be calculated and judged in this coordinate system.

The structures in the structural layout and the vehicle positions in the parking area are measured and calibrated to determine their coordinate points relative to the coordinate origin. After the coordinate system is selected, all static structures in the current environment structure and subsequent vehicles entering the car wash room have corresponding position coordinate points. Wherein, the selected structures may include tracks on the ground, lights on the walls, wall lines, parking lines and so on.

After the three-dimensional coordinate system is established, it is necessary to obtain accurate data of the observation coordinates of the imaging equipment through the method of measurement and calibration. The coordinate points of the three-dimensional drawings of the positions of each structure preset in the working environment are compared with the coordinate points of an image point position data obtained by the actual scanning of the imaging equipment relative to the origin. When there is an error between the two, the error range is preset in the imaging In the data, corrections are made, such as adjusting the rotation axis parameters of the imaging device to adjust the imaging coordinate points, so that the calibration and measurement coordinate points of the same part are as consistent as possible, thereby improving the accuracy of image recognition. Through multiple scanning imaging and on-the-spot measurement, the spatial attitude (rotation axis parameters in the three directions of X-axis, Y-axis, and Z-axis) and spatial coordinates of each imaging sensor are obtained through geometric calculation to ensure that the calibrated and measured coordinates are Consistent, realizing the unification of the coordinates of the on-site calibration, the coordinates of the drawn 3D drawing, and the coordinates obtained by the imaging sensor actually scanned. When errors occur, the working parameters of the imaging sensor can be adjusted in real time and calibrated continuously to maximize the three. Unite.

Step S102: Scan the vehicles parked in the parking area with the imaging device, obtain point cloud data of vehicle shapes at different times, and present them uniformly in the three-dimensional coordinate system. Wherein, the imaging device includes a fixed-camera sensor and a mobile-camera sensor.

The fixed position sensor is arranged in a specific place of the working environment, such as the top corner of the car wash room, the top of the fixed frame, the bottom side of the fixed track, etc. On the one hand, the fixed position sensor can monitor specific parts of the vehicle from a specific angle. Image acquisition, on the other hand, can also obtain the status of vehicles leaving and entering the warehouse, and conduct real-time monitoring of the current operating environment.

The mobile position sensor scans in real-time motion to obtain the shape features of the vehicle. The motion includes mobile scanning, rotational scanning or a combination of the two. Mobile scanning includes horizontal scanning, such as a straight line along the length of the vehicle. Scanning during sliding or circular scanning or arc scanning around the vehicle body, and scanning in the vertical direction, such as scanning in a straight line in the vehicle height direction, the motion trajectory path of the imaging device is determined by the local computer Or the background server calculates and determines that its motion route is fixed and accurate, and the rotation of the imaging angle of the imaging sensor can also be controlled simultaneously during the moving scanning process. In the embodiment of the present application, the acquisition of image features of the vehicle shape by the imaging device is carried out through the cooperation of two types of sensors, the fixed position sensor and the mobile position sensor. The fixed position sensor can obtain the outline of the vehicle from a large angle, and the mobile position sensor mainly obtains the local characteristics of the vehicle from the body part. The combination of the fixed position sensor and the mobile position sensor Can maximize the integrity of the scan. The advantage of real-time scanning imaging during motion is that, on the one hand, when the imaging device is moving, its position will change in the three-dimensional coordinate system, and the obtained imaging data is also determined based on the real-time positional relationship of the imaging device. In addition, the motion of the imaging device The relationship between displacement and time, as well as the positional relationship of the imaging device relative to the vehicle at a certain moment, correspond to each other. Since the displacement of the imaging device is relatively definite and clear, the acquired vehicle imaging data is also relatively clear. The imaging device can obtain point cloud data of different parts of the vehicle from different angles at any time, and accumulate and superimpose the point cloud data, which significantly improves the resolution of the imaging aggregation data and improves the imaging quality. In some embodiments, The imaging resolution can be reduced from about 10cm of the original high-end lidar to less than 1cm, and the imaging accuracy can be reduced from 2cm of the original high-end lidar to less than 1cm.

The acquisition of vehicle appearance features can be carried out through the cooperation of multiple different types of imaging devices, including radar (such as lidar, ultrasonic radar, millimeter wave radar, etc.), image sensors (such as high-definition cameras, binocular or multi-eye cameras, etc.) , TOF camera, thermal imaging camera, etc.), one or more of distance sensors. In some optional embodiments, different types of imaging devices (that is, different types of sensors) can be used to collect independently at the same time, and each sensor has its own unique advantages. After fusion processing, data redundancy will be formed, from which better image feature points can be selected to reduce the defect that the imaging data of a single type of sensor is prone to missing or the imaging quality of a specific part of a pair of vehicles is poor, so as to be more complete. Capturing the complete features of the vehicle can also overcome the poor imaging quality of the vehicle's dark mirror surface.

The mobile imaging sensor (that is, the sensor of the mobile position) can move with the machine on the track. The machine can be one, two or more than two. The imaging sensor (that is, the sensor in the imaging device) has a section with the body The interval, the installation height of the imaging sensor is better than the highest point of the vehicle body, to avoid missing the roof imaging data, and to obtain a better imaging angle when imaging other vehicle positions. In traditional technology, the incident source of the imaging sensor is in the direction of the vehicle's top view and side view, which is a vertical incidence, and imaging data cannot be captured in vertical parts such as the front and rear of the vehicle. When the imaging device in the embodiment of the present application scans different positions of the car body, the sensor is controlled to adjust different spatial attitudes, such as following the shape of the car to ensure that the incident source is as close as possible to the normal line of the car. , which can complete the scanning following the shape of the vehicle. Specifically, during the continuous movement and/or rotation of the imaging sensors, at least at a certain moment, the incident angle of the radiation signal generated by one or more imaging sensors coincides with or approaches the normal of the vehicle surface. Taking radar, distance sensor and TOF (Time-of-Flight, Time-of-Flight) camera and other sensors that actively transmit signals as examples, the incident from the normal line, the reflected signal returns along the normal line, because it is nearly perpendicular to the surface of the object, so The reflection performance of the signal is the strongest, which has the greatest possibility of obtaining high-quality imaging data and ensuring high accuracy and high confidence of the data, specifically manifested in significantly improved available resolution, three-dimensional measurement accuracy, and significantly improved dark mirror problems; It is convenient for the precise planning of subsequent cleaning and air-drying trajectories. In addition, the imaging sensor is in the form of rotation and movement. Even if the signal source at a certain moment is not close to the normal of the vehicle shape, it will be close to the normal of the vehicle shape at the next moment or at other moments. Or coincide with the vehicle shape normal. The above-mentioned method overcomes the core problem of limited incident angle of the imaging sensor, cooperates with computer algorithms, significantly weakens the problem of dark mirrors, significantly improves the imaging coverage and imaging quality of the car body, and significantly improves the three-dimensional measurement accuracy.

In addition, the image acquisition of the imaging device adopts the combination of global scanning and partial scanning. Specifically, the main sensor and the local sensor are set up. The main sensor is responsible for the overall scanning of vehicle data, and the local sensor is responsible for scanning and imaging the area where the main sensor cannot accurately obtain data. The main sensor and the local sensor scan and image at the same time. The overall data is based on the scanning data of the main sensor. The main sensor should ensure the largest possible scanning coverage area, and the local sensor is responsible for the scanning data of special positions such as rearview mirrors and bumpers. When arranging, the height of the main sensor is usually higher to obtain a relatively good scanning area, and the location of the local sensor is mainly based on the location to be scanned. Sensors can also be used for motion control (including movement, rotation, etc.).

As an optional implementation manner of the present application, before scanning the vehicles in the parking area, a preprocessing operation may also be performed. Preprocessing operations include one or more of coordinate system processing, physics processing, and environment processing. The preprocessing process includes one or more processes of coordinate system processing, physical processing and environment processing.

The coordinate system processing includes the following steps: performing consistent calibration on different imaging devices, and unifying their coordinate systems. Different types of imaging sensors and corresponding machine positions are calibrated consistently, and the coordinate system is unified, so that the point cloud data scanned by different sensors can be presented in the same three-dimensional space, forming data redundancy and preventing the loss of scanned data.

Physical treatment includes the following steps: Before starting the scan, apply a layer of anti-reflective material on the surface of the vehicle body. The anti-reflective material can be cleaning foam solvent or other materials that can reduce specular reflection. Among them, the cleaning solvent covers at least the dark mirror part and the transparent glass part of the body surface; through the above-mentioned physical treatment means, increasing the diffuse reflectance can reduce the reflection of the dark mirror and transparent mirror, which is helpful for subsequent image scanning.

Environmental processing includes the following steps: turn off the light of the current working environment or reduce the light intensity of the light, or reduce unnecessary light irradiation from a certain angle. In this way, background interference of ambient light can be reduced, thereby reducing noise sources of imaging data acquisition.

Step S103: Optimizing the point cloud data.

Because the imaging equipment may show varying degrees of imaging quality degradation or even failure to image due to factors such as the distance to the imaging surface, the incident angle of the imaging surface, the material properties of the imaging surface, and the external environment, it is necessary to monitor the formed points. The cloud data is detected to determine the imaging quality of the point cloud data. Specifically, according to the different spatial coordinate positions and incident angles of different surfaces of the vehicle, and based on the empirical data method, the point cloud density and point cloud clustering scale of the continuous surface are judged. , data continuity and other factors, determine the scanning quality of each imaging area, and use different optimization processing methods or use different optimization parameters in a targeted manner. Optimization processing may include one or more of the following methods:

Method 1: By setting image noise filters (such as Gaussian filters, mean filters, median filters, and bilateral filters), the collected point cloud images are filtered to filter out noise imaging data caused by external factors. Among them, the external factors that are prone to noise mainly include air dust, ambient light, etc., after filtering, noise interference can be reduced to the greatest extent.

The second method is to remove the smearing data existing in the point cloud data by setting a special anti-smearing algorithm filter (such as weighted circular convolution method), because the signal emitted by optical sensors such as laser radar is not a straight line, but a signal with Small-angle fan-shaped light beam, so the signal will form a spot with a certain area in front. When part of the spot is projected on the surface of the object and the other part is not on the surface of the object, the signal that is not on the surface of the object will continue to fly until it meets the next object. , resulting in smearing phenomenon, when the smearing phenomenon occurs, there will be two or more data returned by transmitting a signal source, the signal data returned by the second or the third signal data is likely to be an error point, and the sensor acquires The erroneous points of the point cloud data are removed through the tailing algorithm, and only the more reliable point cloud data is retained. The filtering range of the erroneous points can be adjusted according to the preset filter algorithm.

The third way is to sort out high-quality point cloud data and low-quality point cloud data, and optimize the post-processing of low-quality point cloud data.

Low-quality point cloud data mainly includes the following characteristics: 1. The imaging sensor presents unpredictable point cloud density at this imaging angle; 2. Uneven point cloud density; 3. The point cloud data is distributed in small groups; 4. 1. Insufficient continuity of the vehicle surface; 5. Insufficient point cloud imaging quality inherent in the rearview mirror, antenna and other positions of the vehicle.

Judgment that the imaging sensor exhibits unpredictable point cloud density at this imaging angle. In some optional implementations, the corresponding acceptable point cloud density range values can be set for multiple imaging angles according to actual needs. When the data obtained by an imaging sensor, the point cloud density is not at an acceptable point When the cloud density is within the range value, it can be determined that the imaging sensor presents an unpredictable point cloud density at the corresponding imaging angle, so that the corresponding point cloud data is classified into a low-quality category.

Judgment about non-uniform point cloud density. In some optional implementations, according to actual needs, an acceptable value for representing point cloud density uniformity can be set for different imaging sensors (for example, standard deviation or variance can be used as the value), when a certain When the value of the point cloud density uniformity of the imaging sensor exceeds an acceptable value for representing the point cloud density uniformity, it is determined that the point cloud density is not uniform, thereby classifying the corresponding point cloud data into a low-quality category.

Judgment on the distribution of point cloud data in small clusters. In some optional implementations, the space where the number of point clouds per unit volume exceeds the preset data threshold can be regarded as a community. When the distance between a community and other one or more communities exceeds the preset distance threshold, then It is determined that the point cloud data is distributed in small groups.

Judgments about the lack of continuity of the vehicle's surface. In some optional implementations, the distance between adjacent sampling points in the vehicle surface line can be calculated. When the distance between adjacent sampling points exceeds the preset sampling point distance threshold, and such adjacent sampling points Insufficient vehicle surface continuity is determined when the logarithm of λ exceeds a preset logarithmic threshold.

Judgment on the insufficient quality of point cloud imaging inherent in the vehicle's rearview mirror, antenna, etc. It is possible to preset an acceptable number of point clouds inherent in the position of the vehicle's rearview mirror, antenna, etc., and when the corresponding number of point clouds obtained is less than the preset acceptable number of point clouds, it is determined that the quality of the point cloud imaging is insufficient.

The quality judgment of point cloud data can be realized through the artificial intelligence model. After judging the low-quality point cloud data, different post-processing optimizations are carried out according to different situations. Post-processing optimization may include removing abnormal points in point cloud data, automatically filling in missing point cloud data, or adjusting the working parameters of imaging equipment and filters. Remove the abnormal points in the point cloud data: the imaging of the surface of the object at different positions by the sensors of different camera positions has obvious rules in the point cloud density, and the point clouds that violate the rules are the abnormal points, and the outlier point filter and the point cloud density judgment are used Filtering is used to post-process the point cloud data to remove abnormal points. Automatically fill in the missing point cloud data: judge the continuity and abnormal points of the data through the region growing and gradient descent algorithm, and perform fuzzy filling when it is judged that there is some discontinuity in the vehicle's shape image data, such as hollow or blank areas Processing, fuzzy filling can automatically generate enlarged images for blank areas based on interpolation filling algorithms. After separating the low-quality point cloud data and performing post-processing optimization, the parameters of the equipment can also be adaptively adjusted, including: adjusting the working parameters of the imaging sensor, adjusting the number of sensor positions, and adjusting the working parameters of the filter . The working parameters of the imaging sensor mainly refer to the transmission power, the angle of incidence, the linear distance of scanning, the moving speed, the rotation speed, the sampling frequency, etc. Sensitivity of the received sensor to increase the sensitivity of the sensor, or increase the sensor position, increase the sampling frequency, etc. to obtain more data redundancy. The working parameters of the filter mainly refer to parameters related to external environmental noise interference and smearing, such as power, wavelength, frequency, etc., to maximize the filtering effect and de-smearing effect, such as suppressing similar wavelengths of ambient light, thereby reducing sources of optical noise. The filtering range value of the filter is also an important adjustment parameter. In the part where the imaging quality is not high, the filtering range boundary of the filter is narrowed. For example, in the rearview mirror, antenna, bumper and other radar imaging quality High position, appropriately narrow the range of wrong judgments, and keep as much imaging data as possible.

Method 4: According to the symmetry characteristics of the vehicle, when it is detected that the image data obtained by scanning on one side is bad, mirror processing is performed on the qualified image data obtained by scanning on the other side.

Almost all vehicles on the market are symmetrically designed. For example, the exterior rearview mirrors of automobiles will most likely not exist in isolation. Based on the above characteristics, when one side of the scan is defective, the mirror image replacement is performed using the scan result of the other side that is symmetrical. Specifically, through sensor motion scanning, it is judged whether the data is missing. If a certain part of the data is missing more seriously, but according to the central axis of the vehicle, look for the scanning result on the other side corresponding to the missing part. In actual operation, the hub data Imaging is the best in theory, so it is very convenient to use the contour data and coordinates on both sides as a reference. In this coordinate system, establish a symmetrical plane, perform mirror replacement of left and right image data, and finally synthesize a complete three-dimensional image data .

In addition, based on the established model database containing a large number of complete vehicle shapes of different models, the vehicle model data of the same model can be queried from the model database according to the detected current vehicle model for replacement; or, query whether the current vehicle has historical scans Data, if there is, select the corresponding vehicle model data from the historical scan data to replace. This method is only applied when the scanned vehicle image is optimized and still cannot meet the basic imaging requirements.

In an optional embodiment, it is also possible to accumulate big data and establish artificial intelligence models for scanned objects with the same nature on the vehicle. The same nature means that the structure, shape and layout are basically the same, such as car windows, car When scanning the surface of objects in a certain area of the vehicle, such as sunroofs, car rearview mirrors, and car headlights, the artificial intelligence model established automatically recognizes and optimizes the algorithm for the objects in the area. surface, the artificial intelligence model automatically identifies the nature of the object and predicts a reasonable scanning distance, and at the same time adjusts the working parameters of the imaging device (such as power, wavelength, etc.) and/or the working parameters of the filter (such as passband bandwidth, center frequency, cutoff frequency, VSWR, delay time, filter range value). For example, when scanning a whole vehicle, the window is a transparent object that is difficult to scan and perceive. After the artificial intelligence recognizes the window area, it predicts a reasonable scanning distance and adjusts the working parameters of radar, ultrasonic sensors, and visual sensors in real time. Improve the sensitivity of the sensor and reduce the noise threshold, and at the same time give the smearing filter the correct reference distance value and other methods to improve the scanning and imaging quality. Generally speaking, the shorter the scanning distance, the higher the imaging quality, the greater the power or the longer the wavelength, the higher the imaging quality. For different parts of the vehicle, the scanning distance, the number of images per second, and the wavelength will be adaptively adjusted , in addition, for the smear phenomenon generated by scanning different parts, the reference distance threshold for smear removal will also have different tolerances. By scanning different parts of the vehicle and performing big data modeling, each vehicle can be obtained An artificial intelligence model with homogeneous parts can learn and understand the data defects existing in the machine imaging process and improve them, and perform different parameter adjustments or automatic optimization of algorithms in different parts to improve the efficiency and accuracy of scanning imaging, especially For the dark mirror part, according to its commonality, it can greatly save the scanning time, and also overcome the problem of insufficient imaging quality at the black mirror and transparent glass.

In some embodiments, the vehicle point cloud recognition imaging method can be applied to the application environment shown in FIG. 5 . In FIG. 5 , the server 501 can execute the vehicle point cloud recognition and imaging method, for example, step S101 , step S102 and step S103 can be executed. Of course, the server 501 can also perform other more steps. The imaging device 502 can scan the vehicle to obtain point cloud data, and send the point cloud data to the server 501 . In some optional implementation manners, the server 501 may also control the imaging device 502 to scan the vehicle by sending a control signal.

In some of the above-mentioned embodiments, during the execution of the vehicle point cloud recognition imaging method, the coordinate consistency calibration of the imaging sensor and the physical processing of the environmental light body are carried out before the point cloud data is acquired, and the quality of the point cloud is checked during scanning and imaging. Sorting and judging, the low-quality point cloud data is post-optimized, and the accuracy of point cloud imaging is comprehensively improved.

In some of the above-mentioned embodiments, in the process of implementing the vehicle point cloud recognition imaging method, large data accumulation and modeling are carried out on objects with the same nature as the vehicle, and different imaging device parameters and filter parameters are adjusted according to different scanning areas , not only improves the scanning accuracy, but also overcomes the problem of insufficient imaging quality of black mirror and transparent glass.

In some of the above-mentioned embodiments, during the execution of the vehicle point cloud recognition and imaging method, the post-optimization processing of low-quality point cloud data includes multiple processing methods, which can form complete point cloud data to the greatest extent.

As shown in FIG. 2 , the embodiment of the present application provides a vehicle point cloud recognition imaging system 200 , which includes: a three-dimensional coordinate system establishment module 201 , an imaging module 202 and a point cloud data optimization module 203 .

The three-dimensional coordinate system establishment module 201 is used to establish a unified three-dimensional coordinate system according to the working environment of the vehicle, and the point cloud data collected subsequently are measured and calculated in this coordinate system.

The imaging module 202 is used to scan the vehicles parked in the parking area through the imaging device, obtain the point cloud data of the vehicle shape at different times and present them uniformly in the three-dimensional coordinate system, wherein the imaging device includes a fixed-position sensor and mobile sensors. Fig. 3 is a schematic diagram of a scene arrangement of a mobile-position sensor in the embodiment of the present application. Exemplarily, the imaging device moves and scans in the horizontal direction along with machines A and B on slide rails on both sides. Machine A and machine B are mobile sensors.

The point cloud data optimization module 203 is used for optimizing the point cloud data.

In some embodiments, the point cloud data optimization module 203 may include:

The regional imaging quality judging unit (not shown) is used to judge the information density and information relevance of the scanned point cloud data, and obtain the imaging quality scores of each region in the point cloud data for post-processing optimization; wherein, the Post-processing optimization includes: removing abnormal points in point cloud data, automatically filling in missing point cloud data, adjusting working parameters of imaging equipment and filters;

A filter unit (not shown) is used to remove outlier noise and trailing noise data in three-dimensional imaging;

The mirror image processing unit (not shown) is used for performing mirror image processing on the qualified image data scanned by the other side when the image data scanned by one side is detected to be defective according to the symmetry characteristics of the vehicle.

Due to continuous scanning and imaging, large data accumulation has been formed. The imaging system of the present application is also provided with a model building unit, which is used to accumulate the shape image data of different parts of the vehicle obtained by dynamic and real-time scanning of the machine equipment, and respectively establishes the data of different parts of the vehicle. The artificial intelligence model for image scanning of parts (such as car windows, wheel hubs, car headlights, car trunks, etc.), by continuously learning the advantages and disadvantages of different imaging devices, adjusting the parameters of imaging devices or filters when scanning corresponding parts, Further, the accuracy of image acquisition and scanning efficiency are improved.

For the specific limitations of the vehicle point cloud recognition and imaging system 200 , please refer to the above definition of the vehicle point cloud recognition and imaging method, which will not be repeated here. Each module or unit in the above-mentioned vehicle point cloud recognition imaging system 200 may be fully or partially realized by software, hardware or a combination thereof. The above-mentioned modules or units may be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory of the computer device in the form of software, so that the processor can invoke and execute the corresponding operations of the above modules.

The vehicle point cloud recognition and imaging system 200 in the embodiment of the present application can have the same technical effect as the vehicle point cloud recognition and imaging method in the above embodiment, so it will not be repeated here.

In some embodiments, an embodiment of the present application provides a computer device, including a memory and one or more processors, where computer-readable instructions are stored in the memory, and when the computer-readable instructions are executed by the one or more processors, Causes one or more processors to perform the following steps:

Unified presentation of point cloud data in a three-dimensional coordinate system;

Optimizing processing of point cloud data.

In some other embodiments, when the computer-readable instructions stored in the memory of the computer device are executed by one or more processors, one or more processors can perform the vehicle point cloud recognition imaging method in the foregoing embodiments. other steps.

The computer device provided in the embodiment of the present application may be a server, and its internal structure diagram may be as shown in FIG. 4 . The computer device includes a processor, a memory, and a network interface connected through a system bus. Wherein, the processor of the computer device is used to provide calculation and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer readable instructions. The internal memory provides an environment for the execution of the operating system and computer readable instructions in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer-readable instructions are executed by the processor, the vehicle point cloud recognition imaging method in some embodiments herein is implemented.

In some embodiments, the embodiments of the present application also provide one or more non-volatile computer-readable storage media storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, a or multiple processors perform the following steps:

Optimizing processing of point cloud data.

In some other embodiments, when the computer-readable instructions stored in the non-volatile computer-readable storage medium are executed by one or more processors, one or more processors can execute the vehicle point in the foregoing embodiments. Other steps in the cloud recognition imaging method.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented through computer programs to instruct related hardware. The aforementioned computer programs can be stored in a non-volatile computer-readable storage medium In this case, when the computer program is executed, it may include the processes of the embodiments of the above-mentioned methods. Wherein, any references to memory, storage, database or other media used in the various embodiments provided by the embodiments of the present disclosure may include non-volatile and/or volatile memory. Nonvolatile memory can include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Chain Synchlink DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

It is worth noting that the above are only some embodiments of the application, and do not limit the scope of patent protection of the application. equivalents are substituted. Therefore, all equivalent structural changes made by using the instructions and illustrations of this application, or directly or indirectly applied to other related technical fields are also included in the scope of this application.

Claims

A vehicle point cloud recognition imaging method, characterized in that, comprising:

Establish a unified three-dimensional coordinate system according to the operating environment of the vehicle;

The vehicle parked in the parking area is scanned by the imaging device, and the point cloud data of the vehicle shape at different times is obtained and presented in the three-dimensional coordinate system, wherein the imaging device includes a fixed-position sensor and a mobile in-camera sensors; and

Optimizing the point cloud data.
The imaging method according to claim 1, further comprising: before scanning the vehicles in the parking area, performing a pre-processing operation; the pre-processing operation includes:

Coordinate consistent calibration of different imaging devices; and/or,

Sprinkle anti-reflective material on the surface of the vehicle body, wherein the anti-reflective material covers at least the dark mirror part and the transparent glass part of the surface of the vehicle body; and/or,

Turn off the lights in the current working environment or reduce the light intensity of the lights.
The imaging method according to claim 1 or 2, wherein the optimization process includes one or more of the following methods:

Filter out noise imaging data caused by external factors;

Eliminate trailing data existing in point cloud data;

Judge the imaging quality of point cloud data, sort out high-quality point cloud data and low-quality point cloud data, and optimize the post-processing of low-quality point cloud data;

According to the symmetry characteristics of the vehicle, when it is detected that the image data scanned on one side is bad, the qualified image data scanned on the other side is mirrored.
The imaging method according to claim 3, wherein said post-processing optimization includes: removing abnormal points in point cloud data, automatically filling in missing point cloud data, or adjusting working parameters and filtering of imaging equipment The operating parameters of the device.
The imaging method according to any one of claims 1-3, wherein the optimization process also includes: accumulating large data on scanned objects with the same nature on the vehicle and establishing artificial intelligence models respectively, When the surface of an object in a certain area of the vehicle is scanned, the object in the area is automatically recognized and the algorithm is optimized through the established artificial intelligence model.
The imaging method according to claim 5, wherein the automatic identification and algorithm optimization include:

When the surface of an object in a certain area of the vehicle is scanned, the artificial intelligence model automatically recognizes the nature of the object and predicts a reasonable scanning distance, and at the same time adjusts the working parameters of the imaging device and/or the working parameters of the filter.
The imaging method according to claim 3, wherein said judging the imaging quality of said point cloud data comprises: according to different spatial coordinate positions and incident angles of different surfaces of the vehicle, and judging continuous surfaces based on empirical data The point cloud density, point cloud clustering scale, determine the scanning quality of each imaging area, and use different optimization processing methods or use different optimization parameters.
The imaging method according to any one of claims 1-7, wherein said establishing a unified three-dimensional coordinate system comprises: determining a certain point in the current environment as the coordinate origin according to the structural layout of the working environment where the vehicle is located, and simultaneously Determine the type and direction of the coordinate system; measure and calibrate the position of the structures in the structure layout and the vehicle in the parking area, and determine their position coordinates relative to the origin of the coordinates.
The imaging method according to claim 8, further comprising: after the unified three-dimensional coordinate system is established, correcting the image data obtained by the actual scanning of the imaging device and the position data measured and calibrated, and the correction includes :

Based on the measured position coordinates of structures and vehicles relative to the origin, cooperate with the coordinates of the three-dimensional drawing of the operation scene sketched in advance, and compare with the coordinates of the image data obtained by scanning the imaging equipment relative to the origin, and calculate the coordinates of each imaging equipment Space attitude and space coordinates, calibration imaging accuracy error; and

In response to the existence of errors, the error range is preset in the imaging data and corrected, so that the coordinates of the on-site measurement and calibration, the preset coordinates of the 3D drawing, and the coordinates obtained by the actual scanning of the imaging device are unified.
The imaging method according to claim 4, wherein the adjusting the working parameters of the imaging device and the working parameters of the filter comprises: reducing the boundary filtering range value of the filter for parts with low imaging quality.
A vehicle point cloud recognition imaging system is characterized in that it comprises:

The three-dimensional coordinate system establishment module is used to establish a unified three-dimensional coordinate system according to the operating environment where the vehicle is located;

The imaging module is used to scan the vehicles parked in the parking area through the imaging device, obtain the point cloud data of the vehicle shape at different times and present them uniformly in the three-dimensional coordinate system, wherein the imaging device includes a fixed position type sensors and mobile position sensors; and

The point cloud data optimization module is used for optimizing the point cloud data.
The imaging system of claim 11, comprising:

The area imaging quality judging unit is used to judge the information density and information relevance of the scanned point cloud data, and obtain the imaging quality scores of each area in the point cloud data for post-processing optimization;

A filter unit for removing outlier noise and trailing noise data in 3D imaging; and

The mirror image processing unit is used to perform mirror image processing on the qualified image data scanned on the other side when the image data scanned on one side is detected to be defective according to the symmetry characteristics of the vehicle.
A computer device, characterized by comprising a memory and one or more processors, wherein computer readable instructions are stored in the memory, and when the computer readable instructions are executed by the one or more processors, the The one or more processors perform the following steps:

Establish a unified three-dimensional coordinate system according to the operating environment of the vehicle;

Acquiring the point cloud data of the vehicle shape at different times obtained by scanning the vehicles parked in the parking area by the imaging device; wherein, the imaging device includes a fixed-position sensor and a mobile-position sensor;

uniformly presenting the point cloud data in the three-dimensional coordinate system; and

Optimizing the point cloud data.
One or more non-transitory computer-readable storage media storing computer-readable instructions, wherein when the computer-readable instructions are executed by one or more processors, the one or more processors Perform the following steps:

Establish a unified three-dimensional coordinate system according to the operating environment of the vehicle;

Acquiring the point cloud data of the vehicle shape at different times obtained by scanning the vehicles parked in the parking area by the imaging device; wherein, the imaging device includes a fixed-position sensor and a mobile-position sensor;

uniformly presenting the point cloud data in the three-dimensional coordinate system; and

Optimizing the point cloud data.