CN115908558A

CN115908558A - Roadside parking management method and system based on course angle posture

Info

Publication number: CN115908558A
Application number: CN202211363486.6A
Authority: CN
Inventors: 冯澍; 闫昊; 丁丽珠; 王艳清
Original assignee: Smart Intercommunication Technology Co ltd
Current assignee: Smart Intercommunication Technology Co ltd
Priority date: 2022-11-02
Filing date: 2022-11-02
Publication date: 2023-04-04

Abstract

The application discloses a roadside parking management method and system based on course angle postures. The method comprises the steps of sequentially carrying out size transformation, key point data reconstruction and random erasure data enhancement on a vehicle image of each vehicle to obtain an enhanced vehicle image; inputting the enhanced vehicle image into a backbone network to obtain a backbone characteristic diagram; inputting the backbone characteristic diagram into a characteristic aggregation network to obtain an aggregation characteristic diagram; inputting the aggregated feature map into a key point prediction network to obtain a predicted attitude center point and a plurality of predicted offsets of each vehicle; the aggregation characteristic diagram is input into a vehicle course angle regression network, and a predicted course angle sine value and a predicted course angle cosine value of each vehicle are obtained; and performing key point prediction on each image of the vehicle to be tested extracted from the image of the traffic scene to be tested according to the trained key point detection model, obtaining a posture central point, a plurality of offsets, a course angle sine value and a course angle cosine value of each vehicle to be tested, and performing roadside parking management.

Description

Roadside parking management method and system based on course angle posture

Technical Field

The application relates to the technical field of image processing, in particular to a roadside parking management method and system based on a course angle posture.

Background

In recent years, high-level video technology has been rapidly developed, and by installing a high-level video camera on the roadside, parking of a vehicle can be judged and managed, and by detecting the vehicle, detecting the posture of the vehicle body, and then performing data analysis with a known parking position, the state of the vehicle, such as whether the vehicle is located in the parking space, whether the vehicle is parked with a line, whether the vehicle is parked in a no-parking area, and the like, can be judged. Whether the vehicle is illegal to stop or not is judged according to the posture of the vehicle body, so that the method has positive promoting effect on all aspects of urban traffic management, driving safety and the like.

According to the traditional method, vehicle attitude estimation is carried out through a two-dimensional vehicle rectangular detection frame to achieve roadside parking management. However, under the influence of surrounding traffic scenes, the vehicle is shielded, the pose of the vehicle is judged only by the two-dimensional vehicle rectangular detection frame, and the position of the vehicle cannot be accurately judged, so that the roadside parking management efficiency is low.

Disclosure of Invention

The method aims to solve the technical problem that roadside parking management efficiency is low due to the fact that the position of a vehicle cannot be accurately judged in the traditional method. In order to achieve the purpose, the application provides a roadside parking management method and system based on a course angle posture.

The application provides a roadside parking management method based on course angle posture, which comprises the following steps:

acquiring a modeling simulation traffic scene image data set, wherein the modeling simulation traffic scene image data set comprises a plurality of traffic scene images, each traffic scene image is marked with real information of a two-dimensional detection frame of each vehicle, a plurality of real attitude key points corresponding to a three-dimensional surrounding frame of each vehicle, a real attitude central point, a plurality of real offsets from the real attitude key points to the real attitude central point respectively, a real course angle of each vehicle, a real course angle sine value and a real course angle cosine value;

extracting each vehicle in each traffic scene image according to the real information of the two-dimensional detection frame to obtain a vehicle image of each vehicle;

carrying out size transformation and key point data reconstruction on the vehicle images to obtain reconstructed vehicle images of each vehicle, and carrying out random erasure data enhancement on the reconstructed vehicle images to obtain enhanced vehicle images of each vehicle;

inputting the enhanced vehicle image into a backbone network of a key point detection model for feature extraction to obtain a backbone feature map data set;

inputting each backbone feature map in the backbone feature map data set into a feature aggregation network of the key point detection model for feature fusion to obtain an aggregation feature map data set;

inputting each aggregated feature map in the aggregated feature map dataset into a key point prediction network of the key point detection model to perform point location prediction, and obtaining a predicted attitude center point and a plurality of predicted offsets of each vehicle;

inputting each aggregated characteristic map in the aggregated characteristic map data set into a vehicle course angle regression network of the key point detection model for course angle prediction, obtaining a predicted course angle sine value and a predicted course angle cosine value of each vehicle, and obtaining a predicted course angle of each vehicle according to the predicted course angle sine value and the predicted course angle cosine value;

constructing a loss function of the key point detection model according to the predicted course angle, the predicted attitude center point, the plurality of predicted offsets, the real course angle, the real attitude center point and the plurality of real offsets, and performing training optimization on the key point detection model according to the loss function to obtain a trained key point detection model;

acquiring a traffic scene image to be tested, and predicting key points of each vehicle image to be tested extracted from the traffic scene image to be tested according to the trained key point detection model to obtain a posture central point, a plurality of offsets, a course angle sine value and a course angle cosine value of each vehicle to be tested;

and obtaining a plurality of attitude key points of each vehicle to be tested according to the attitude center points and the plurality of offsets, obtaining a course angle of each vehicle to be tested according to the sine value of the course angle and the cosine value of the course angle, and performing roadside parking management according to the plurality of attitude key points and the course angle.

In one embodiment, the performing size transformation and key point data reconstruction on the vehicle image to obtain a reconstructed vehicle image of each vehicle, and performing random erasure data enhancement on the reconstructed vehicle image to obtain an enhanced vehicle image of each vehicle includes:

setting the original length of the vehicle image as 256 pixels, and transforming the original width of the vehicle image according to a length transformation ratio to obtain a new width of the vehicle image;

filling the new width of the vehicle image by 0 pixel to 256 pixels to obtain a size conversion image;

and transforming the plurality of real attitude key points, the real attitude center point and the plurality of real offsets according to the transformation ratio of the size transformation image to the original size of the vehicle image to obtain the reconstructed vehicle image.

In one embodiment, the constructing a loss function of the keypoint detection model according to the predicted course angle, the predicted attitude center point, the plurality of predicted offsets, the actual course angle, the actual attitude center point, and the plurality of actual offsets, and performing training optimization on the keypoint detection model according to the loss function to obtain a trained keypoint detection model includes:

constructing a regression loss function of the vehicle course angle according to the predicted course angle and the real course angle of each vehicle;

constructing a regression loss function of the vehicle attitude center point according to the predicted attitude center point and the real attitude center point of each vehicle;

constructing a regression loss function of the offset of the vehicle attitude key point according to the plurality of predicted offsets and the plurality of real offsets of each vehicle;

and constructing a loss function of the key point detection model according to the regression loss function of the vehicle course angle, the regression loss function of the vehicle attitude central point and the regression loss function of the offset of the vehicle attitude key point.

In one embodiment, the obtaining of the traffic scene image to be tested and the key point prediction of each vehicle image to be tested extracted from the traffic scene image to be tested according to the trained key point detection model to obtain the attitude center point, the multiple offsets, the course angle sine value and the course angle cosine value of each vehicle to be tested includes:

carrying out vehicle target detection on the traffic scene image to be detected according to a vehicle target detection algorithm to obtain two-dimensional detection frame information of each vehicle to be detected;

extracting the traffic scene image to be detected according to the two-dimensional detection frame information to obtain a vehicle image to be detected of each vehicle to be detected;

carrying out size transformation and key point data reconstruction on the vehicle image to be detected to obtain a reconstructed vehicle image to be detected of each vehicle to be detected, and carrying out random erasure data enhancement on the reconstructed vehicle image to be detected to obtain an enhanced vehicle image to be detected of each vehicle to be detected;

and inputting the enhanced vehicle image to be tested into the trained key point detection model, and outputting the attitude center point, the plurality of offsets, the course angle sine value and the course angle cosine value of each vehicle to be tested.

In an embodiment, the obtaining a plurality of attitude key points of each vehicle to be tested according to the attitude center point and the plurality of offsets, obtaining a heading angle of each vehicle to be tested according to a sine value of the heading angle and a cosine value of the heading angle, and performing roadside parking management according to the plurality of attitude key points and the heading angle includes:

performing coordinate conversion of a world coordinate system on a plurality of posture key points of each vehicle to be detected to obtain the position of each vehicle to be detected;

and performing roadside parking management according to the position of the vehicle to be tested and the course angle.

In one embodiment, the present application provides a roadside parking management system based on a heading angle pose, comprising:

the data acquisition module is used for acquiring a modeling simulation traffic scene image dataset which comprises a plurality of traffic scene images, wherein each traffic scene image is marked with real information of a two-dimensional detection frame of each vehicle, a plurality of real attitude key points corresponding to a three-dimensional surrounding frame of each vehicle, a real attitude center point, a plurality of real offsets from the real attitude key points to the real attitude center point respectively, a real course angle of each vehicle, a real course angle sine value and a real course angle cosine value;

the image extraction module is used for extracting each vehicle in each traffic scene image according to the real information of the two-dimensional detection frame to obtain a vehicle image of each vehicle;

the data enhancement module is used for carrying out size transformation and key point data reconstruction on the vehicle images to obtain reconstructed vehicle images of each vehicle, and carrying out random erasure data enhancement on the reconstructed vehicle images to obtain enhanced vehicle images of each vehicle;

the backbone network module is used for inputting the enhanced vehicle image into a backbone network of a key point detection model for feature extraction to obtain a backbone feature map data set;

the feature aggregation network module is used for inputting each backbone feature map in the backbone feature map data set into a feature aggregation network of the key point detection model for feature fusion to obtain an aggregation feature map data set;

the key point prediction network module is used for inputting each aggregated feature map in the aggregated feature map data set into a key point prediction network of the key point detection model to perform point location prediction, so as to obtain a predicted attitude center point and a plurality of predicted offsets of each vehicle;

the course angle regression network module is used for inputting each aggregation characteristic diagram in the aggregation characteristic diagram data set into a vehicle course angle regression network of the key point detection model to predict course angles, obtaining a predicted course angle sine value and a predicted course angle cosine value of each vehicle, and obtaining the predicted course angle of each vehicle according to the predicted course angle sine value and the predicted course angle cosine value;

the model training module is used for constructing a loss function of the key point detection model according to the predicted course angle, the predicted attitude center point, the plurality of predicted offsets, the real course angle, the real attitude center point and the plurality of real offsets, and training and optimizing the key point detection model according to the loss function to obtain a trained key point detection model;

the detection module is used for acquiring a traffic scene image to be detected, predicting key points of each vehicle image to be detected extracted from the traffic scene image to be detected according to the trained key point detection model, and acquiring a posture central point, a plurality of offsets, a course angle sine value and a course angle cosine value of each vehicle to be detected;

and the parking management module is used for obtaining a plurality of attitude key points of each vehicle to be tested according to the attitude center points and the plurality of offsets, obtaining a course angle of each vehicle to be tested according to the sine value of the course angle and the cosine value of the course angle, and performing roadside parking management according to the plurality of attitude key points and the course angle.

In one embodiment, the data enhancement module comprises:

the size conversion module is used for setting the original length of the vehicle image to be 256 pixels and converting the original width of the vehicle image according to a length conversion ratio to obtain a new width of the vehicle image;

the pixel filling module is used for filling the new width of the vehicle image by 0 pixel to 256 pixels to obtain a size conversion image;

and the key point data reconstruction module is used for transforming the plurality of real attitude key points, the real attitude central points and the plurality of real offsets according to the transformation ratio of the size transformation image and the original size of the vehicle image to obtain the reconstructed vehicle image.

In one embodiment, the model training module comprises:

the first regression loss function module is used for constructing a regression loss function of the vehicle course angle according to the predicted course angle and the real course angle of each vehicle;

the second regression loss function module is used for constructing a regression loss function of the vehicle attitude central point according to the predicted attitude central point and the real attitude central point of each vehicle;

the third regression loss function module is used for constructing a regression loss function of the offset of the vehicle attitude key point according to the plurality of predicted offsets and the plurality of real offsets of each vehicle;

and the total loss function module is used for constructing a loss function of the key point detection model according to the regression loss function of the vehicle course angle, the regression loss function of the vehicle attitude central point and the regression loss function of the offset of the vehicle attitude key point.

In one embodiment, the detection module comprises:

the two-dimensional detection frame information acquisition module is used for carrying out vehicle target detection on the traffic scene image to be detected according to a vehicle target detection algorithm to acquire two-dimensional detection frame information of each vehicle to be detected;

the to-be-detected vehicle image acquisition module is used for matting the to-be-detected traffic scene image according to the two-dimensional detection frame information to acquire a to-be-detected vehicle image of each to-be-detected vehicle;

the to-be-detected enhanced vehicle image acquisition module is used for carrying out size transformation and key point data reconstruction on the to-be-detected vehicle image to obtain a to-be-detected reconstructed vehicle image of each to-be-detected vehicle, and carrying out random erasure data enhancement on the to-be-detected reconstructed vehicle image to obtain a to-be-detected enhanced vehicle image of each to-be-detected vehicle;

and the to-be-tested vehicle attitude acquisition module is used for inputting the to-be-tested enhanced vehicle image into the trained key point detection model and outputting the attitude central point, the plurality of offsets, the course angle sine value and the course angle cosine value of each to-be-tested vehicle.

In one embodiment, the parking management module comprises:

the coordinate conversion module is used for performing world coordinate system coordinate conversion on the plurality of posture key points of each vehicle to be tested to obtain the position of each vehicle to be tested;

and the management module is used for performing roadside parking management according to the position of the vehicle to be tested and the course angle.

In the method and the system for roadside parking management based on the course angle posture, virtual data of a traffic scene and label information thereof are generated through a simulation modeling technology, and the method and the system can be applied to a real roadside parking scene image. In the virtual data generation process, can obtain accurate vehicle three-dimensional surrounding frame automatically according to digital vehicle model, need not to carry out artifical mark, also avoided the error that artifical mark brought, usable different vehicle model can generate different parking scene image data simultaneously to and adjust different camera visual angles, obtain the data at different visual angles, also greatly reduced data acquisition's time and labour cost.

Data processing enhancement is carried out through size transformation, key point data reconstruction and random erasure, so that the robustness of the model in the face of a sheltering scene can be enhanced, and overfitting of the model is avoided. The enhanced vehicle images of each vehicle are used for carrying the labeled information and are sequentially input into a backbone network, a characteristic aggregation network, a key point prediction network and a vehicle course angle regression network to perform regression prediction of the attitude key point and the course angle, so that the accuracy of vehicle attitude prediction of each vehicle is improved. And inputting each vehicle image to be tested into the trained key point detection model for prediction, and obtaining a corresponding attitude center point, a plurality of offsets, a course angle sine value and a course angle cosine value so as to obtain a plurality of attitude key points and course angles of each vehicle in an image coordinate system. Therefore, the plurality of attitude key points in the image coordinate system are subjected to coordinate conversion into a plurality of attitude key points in the world coordinate system, and the parking position and the parking posture of the vehicle can be obtained by judging the parking position in the same world coordinate system in combination with the constraint of the vehicle body course angle, so that the parking judgment and management of the vehicle are realized, and whether the vehicle has behaviors of illegal parking, line pressing parking and the like is judged. The vehicle attitude is judged by performing regression on the eight key points of the vehicle and predicting the vehicle course angle, and the vehicle body attitude can be predicted according to the visible key points under the condition of serious vehicle shielding, so that the robustness is strong.

Drawings

FIG. 1 is a schematic flow chart illustrating steps of a roadside parking management method based on a heading angle posture provided by the present application.

FIG. 2 is a schematic diagram of 8 key points in one embodiment provided herein.

FIG. 3 is a schematic structural diagram of a roadside parking management system based on a heading angle attitude provided by the present application.

Detailed Description

The technical solution of the present application is further described in detail by the accompanying drawings and embodiments.

Referring to fig. 1, the present application provides a roadside parking management method based on a heading angle posture, including:

s10, acquiring a modeling simulation traffic scene image data set, wherein the modeling simulation traffic scene image data set comprises a plurality of traffic scene images, and each traffic scene image is marked with real information of a two-dimensional detection frame of each vehicle, a plurality of real attitude key points corresponding to a three-dimensional surrounding frame of each vehicle, a real attitude central point, a plurality of real offsets from the real attitude key points to the real attitude central point, a real course angle of each vehicle, a real course angle sine value and a real course angle cosine value;

s20, extracting each vehicle in each traffic scene image according to the real information of the two-dimensional detection frame to obtain a vehicle image of each vehicle;

s30, carrying out size transformation and key point data reconstruction on the vehicle images to obtain reconstructed vehicle images of each vehicle, and carrying out random erasure data enhancement on the reconstructed vehicle images to obtain enhanced vehicle images of each vehicle;

s40, inputting the enhanced vehicle image into a backbone network of the key point detection model for feature extraction to obtain a backbone feature map data set;

s50, inputting each backbone feature map in the backbone feature map data set into a feature aggregation network of the key point detection model for feature fusion to obtain an aggregation feature map data set;

s60, inputting each aggregation characteristic diagram in the aggregation characteristic diagram data set into a key point prediction network of a key point detection model for point location prediction, and obtaining a predicted attitude center point and a plurality of predicted offsets of each vehicle;

s70, inputting each aggregation characteristic diagram in the aggregation characteristic diagram data set into a vehicle course angle regression network of a key point detection model to predict a course angle, obtaining a predicted course angle sine value and a predicted course angle cosine value of each vehicle, and obtaining a predicted course angle of each vehicle according to the predicted course angle sine value and the predicted course angle cosine value;

s80, constructing a loss function of the key point detection model according to the predicted course angle, the predicted attitude center point, the plurality of predicted offsets, the real course angle, the real attitude center point and the plurality of real offsets, and training and optimizing the key point detection model according to the loss function to obtain a trained key point detection model;

s90, acquiring a traffic scene image to be tested, and predicting key points of each vehicle image to be tested extracted from the traffic scene image to be tested according to the trained key point detection model to obtain a posture center point, a plurality of offsets, a course angle sine value and a course angle cosine value of each vehicle to be tested;

s100, obtaining a plurality of attitude key points of each vehicle to be tested according to the attitude center points and the plurality of offsets, obtaining a course angle of each vehicle to be tested according to a course angle sine value and a course angle cosine value, and performing roadside parking management according to the plurality of attitude key points and the course angle.

In this embodiment, in S10, a large number of virtual data images of traffic scenes are obtained by using a modeling simulation technique. The modeling simulation technology includes, but is not limited to, based on game engines such as Unity and Unreal, modeling of the roadside parking scene is achieved by using simulation software such as Carla and Arisim, and virtual roadside parking scene image data are obtained. The virtual traffic scene image data obtained through modeling and the real traffic scene image data have the same spatial layout and the like, for example, the virtual traffic scene image data and the real traffic scene image data have the same camera shooting height and angle, and have the targets of vehicle parking behaviors including a process that a vehicle enters a parking space, a process that the vehicle is in the parking space and a process that the vehicle exits the parking space, a parking space line, a road side green belt and the like. When a vehicle model is constructed in the modeling process, the real information of the two-dimensional detection frame of each vehicle and a plurality of real attitude key points corresponding to the three-dimensional surrounding frame of each vehicle can be obtained. According to the real information of the two-dimensional detection frame of each vehicle, the coordinate position of the corresponding real attitude center point can be obtained, and then a plurality of real offsets from a plurality of real attitude key points to the real attitude center point respectively can be obtained. The heading angle can be understood as the vehicle body heading angle, and the included angle between the vehicle running direction and the direction of the horizontal axis of the coordinate is in the range of [ -pi, pi ].

The real information of the two-dimensional detection frame of each vehicle comprises x coordinate information and y coordinate information of the upper left corner and the lower right corner of a rectangular frame of the vehicle, wherein the x coordinate information and the y coordinate information are respectively (x) ₁ ，y ₁ ) And (x) ₂ ，y ₂ ). The real attitude center point represents the center point coordinate of the two-dimensional detection frame and can be obtained through the coordinate information of the two-dimensional detection frame. The three-dimensional bounding box information of each vehicle includes the type of the real pose key point, the coordinates of the real pose key point, and the attribute of whether the real pose key point is visible. The plurality of real attitude key points represent information of the three-dimensional surrounding frame and represent three-dimensional outline information of the vehicle.

Referring to fig. 2, in an embodiment, each vehicle corresponds to eight key points, which respectively represent eight vertexes of a solid of a three-dimensional bounding box, which are respectively four key points of the vehicle contacting the ground and four key points of the roof in the air, and it can also be understood that, according to the orientation of the vehicle head, the key point on the left side of the vehicle head in the direction of the vehicle head in contact with the ground is taken as a point bottom-front-1 No. 1, and then is rotated clockwise, the other three ground points are respectively taken as a point bottom-front-2, a point bottom-back-3, and a point bottom-back-4, the key point on the left side of the vehicle head in the direction of the roof is taken as a point top-front-5 No. 5, and is rotated clockwise, and the other three roof key points are respectively taken as a point top-front-6, a point-back-7, and a point top-k-8. Eight types of key points of body attitude are described by the eight vertices of a three-dimensional bounding box that encloses the vehicle. The coordinates of the real pose key points are x and y coordinates in an image coordinate system. The attributes of the real pose key points comprise visible and invisible attributes, wherein the attribute of the visible point is recorded as 1, and the attribute of the invisible point is recorded as 0. Invisible points can be understood as the occlusion of the body of the vehicle in the captured image of the vehicle of the pose keypoint, e.g., points 1 and 4 in fig. 2, if occluded, have the attribute of invisible points. Conversely, the visible point can be understood as that the posture key point is not blocked by the vehicle body in the captured vehicle image, for example, the point 2, the point 3, the point 5, the point 6, the point 7, and the point 8 in fig. 2 are not blocked, and thus the visible point attribute is provided.

The position information of the two-dimensional detection frame and the position information of the three-dimensional surrounding frame are relatively compared by a plurality of real offsets from the real attitude key points to the real attitude central point, so that the specific positions of the vehicles are more accurately positioned, and the specific position of each vehicle can be more accurately positioned.

The real information of the two-dimensional detection frames, the real attitude key points, the real attitude center points, the real offsets from the real attitude key points to the real attitude center points, the real course angle of each vehicle, the sine value of the real course angle and the cosine value of the real course angle are used as tag data. And taking each traffic scene image containing label data as the input of the key point detection model to carry out model training optimization.

In S20, each two-dimensional detection frame real information represents information of the length and width of the vehicle. And each vehicle in the traffic scene image can be extracted through the real information of the two-dimensional detection frame of each vehicle, and each vehicle image is extracted from the traffic scene image to obtain a vehicle image corresponding to each vehicle. One vehicle corresponds to one vehicle image.

In S30, the original size of each vehicle image is uniformly adjusted to 256 × 256, and size conversion is performed. Meanwhile, according to the change relation between the original size and 256 multiplied by 256, coordinate transformation adjustment is realized on a plurality of real attitude key points, a plurality of real attitude center points and a plurality of real offsets from the real attitude key points to the real attitude center points of the vehicle image, key point data reconstruction is realized, transformation with the same proportion is carried out, and a reconstructed vehicle image of each vehicle is obtained. By carrying out data enhancement on the reconstructed vehicle image in a random erasing mode, the robustness of the model in the face of scenes such as vehicle shielding, pedestrian shielding or green plant shielding can be enhanced, and overfitting of the model is avoided.

In S40, the backbone network, the feature aggregation network, the keypoint prediction network, and the vehicle heading angle regression network form a keypoint detection model. The backbone network is used for extracting the features of the image, the size of the input image is H multiplied by W multiplied by C, H and W respectively represent the height and width of the image, C represents the number of channels and represents an RGB three-channel image. Backbone networks include, but are not limited to, using ResNet, VGG, mobileNet, and the like.

In S50, the feature aggregation network is used to aggregate features of high and low layers extracted from different layers in the backbone network, so as to provide more feature representations for subsequent keypoint prediction. Because the high-level semantic features are close to the output end of the network but have lower resolution, and the high-fraction features are close to the input end but have fewer semantic features, the features between different layers of the network in the backbone network are aggregated, so that the fusion between the high-level features and the low-level features can be realized, and the detection precision of a subsequent key point detection task is improved.

In S60, the input of the keypoint prediction network is the feature map after feature aggregation, and the size is (H/4) × (W/4) × C, C =256. And the key point prediction network is used for classifying the characteristics of the center points and the key points, positioning the positions of the center points and the key points and outputting predicted attitude center points and predicted offset. The predicted offset corresponds to the real offset and represents the predicted offset from the key point of the predicted attitude to the central point of the predicted attitude.

In S70, the vehicle heading angle refers to an included angle between the vehicle driving direction and a direction of a horizontal axis of coordinates, the included angle range is [ -pi, pi ], and regression of the angle is performed by calculating a sine value and a cosine value of the vehicle heading angle. The vehicle course angle regression network is composed of a plurality of layers of convolution neural network layers, an activation function layer and a normalization layer. The input of the vehicle course angle regression network is an aggregation characteristic diagram, and the output is a sine value of a predicted course angle and a cosine value of the predicted course angle, so that the predicted course angle of each vehicle is determined.

In S80, the deviation amount of the attitude key point with respect to the attitude center point, which constitutes the three-dimensional surrounding frame of the vehicle, is used as an index for the key point detection model learning, and the attitude center point is used as an object of regression, and the heading angle of each vehicle is used as an object of regression. The attitude central point, the plurality of attitude key points and the course angle are blended for constraint, so that the regression difficulty of the key point detection task can be reduced, and the detection accuracy is improved. And constructing a loss function of the key point detection model by the predicted course angle, the predicted attitude center point, the plurality of predicted offsets, the real course angle, the real attitude center point and the plurality of real offsets to realize model optimization and parameter updating.

In S90, the traffic scene image to be detected includes a plurality of vehicle images to be detected. And extracting each vehicle image to be detected from the traffic scene image to be detected, so as to obtain the vehicle image to be detected corresponding to each vehicle to be detected. The image of the vehicle to be tested is input into the trained key point detection model for key point prediction, so that the attitude center point, a plurality of offsets, the course angle sine value and the course angle cosine value of each vehicle can be obtained, and the specific position of each vehicle to be tested is predicted in an all-round and accurate manner.

In S100, the offset is an offset value of the attitude key point relative to the attitude center point, and a plurality of attitude key points of each vehicle to be measured can be obtained according to the attitude center point and the plurality of offsets. And obtaining the course angle of each vehicle to be measured according to the sine value and the cosine value of the course angle.

The roadside parking management method based on the heading angle posture generates virtual data and tag information of a traffic scene through a simulation modeling technology, and can be applied to a real roadside parking scene image. In the virtual data generation process, the accurate vehicle three-dimensional surrounding frame can be automatically obtained according to the digital vehicle model, manual marking is not needed, errors caused by manual marking are avoided, meanwhile, different parking scene image data can be generated by using different vehicle models, different camera visual angles are adjusted, data at different visual angles are obtained, and the time and labor cost for data acquisition are greatly reduced. Meanwhile, data of different vehicle types, different visual angles, different shielding conditions and different parking conditions can be simulated, and scenes of the data are greatly enriched, so that the data can be effectively dealt with when different conditions in real scenes are met.

In one embodiment, S10, acquiring a modeled simulated traffic scene image dataset, where the modeled simulated traffic scene image dataset includes a plurality of traffic scene images, and each traffic scene image is labeled with real information of a two-dimensional detection frame of each vehicle, a plurality of real pose key points corresponding to a three-dimensional enclosure frame of each vehicle, a real pose center point, a plurality of real offsets from the real pose key points to the real pose center point, a real heading angle of each vehicle, a real heading angle sine value, and a real heading angle cosine value, and includes:

s110, obtaining a vehicle real posture central point according to the real upper left corner coordinate and the real lower right corner coordinate of each vehicle;

and S120, calculating a plurality of real offsets of the plurality of vehicle real attitude key points of each vehicle to the vehicle real attitude central point.

In this embodiment, the real information of the two-dimensional detection frame of each vehicle includes a real upper left corner coordinate and a real lower right corner coordinate of the rectangular frame of each vehicle. The central point of the vehicle attitude is C _car The dimensions are 1 x 2 dimensions, 2 denoting the x and y coordinates. The calculation mode of the coordinates of the real attitude center point of each vehicle is as follows:

x_center＝(x ₁ +x ₂ )/2；

y_center＝(y ₁ +y ₂ )/2。

position C of the true attitude center point of each vehicle _car Is (x _ center, y _ center), and has dimension 1-2.

The combined offset of the true pose keypoints to the true pose center point for each vehicle is denoted as J, with dimension (W/R × H/R × 1 × 2). In one embodiment, the L1 distance is used to calculate the respective offset of each of the real pose keypoints for each vehicle from the real pose center point.

The calculation formula of the real offset from the real attitude key point to the real attitude center point is as follows:

I ₁ 、I ₂ and respectively representing two vectors of each real posture key point and the real posture center point, wherein p represents the dimensionality of the two vectors, namely 1-2 dimensionality.

The true pose keypoints can be represented as the position of the pose center point plus the true offset from the center point:

l _k ＝(x_center,y_center)+J _j

j∈1，2…k。

in one embodiment, S30, performing size transformation and key point data reconstruction on the vehicle image to obtain a reconstructed vehicle image of each vehicle, and performing random erasure data enhancement on the reconstructed vehicle image to obtain an enhanced vehicle image of each vehicle, includes:

s310, setting the original length of the vehicle image to be 256 pixels, and transforming the original width of the vehicle image according to the length transformation ratio to obtain the new width of the vehicle image;

s320, filling the new width of the vehicle image by 0 pixel to 256 pixels to obtain a size conversion image;

and S330, transforming the plurality of real attitude key points, the real attitude central points and the plurality of real offsets according to the transformation ratio of the size transformation image to the original size of the vehicle image to obtain a reconstructed vehicle image.

In this embodiment, each vehicle image is extracted from the traffic scene image according to the real information of the two-dimensional detection frame of each vehicle, and the width and height information of each vehicle image can be calculated and can also be understood as length information and width information. The size of the original length of the vehicle image is set to a fixed value of 256 pixels. A conversion ratio exists between the original length and 256 pixels, and the size of the original width of the vehicle image is converted according to the length change ratio to form a new width. When the new width is smaller than 256 pixels, the insufficient portion is filled with a pixel value of 0, and finally each vehicle image is formed to have a size of 256 × 256. And correspondingly transforming a plurality of real attitude key points, real attitude central points and a plurality of real offsets according to the transformation ratio between the vehicle image and the size transformation image, obtaining key point coordinate information based on the vehicle image after extraction and size transformation again, and obtaining a reconstructed vehicle image after key point data reconstruction.

In one embodiment, the original image size of each vehicle image is 500 × 300 (length × width), the length is converted from 500 to 256, and the width is converted from 300 to 300 × (256/500) =153.6. The width 153.6 (which may be rounded to 154) is smaller than 256, and the deficiency of 154 to 256 is filled with pixel 0, so that the size of the size-converted image is converted to 256 × 256. The original coordinates of one of the plurality of real pose keypoints are (200, 100), the new coordinates of the keypoints after reconstructing the vehicle image are (200/500 × 256=102.4, 100/300 × 256=85.3), and the new coordinates are (102, 85) after rounding. The new coordinates of the other remaining true pose keypoints are calculated in the same manner. And further, a real attitude center point and a plurality of real offsets can be further obtained according to the new coordinates after transformation, and a reconstructed vehicle image is obtained.

In one embodiment, S40, inputting the enhanced vehicle image into a backbone network of the key point detection model for feature extraction, and obtaining a backbone feature map data set, includes:

s410, inputting the enhanced vehicle image of each vehicle into the convolutional layer, and outputting a first feature map data set;

s420, inputting each first feature map in the first feature map data set to a normalization layer, and outputting a second feature map data set;

and S430, inputting each second feature map in the second feature map data set into the activation function layer, and outputting a third feature map data set, wherein the third feature map data set comprises a plurality of third feature maps.

In this embodiment, the backbone network performs a superposition operation of a plurality of convolution combination layers by using a convolution combination mode of convolution layer-normalization layer-activation function layer, and performs downsampling once in each convolution combination operation. The downsampling multiple is R. Normalization layers, including but not limited to an instance normalization layer, an adaptive instance normalization layer, and the like. The nonlinear activation layer includes, but is not limited to, nonlinear activation functions such as ReLU, leaky ReLU, and the like. In one embodiment, two downsampling operations are performed in the backbone network feature extraction stage, that is, R =4, so that accuracy in subsequent keypoint detection can be ensured. The input image size of the feature extraction network is H × W × C, and after three times of downsampling processes, the feature map size becomes (H/4) × (W/4) × C, where C =256.

In one embodiment, S50, inputting each backbone feature map in the backbone feature map data set into a feature aggregation network of the keypoint detection model for feature fusion, to obtain an aggregated feature map data set, includes:

and S510, inputting the first feature map, the second feature map and the third feature map into a feature aggregation network for feature fusion, and obtaining an aggregation feature map of an aggregation feature map data set.

In this embodiment, the features of the high and low layers extracted from different layers in the backbone network are fused by the feature aggregation network, so that the features of the layers of the high and low layers are fused to form an aggregation feature map. The detection precision of the subsequent key point detection task is improved by detecting and analyzing the aggregated characteristic diagram of the aggregated characteristic diagram data set.

In one embodiment, the S80, constructing a loss function of the keypoint detection model according to the predicted course angle, the predicted attitude center point, the plurality of predicted offsets, the actual course angle, the actual attitude center point, and the plurality of actual offsets, and performing training optimization on the keypoint detection model according to the loss function to obtain a trained keypoint detection model, includes:

s810, constructing a regression loss function of the vehicle course angle according to the predicted course angle and the real course angle of each vehicle;

s820, constructing a regression loss function of the vehicle attitude center point according to the predicted attitude center point and the real attitude center point of each vehicle;

s830, constructing a regression loss function of the offset of the vehicle attitude key point according to the plurality of predicted offsets and the plurality of real offsets of each vehicle;

and S840, constructing a loss function of the key point detection model according to the regression loss function of the vehicle course angle, the regression loss function of the vehicle attitude center point and the regression loss function of the offset of the vehicle attitude key point.

In this embodiment, model optimization and parameter update are performed by constructing a loss function for model training. The loss function of the key point detection model is a multitask loss function and comprises three parts, namely a regression loss function of a vehicle course angle, a regression loss function of a vehicle attitude central point and a regression loss function of the offset of a vehicle attitude key point.

The loss function of the keypoint detection model is:

L＝α ₁ L _reg +α ₂ L _ha +α ₃ L _offset ；

wherein L is _reg 、L _ha 、L _offset Regression loss function for respectively representing vehicle attitude central point and regression loss of vehicle course angleRegression loss function alpha of deviation amount of loss function and vehicle attitude key point ₁ 、α ₂ 、α ₃ Respectively, are weight coefficients. And may be set to 1 in one embodiment.

In one embodiment, L _reg 、L _ha 、L _offest The regression Loss function may use an L1 Loss mean absolute error Loss function. The L1 Loss mean absolute error Loss function can be expressed as:

wherein, y ₁ 、y ₂ Two vectors which need to be calculated are represented, namely a predicted value and a true value, and m represents the total data quantity. The final goal of the regression loss function is to minimize the absolute difference between the total predicted value and the true value.

In one embodiment, S90, acquiring a traffic scene image to be detected, performing key point prediction on each vehicle image to be detected extracted from the traffic scene image to be detected according to the trained key point detection model, and acquiring a posture center point, multiple offsets, a course angle sine value, and a course angle cosine value of each vehicle to be detected, includes:

s910, carrying out vehicle target detection on the traffic scene image to be detected according to a vehicle target detection algorithm to obtain two-dimensional detection frame information of each vehicle to be detected;

s920, extracting the traffic scene image to be detected according to the two-dimensional detection frame information to obtain a vehicle image to be detected of each vehicle to be detected;

s930, carrying out size transformation and key point data reconstruction on the vehicle image to be detected to obtain a reconstructed vehicle image to be detected of each vehicle to be detected, and carrying out random erasing data enhancement on the reconstructed vehicle image to be detected to obtain an enhanced vehicle image to be detected of each vehicle to be detected;

s940, the image of the enhanced vehicle to be tested is input into the trained key point detection model, and the attitude center point, a plurality of offsets, the course angle sine value and the course angle cosine value of each vehicle to be tested are output;

in this embodiment, the vehicle target detection algorithm can realize target detection of each vehicle in the traffic scene image to be detected, obtain two-dimensional detection frame information of each vehicle to be detected, and extract the vehicle image to be detected corresponding to each vehicle to be detected from the traffic scene image to be detected according to the two-dimensional detection frame information. The relevant description of the step S930 is the same as that of S30, except that the object of S930 is an image of the vehicle to be measured, and the image of the enhanced vehicle to be measured of each vehicle to be measured is obtained through size transformation, key point data reconstruction, and random erasure, respectively. In one embodiment, the vehicle target detection algorithm includes, but is not limited to, using a mainstream YOLO, SSD, etc. based target detection algorithm.

Therefore, the enhanced vehicle image to be tested is input into the trained key point detection model, and the attitude center point, the multiple offsets, the course angle sine value and the course angle cosine value of each vehicle to be tested are obtained. And obtaining a plurality of attitude key points of each vehicle to be detected according to the attitude center point and the plurality of offsets based on the calculation formula of the key points in the embodiment. And obtaining the course angle of each vehicle to be measured according to the course angle sine value and the course angle cosine value.

Therefore, the roadside parking management method based on the course angle posture is based on a top-down key point detection mode, a first-stage target detection result is obtained through a vehicle target detection algorithm, learning of a second-stage key point detection task is mainly conducted, in the process, the accuracy of vehicle key point detection is improved by utilizing regression of vehicle posture key points and regression of vehicle body course angles and adding a data enhancement method of random erasure, and meanwhile, the roadside parking management method based on the course angle posture has a good detection result under a scene with serious shielding.

In one embodiment, the S100, obtaining a plurality of attitude key points of each vehicle to be tested according to the attitude center point and a plurality of offsets, obtaining a heading angle of each vehicle to be tested according to a heading angle sine value and a heading angle cosine value, and performing roadside parking management according to the plurality of attitude key points and the heading angle, includes:

s101, performing world coordinate system coordinate transformation on a plurality of posture key points of each vehicle to be tested to obtain the position of each vehicle to be tested;

and S102, performing roadside parking management according to the position and the course angle of the vehicle to be detected.

In this embodiment, the coordinates of the plurality of pose key points are obtained in the image coordinate system. By converting the coordinates of the plurality of pose key points in the image coordinate system into coordinates in the world coordinate system, the specific coordinate position in the same world coordinate system as the berthing position can be obtained. The camera position of the roadside parking is fixed, the position of the roadside parking position can be obtained, the coordinate conversion is carried out on the multiple attitude key point positions of each vehicle based on the image coordinate system, the multiple attitude key point positions in the world coordinate system and the parking position in the same world coordinate system are obtained for judgment, the accuracy of vehicle body attitude judgment is further enhanced by combining the constraint of a course angle, whether the vehicle is parked in the parking position or not is judged, whether illegal parking behaviors such as line pressing parking, parking forbidden regions parking and the like exist or not is judged, therefore, the management and decision of the roadside parking are realized, and the roadside parking management is realized.

Referring to FIG. 3, in one embodiment, the present application provides a roadside parking management system 01 based on heading angle attitude. The roadside parking management system 01 based on the course angle posture comprises a data acquisition module 10, an image extraction module 20, a data enhancement module 30, a backbone network module 40, a feature aggregation network module 50, a key point prediction network module 60, a course angle regression network module 70, a model training module 80, a detection module 90 and a parking management module 100.

The data obtaining module 10 is configured to obtain a modeled simulated traffic scene image dataset, where the modeled simulated traffic scene image dataset includes a plurality of traffic scene images, and each traffic scene image is labeled with real information of a two-dimensional detection frame of each vehicle, a plurality of real attitude key points corresponding to a three-dimensional enclosure frame of each vehicle, a real attitude center point, a plurality of real offsets from the real attitude key points to the real attitude center point, a real heading angle of each vehicle, a real heading angle sine value, and a real heading angle cosine value. The image extraction module 20 is configured to extract each vehicle in each traffic scene image according to the two-dimensional detection frame real information, so as to obtain a vehicle image of each vehicle.

The data enhancement module 30 is configured to perform size transformation and key point data reconstruction on the vehicle images to obtain reconstructed vehicle images of each vehicle, and perform random erasure data enhancement on the reconstructed vehicle images to obtain enhanced vehicle images of each vehicle. The backbone network module 40 is configured to input the enhanced vehicle image into a backbone network of the key point detection model to perform feature extraction, so as to obtain a backbone feature map data set. The feature aggregation network module 50 is configured to input each of the backbone feature maps in the backbone feature map data set to a feature aggregation network of the keypoint detection model for feature fusion, so as to obtain an aggregated feature map data set.

The key point prediction network module 60 is configured to input each aggregated feature map in the aggregated feature map data set into a key point prediction network of the key point detection model to perform point location prediction, so as to obtain a predicted attitude center point and a plurality of predicted offsets of each vehicle. The course angle regression network module 70 is used for inputting each aggregation characteristic diagram in the aggregation characteristic diagram data set into the vehicle course angle regression network of the key point detection model to predict the course angle, obtaining the sine value of the predicted course angle and the cosine value of the predicted course angle of each vehicle, and obtaining the predicted course angle of each vehicle according to the sine value of the predicted course angle and the cosine value of the predicted course angle.

The model training module 80 is configured to construct a loss function of the keypoint detection model according to the predicted course angle, the predicted attitude center point, the plurality of predicted offsets, the real course angle, the real attitude center point, and the plurality of real offsets, and train and optimize the keypoint detection model according to the loss function to obtain a trained keypoint detection model. The detection module 90 is configured to acquire a traffic scene image to be detected, perform key point prediction on each image of a vehicle to be detected extracted from the traffic scene image to be detected according to the trained key point detection model, and acquire a posture center point, multiple offsets, a course angle sine value, and a course angle cosine value of each vehicle to be detected. The parking management module 100 is configured to obtain a plurality of attitude key points of each vehicle to be tested according to the attitude center point and the plurality of offsets, obtain a course angle of each vehicle to be tested according to a course angle sine value and a course angle cosine value, and perform roadside parking management according to the plurality of attitude key points and the course angle.

In this embodiment, reference may be made to the description of S10 in the above embodiment for the related description of the data obtaining module 10. The relevant description of the image extraction module 20 may refer to the relevant description of S20 in the above embodiment. The relevant description of the data enhancement module 30 can refer to the relevant description of S30 in the above embodiment. The relevant description of the backbone network module 40 may refer to the relevant description of S40 in the above embodiment. The relevant description of the feature aggregation network module 50 may refer to the relevant description of S50 in the above embodiment. The relevant description of the keypoint prediction network module 60 may refer to the relevant description of S60 in the above embodiment. The relevant description of the course angle regression network module 70 can refer to the relevant description of the S70 in the above embodiment. The relevant description of the model training module 80 can refer to the relevant description of S80 in the above embodiment. The relevant description of the detection module 90 may refer to the relevant description of S90 in the above embodiment. The relevant description of the parking management module 100 may refer to the relevant description of S100 in the above embodiment.

In one embodiment, the data enhancement module 30 includes a size transformation module, a pixel fill module, and a keypoint data reconstruction module. The size conversion module is used for setting the original length of the vehicle image to be 256 pixels, and converting the original width of the vehicle image according to the length conversion ratio to obtain the new width of the vehicle image. And the pixel filling module is used for filling the new width of the vehicle image by 0 pixel to 256 pixels to obtain a size conversion image. And the key point data reconstruction module is used for transforming the plurality of real attitude key points, the real attitude central points and the plurality of real offsets according to the transformation ratio of the size transformation image and the original size of the vehicle image to obtain a reconstructed vehicle image.

In this embodiment, the relevant description of the size transformation module may refer to the relevant description of S310 in the above embodiment. The related description of the pixel filling module can refer to the related description of S320 in the above embodiment. The relevant description of the key point data reconstruction module may refer to the relevant description of S330 in the above embodiment.

In one embodiment, the model training module 80 includes a first regression loss function module, a second regression loss function module, a third regression loss function module, and a total loss function module. The first regression loss function module is used for constructing a regression loss function of the vehicle course angle according to the predicted course angle and the real course angle of each vehicle. And the second regression loss function module is used for constructing a regression loss function of the vehicle attitude center point according to the predicted attitude center point and the real attitude center point of each vehicle. And the third regression loss function module is used for constructing a regression loss function of the offset of the vehicle attitude key point according to the plurality of predicted offsets and the plurality of real offsets of each vehicle. The total loss function module is used for constructing a loss function of the key point detection model according to a regression loss function of the vehicle course angle, a regression loss function of the vehicle attitude central point and a regression loss function of the offset of the vehicle attitude key point.

In this embodiment, reference may be made to the description of S810 in the above embodiment for a related description of the first regression loss function module. The description of the second regression loss function module may refer to the description of S820 in the above embodiment. The relevant description of the third regression loss function module may refer to the relevant description of S830 in the above embodiment. The description of the total loss function module can refer to the description of S840 in the above embodiment.

In one embodiment, the detection module 90 includes a two-dimensional detection frame information acquisition module, a vehicle image acquisition module to be detected, an enhanced vehicle image acquisition module to be detected, and a vehicle attitude acquisition module to be detected. The two-dimensional detection frame information acquisition module is used for carrying out vehicle target detection on the traffic scene image to be detected according to a vehicle target detection algorithm to acquire two-dimensional detection frame information of each vehicle to be detected. The to-be-detected vehicle image acquisition module is used for matting the to-be-detected traffic scene image according to the two-dimensional detection frame information to obtain the to-be-detected vehicle image of each to-be-detected vehicle. The to-be-detected enhanced vehicle image acquisition module is used for carrying out size transformation and key point data reconstruction on the to-be-detected vehicle image to obtain a to-be-detected reconstructed vehicle image of each to-be-detected vehicle, and carrying out random data erasure enhancement on the to-be-detected reconstructed vehicle image to obtain an to-be-detected enhanced vehicle image of each to-be-detected vehicle. The to-be-tested vehicle attitude acquisition module is used for inputting the to-be-tested enhanced vehicle image into the trained key point detection model and outputting an attitude center point, a plurality of offsets, a course angle sine value and a course angle cosine value of each to-be-tested vehicle.

In this embodiment, the relevant description of the two-dimensional detection frame information obtaining module may refer to the relevant description of S910 in the above embodiment. The related description of the image acquiring module of the vehicle to be tested can refer to the related description of S920 in the above embodiment. The relevant description of the enhanced vehicle image acquisition module to be tested can refer to the relevant description of S930 in the above embodiment. The relevant description of the vehicle attitude acquisition module to be tested can refer to the relevant description of S940 in the above embodiment.

In one embodiment, the parking management module 100 includes a coordinate transformation module and a management module. The coordinate conversion module is used for performing world coordinate system coordinate conversion on the plurality of posture key points of each vehicle to be tested to obtain the position of each vehicle to be tested. The management module is used for performing roadside parking management according to the position and the course angle of the vehicle to be tested.

In this embodiment, the relevant description of the coordinate conversion module may refer to the relevant description of S101 in the above embodiment. The relevant description of the management module may refer to the relevant description of S102 in the above embodiment.

In the various embodiments described above, the particular order or hierarchy of steps in the processes disclosed is an example of exemplary approaches. Based upon design preferences, it is understood that the specific order or hierarchy of steps in the processes may be rearranged without departing from the scope of the present disclosure. The accompanying method claims present elements of the various steps in a sample order, and are not intended to be limited to the specific order or hierarchy.

Those of skill in the art will also appreciate that the various illustrative logical blocks, modules, and steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, modules, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design requirements of the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the embodiments of the present application.

The various illustrative logical blocks, or modules, described in the embodiments herein may be implemented or operated by a general purpose processor, a digital signal processor, an Application Specific Integrated Circuit (ASIC), a field programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described. A general-purpose processor may be a microprocessor, but, in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a digital signal processor and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a digital signal processor core, or any other similar configuration.

The steps of a method or algorithm described in the embodiments herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may be stored in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. For example, a storage medium may be coupled to the processor such that the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC, which may be disposed in a user terminal. In the alternative, the processor and the storage medium may reside as discrete components in a user terminal.

The above-mentioned embodiments, objects, technical solutions and advantages of the present application are described in further detail, it should be understood that the above-mentioned embodiments are merely exemplary embodiments of the present application, and are not intended to limit the scope of the present application, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present application should be included in the scope of the present application.

Claims

1. A roadside parking management method based on course angle postures is characterized by comprising the following steps:

acquiring a modeling simulation traffic scene image dataset which comprises a plurality of traffic scene images, wherein each traffic scene image is marked with real information of a two-dimensional detection frame of each vehicle, a plurality of real attitude key points corresponding to a three-dimensional surrounding frame of each vehicle, a real attitude central point, a plurality of real offsets from the real attitude key points to the real attitude central point, a real course angle of each vehicle, a real course angle sine value and a real course angle cosine value;

inputting each aggregated feature map in the aggregated feature map data set into a key point prediction network of the key point detection model for point location prediction to obtain a predicted attitude center point and a plurality of predicted offsets of each vehicle;

constructing a loss function of the key point detection model according to the predicted course angle, the predicted attitude central point, the plurality of predicted offsets, the real course angle, the real attitude central point and the plurality of real offsets, and training and optimizing the key point detection model according to the loss function to obtain a trained key point detection model;

2. The heading angle attitude-based roadside parking management method according to claim 1, wherein the performing size transformation and key point data reconstruction on the vehicle image to obtain a reconstructed vehicle image of each vehicle, and performing random erasure data enhancement on the reconstructed vehicle image to obtain an enhanced vehicle image of each vehicle comprises:

3. The roadside parking management method based on the heading angle pose as recited in claim 1, wherein the constructing a loss function of the key point detection model according to the predicted heading angle, the predicted pose center point, the plurality of predicted offsets, the real heading angle, the real pose center point and the plurality of real offsets, and performing training optimization on the key point detection model according to the loss function to obtain a trained key point detection model comprises:

4. The roadside parking management method based on the course angle attitude of claim 1, wherein the obtaining of the traffic scene image to be tested, the predicting of the key points of each vehicle image to be tested extracted from the traffic scene image to be tested according to the trained key point detection model, and the obtaining of the attitude center point, the plurality of offsets, the course angle sine value and the course angle cosine value of each vehicle to be tested comprises:

5. The roadside parking management method based on the heading angle attitude of claim 1, wherein the obtaining a plurality of attitude key points of each vehicle to be tested according to the attitude center point and the plurality of offsets, obtaining the heading angle of each vehicle to be tested according to the sine value of the heading angle and the cosine value of the heading angle, and performing roadside parking management according to the plurality of attitude key points and the heading angle comprises:

performing coordinate conversion of a world coordinate system on the plurality of posture key points of each vehicle to be detected to obtain the position of each vehicle to be detected;

6. A roadside parking management system based on course angle attitude, comprising:

the characteristic aggregation network module is used for inputting each backbone characteristic diagram in the backbone characteristic diagram data set into a characteristic aggregation network of the key point detection model for characteristic fusion to obtain an aggregation characteristic diagram data set;

7. The heading angle attitude based roadside parking management system of claim 6 wherein the data enhancement module comprises:

the size conversion module is used for setting the original length of the vehicle image to be 256 pixels and converting the original width of the vehicle image according to the length conversion ratio to obtain the new width of the vehicle image;

8. The course angle pose based roadside parking management system of claim 6 wherein the model training module comprises:

the second regression loss function module is used for constructing a regression loss function of the vehicle attitude center point according to the predicted attitude center point and the real attitude center point of each vehicle;

9. The heading angle attitude based roadside parking management system of claim 6 wherein the detection module comprises:

the to-be-detected vehicle image acquisition module is used for matting the to-be-detected traffic scene image according to the two-dimensional detection frame information to obtain to-be-detected vehicle images of each to-be-detected vehicle;

10. The course angle pose based roadside parking management system of claim 6, wherein the parking management module comprises:

the coordinate conversion module is used for performing world coordinate system coordinate conversion on a plurality of posture key points of each vehicle to be tested to obtain the position of each vehicle to be tested;