CN114627069A - Method for acquiring wheel touchdown point, storage medium, and electronic apparatus - Google Patents

Method for acquiring wheel touchdown point, storage medium, and electronic apparatus Download PDF

Info

Publication number
CN114627069A
CN114627069A CN202210232479.6A CN202210232479A CN114627069A CN 114627069 A CN114627069 A CN 114627069A CN 202210232479 A CN202210232479 A CN 202210232479A CN 114627069 A CN114627069 A CN 114627069A
Authority
CN
China
Prior art keywords
image
target
wheel
point
substructure
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210232479.6A
Other languages
Chinese (zh)
Inventor
杨一帆
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to CN202210232479.6A priority Critical patent/CN114627069A/en
Publication of CN114627069A publication Critical patent/CN114627069A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle

Abstract

The application discloses a method for acquiring a wheel touchdown point, a storage medium and an electronic device, which can be applied to the field of automatic driving. Wherein, the method comprises the following steps: under the condition that the candidate identification images are acquired and at least one target vehicle is identified from the candidate identification images, determining a wheel touchdown center point of each target vehicle according to the image characteristics of the candidate identification images; acquiring at least two image substructures corresponding to the candidate identification images; determining a target image substructure corresponding to each target vehicle from the at least two image substructures according to an image region where the wheel touchdown center point is located; positional information of the wheel touchdown point of each target vehicle is acquired using the target image substructure. The technical problem that an obtaining mode of effective wheel touchdown point information is lacked is solved.

Description

Method for acquiring wheel contact point, storage medium, and electronic device
Technical Field
The present application relates to the field of computers, and in particular, to a method for acquiring a wheel contact point, a storage medium, and an electronic device.
Background
In recent years, the application of wheel touchdown points is more extensive, but the current acquisition mode of the wheel touchdown point information is not mature enough, and various problems exist, so that the wheel touchdown point information cannot be directly taken to execute a detection task. Therefore, there is a problem that an effective way of acquiring the wheel contact point information is lacking.
In view of the above problems, no effective solution has been proposed.
Disclosure of Invention
The embodiment of the application provides a method for acquiring a wheel touchdown point, a storage medium and electronic equipment, which are used for solving the technical problem that an effective method for acquiring wheel touchdown point information is lacked.
According to an aspect of an embodiment of the present application, there is provided a method for acquiring a wheel touchdown point, including: when a candidate identification image is acquired and at least one target vehicle is identified from the candidate identification image, determining a wheel touchdown center point of each target vehicle according to the image characteristics of the candidate identification image, wherein the wheel touchdown center point is used for indicating the position information of the centers of at least two wheel touchdown points of the target vehicle in the candidate identification image; acquiring at least two image substructures corresponding to candidate identification images, wherein each image substructure in the at least two image substructures corresponds to at least one image area in the candidate identification images, and the image substructures are used for identifying image content information of the candidate identification images to acquire position information of the wheel touch points; determining a target image substructure corresponding to each of the target vehicles from the at least two image substructures according to the image region in which the wheel touchdown center point is located; and acquiring the position information of the wheel touch point of each target vehicle by using the target image substructure.
According to another aspect of the embodiments of the present application, there is provided a wheel contact point acquiring device, including a first determining unit configured to determine a wheel contact point of each target vehicle according to an image feature of a candidate recognition image when the candidate recognition image is acquired and at least one target vehicle is recognized from the candidate recognition image, wherein the wheel contact point is used to indicate position information of centers of at least two wheel contact points of the target vehicle in the candidate recognition image; a first obtaining unit, configured to obtain at least two image substructures corresponding to candidate identification images, where each of the at least two image substructures corresponds to at least one image area in the candidate identification image, and the image substructures are configured to identify image content information of the candidate identification images to obtain position information of the wheel touchdown point; a second determining unit configured to determine, from the at least two image substructures, a target image substructure corresponding to each of the target vehicles, based on the image region in which the wheel contact center point is located; a second acquisition unit configured to acquire position information of the wheel touchdown point of each of the target vehicles using the target image substructure.
As an optional solution, the second determining unit includes: a first obtaining module, configured to obtain an image region set corresponding to the at least two image substructures, where the image region set includes at least two image regions in the candidate recognition image; a second obtaining module, configured to obtain a target image area where the wheel contact center point corresponding to each of the target vehicles is located, where the at least two image areas include the target image area; a first determining module, configured to determine the target image substructure corresponding to each of the target image areas from the at least two image substructures.
As an optional solution, the first obtaining module includes: a first obtaining sub-module, configured to obtain a first region set corresponding to at least two first image sub-structures, where the first region set includes at least two first image regions in the candidate recognition image, and the first image regions correspond to the first image sub-structures; and a second obtaining sub-module, configured to obtain a second region set corresponding to at least two second image substructures, where the second region set includes at least two second image regions in the candidate recognition image, the second image regions correspond to the second image substructures, and a region range of the second image regions is greater than a region range of the first image regions.
As an optional solution, the first determining module includes: a first determining sub-module, configured to determine a first target sub-structure and a second target sub-structure corresponding to each of the target image regions, where the target image sub-structures include the first target sub-structure and the second target sub-structure; or, a second determining sub-module, configured to determine the target first sub-structure corresponding to each of the target image regions when the space occupation amount of the target vehicle is smaller than a first threshold, and determine the target second sub-structure corresponding to each of the target image regions when the space occupation amount of the target vehicle is greater than or equal to the first threshold.
As an optional solution, the second obtaining unit includes: a third obtaining module, configured to obtain a region center point of each target image region; a calculation module, configured to calculate a target offset between each of the area center points and the wheel contact points of the corresponding target vehicle using the target image substructure; a fourth obtaining module, configured to obtain, according to each of the target offset amounts and area attribute information of the corresponding target image area, position information of the wheel touchdown point of each of the target vehicles.
As an optional solution, the apparatus further includes: an input unit, configured to input the candidate recognition image into an image recognition model, where the image recognition model is a neural network model trained using a plurality of sample image data and used for recognizing position information of the wheel touchdown point in the image; a third obtaining unit, configured to obtain a recognition result output by the image recognition model, where the recognition result includes position information of the wheel touchdown point of each of the target vehicles in the candidate recognition images.
As an alternative, the method comprises the following steps: a fourth acquiring unit configured to acquire the plurality of sample image data before the candidate recognition image is input to the image recognition model; a first marking unit configured to mark image data indicating the wheel contact point in each of the sample image data before the candidate recognition image is input to the image recognition model, and obtain a plurality of marked sample image data, where each marked sample image data includes a marked wheel contact point identifier; a first training unit, configured to input the marked sample image data into an initial image recognition model before the candidate recognition image is input into the image recognition model, so as to obtain the image recognition model through training.
As an optional solution, the first training unit includes: a first repeating module, configured to repeatedly perform the following steps until the image recognition model is obtained: a second determining module, configured to determine current sample image data from the marked multiple sample image data, and determine a current image recognition model, where the current sample image data includes a marked current wheel contact identifier; a first recognition module, configured to recognize, through the current image recognition model, first result data indicating the wheel contact center point in the current sample image data; a first processing module, configured to process the first result data through the current image recognition model to obtain second result data indicating position information of the wheel contact point; a second processing module, configured to, when the second result data does not meet the recognition convergence condition, obtain next sample image data as the current sample image data; and a third processing module, configured to determine that the current image recognition model is the image recognition model when the second result data reaches the recognition convergence condition.
As an alternative, the method comprises the following steps: a fifth acquiring unit configured to acquire the plurality of sample image data before the candidate recognition image is input to the image recognition model; a second labeling unit configured to label, before the candidate recognition image is input to the image recognition model, first image data indicating the wheel contact point and second image data indicating the wheel contact center point for each of the sample image data to obtain a plurality of labeled sample image data, wherein each of the labeled sample image data includes a labeled wheel contact point identifier and a labeled wheel contact center point identifier; and a second training unit configured to input the marked sample image data into an initial image recognition model before the candidate recognition image is input into the image recognition model, so as to obtain the image recognition model through training.
As an optional solution, the second training unit includes: a second repeating module, configured to repeatedly execute the following steps until the image recognition model is obtained: a third determining module, configured to determine current sample image data from the marked multiple sample image data, and determine a current image recognition model, where the current sample image data includes a marked current wheel contact identifier; a second recognition module, configured to recognize, through the current image recognition model, first result data indicating the wheel contact center point in the current sample image data; a fourth processing module, configured to, when the first result data does not reach a second convergence condition, obtain next sample image data as the current sample image data; a fifth processing module, configured to process the first result data through the current image recognition model when the first result data reaches the second convergence condition, to obtain second result data indicating position information of the wheel contact point; a sixth processing module, configured to, when the second result data does not reach a second convergence condition, obtain next sample image data as the current sample image data; a seventh processing module, configured to determine that the current image recognition model is the image recognition model when the second result data reaches the second convergence condition.
As an optional solution, the second obtaining unit includes at least one of: a first location module for obtaining first location information of the wheel touchdown point visible to each of the target vehicles using the target image substructure; a second location module to obtain second location information of the wheel touchdown point that is invisible to each of the target vehicles using the target image substructure.
As an optional solution, the second obtaining unit includes: a fifth obtaining module, configured to obtain, by using the target image substructure, a respective touch point set corresponding to each of the target vehicles, where the touch point set includes a plurality of wheel touch points; the above-mentioned device includes: determining a target touch point set from the respective touch point sets corresponding to each of the target vehicles after the position information of the wheel touch points of each of the target vehicles is obtained by using the target image substructure, wherein the number of the wheel touch points in the target touch point set is greater than a second threshold; and screening the wheel contact points in the target contact point set by adopting a non-maximum value inhibition mode, wherein the number of the wheel contact points in the screened target contact point set is less than or equal to the second threshold value.
According to yet another aspect of embodiments herein, there is provided a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the acquisition method of the wheel touchdown point as above.
According to another aspect of the embodiments of the present application, there is also provided an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the method for acquiring the wheel touch point through the computer program.
In the embodiment of the present application, when a candidate identification image is acquired and at least one target vehicle is identified from the candidate identification image, a wheel contact center point of each target vehicle is determined according to an image feature of the candidate identification image, wherein the wheel contact center point is used for indicating position information of centers of at least two wheel contact points of the target vehicle in the candidate identification image; acquiring at least two image substructures corresponding to candidate identification images, wherein each image substructure in the at least two image substructures corresponds to at least one image area in the candidate identification images, and the image substructures are used for identifying image content information of the candidate identification images to acquire position information of the wheel touch points; determining a target image substructure corresponding to each of the target vehicles from the at least two image substructures according to the image region in which the wheel touchdown center point is located; obtaining position information of the wheel touchdown point of each of the target vehicles using the target image substructure; according to the embodiment of the application, the position information of the wheel touch point of each target vehicle is obtained by using the target image substructure, the target image substructure corresponding to each vehicle is determined by using the wheel touch point through the anchor point mechanism, then the target image substructure is used for identifying the wheel touch point with stronger pertinence to the corresponding vehicle, and the information association relationship is established between the identified position information of the wheel touch point and the corresponding vehicle, so that the position information of the wheel touch point is more comprehensive; in addition, the target image substructure corresponding to each target vehicle is determined from the plurality of image substructures by using the wheel touchdown center point, so that the application efficiency of the image substructures is improved, and the technical problem that an effective acquisition mode of wheel touchdown point information is lacked is solved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
fig. 1 is a schematic diagram of an application environment of an alternative wheel touchdown point acquisition method according to an embodiment of the present application;
FIG. 2 is a schematic diagram illustrating a flow of an alternative method for obtaining a wheel touchdown point according to an embodiment of the present application;
FIG. 3 is a schematic diagram of an alternative method of obtaining a wheel touchdown point according to an embodiment of the present application;
FIG. 4 is a schematic diagram of another alternative wheel touchdown point acquisition method according to an embodiment of the present application;
FIG. 5 is a schematic diagram of another alternative wheel touchdown point acquisition method according to an embodiment of the present application;
FIG. 6 is a schematic diagram of an alternative wheel contact point acquisition method according to an embodiment of the present application;
FIG. 7 is a schematic diagram of another alternative wheel touchdown point acquisition method according to an embodiment of the present application;
FIG. 8 is a schematic diagram of another alternative wheel touchdown point acquisition method according to an embodiment of the present application;
FIG. 9 is a schematic diagram of an alternative wheel touchdown point acquisition device according to an embodiment of the present application;
fig. 10 is a schematic structural diagram of an alternative electronic device according to an embodiment of the application.
Detailed Description
In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only partial embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the accompanying drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
According to an aspect of the embodiments of the present application, there is provided a method for acquiring a wheel touchdown point, which may be applied, but not limited, to the environment shown in fig. 1 as an optional implementation. The user device 102 may include, but is not limited to, a display 108, a processor 106, and a memory 104, the network 110 may include, but is not limited to, the user device 102, and the specific user device 102 may be, but is not limited to, an in-vehicle camera understood as a vehicle for shooting a vehicle, and the in-vehicle camera includes an in-vehicle camera display screen (the display 108) and an in-vehicle camera lens for capturing an image.
The specific process comprises the following steps:
step S102, the user equipment 102 acquires a candidate identification image containing a target vehicle through a vehicle-mounted camera lens;
steps S104-S106, user device 102 sends candidate recognition images to server 112 via network 110;
step S108, the server 112 searches for at least two image substructures corresponding to the candidate recognition images through the database 114, determines a target image substructure corresponding to each target vehicle from the at least two image substructures through the processing engine 116, and processes the candidate recognition images by using the target image substructures, so as to obtain position information of wheel contact points of each target vehicle;
the server 112 transmits the position information of the wheel contact point to the user device 102 through the network 110, and the processor 106 in the user device 102 displays the position information of the wheel contact point in the display 108 and stores the position information of the wheel contact point in the memory 104, steps S110-S112.
In addition to the example shown in fig. 1, the above steps may be performed by the user device 102 independently, that is, the steps of processing the image, acquiring the position information of the wheel contact point, and the like are performed by the user device 102, thereby relieving the processing pressure of the server. The user equipment 102 includes, but is not limited to, a handheld device (e.g., a mobile phone), a notebook computer, a desktop computer, a vehicle-mounted device, and the like, and the application does not limit the specific implementation manner of the user equipment 102.
Optionally, as an alternative implementation, as shown in fig. 2, the method for acquiring the wheel touchdown point includes:
s202, under the condition that a candidate identification image is obtained and at least one target vehicle is identified from the candidate identification image, determining a wheel touchdown center point of each target vehicle according to the image characteristics of the candidate identification image, wherein the wheel touchdown center point is used for representing the position information of the centers of at least two wheel touchdown points of the target vehicle in the candidate identification image;
s204, acquiring at least two image substructures corresponding to the candidate identification images, wherein each image substructure of the at least two image substructures corresponds to at least one image area in the candidate identification images, and the image substructures are used for identifying image content information of the candidate identification images to acquire position information of the wheel touchdown points;
s206, determining a target image substructure corresponding to each target vehicle from the at least two image substructures according to the image area where the wheel touchdown center point is located;
and S208, acquiring the position information of the wheel touch point of each target vehicle by using the target image substructure.
Alternatively, in this embodiment, the wheel touchdown point acquisition method can be applied, but not limited to, in a real-time end-to-end wheel touchdown point detection scenario, for example, building a detection model (image sub-structure) based on ultra-lightweight convolutional neural networks, using a method based on key points (wheel touchdown center points) to detect wheel touchdown points, supporting occluded touchdown point prediction, and because the structure selection mode based on the anchor point mechanism is adopted to determine the detection model (the target image substructure) corresponding to each target vehicle, the processing efficiency of the detection model is improved, for example a target vehicle only needs to be processed using a partial detection submodel of the overall detection model, therefore, the deployment (such as a vehicle machine) can be completed on the CPU equipment with low computing power, and the contact points of the wheels of other vehicles in the current driving environment can be further detected in real time.
Optionally, in this embodiment, the wheel grounding point may be, but is not limited to, helping the auto-driven vehicle to obtain a more accurate relative position, direction and size of the vehicle, and is of great significance to the improvement of safety during driving; for example, after acquiring the position information of the wheel touch point of each target vehicle by using the target image substructure, the spatial information of the target vehicle is determined by using the position information of the wheel touch point, wherein the spatial information may include, but is not limited to, at least one of the following: relative position of the vehicle, relative direction of the vehicle, size of the vehicle, travel path of the vehicle, etc.
Optionally, in this embodiment, the method for acquiring the wheel touchdown point may also be applied, but not limited to, in a scene where other vehicles are relatively accurately detected with respect to relative distance, for example, a center (pixel position) corresponding to the obtained touchdown point is calculated based on the obtained touchdown point, and then the calculated touchdown point is converted into a world coordinate position in a vehicle body coordinate system through a camera parameter calibrated in advance and a ground assumption, so that ADAS warning, lane-level vehicle rendering, and the like can be helped to acquire accurate position information of other vehicles.
Optionally, in this embodiment, the method for acquiring the wheel touch point may also be applied, but not limited to, in a scene where a monocular camera performs vehicle 3D target detection, for example, based on the acquired touch point, in combination with 2D target detection and camera calibration parameters, 3D target detection information of a vehicle may be generated, where the information includes a center position (x, y, z), a length, a width, a height, a heading angle, and the like.
Optionally, in this embodiment, the candidate identification image may be acquired by, but not limited to, acquiring an image of a vehicle during operation by using a vehicle-mounted image acquisition device, or acquiring a road condition image by using a road side camera.
Optionally, in this embodiment, the image feature may be understood as, but not limited to, a feature presented by an image pixel in the candidate recognition image, or may be understood as a feature recognized by an image recognition technology, such as a color feature, a shape feature, an image local feature, an image depth feature, an edge feature, a linear feature, a texture feature, and the like; determining the wheel touchdown center of each target vehicle according to the image features of the candidate recognition images can be, but is not limited to, accomplished by using an image recognition model, for example, an initial image recognition model is determined first, and then labeled sample image data is input for training to obtain a trained image recognition model, wherein the labeled sample image data includes image data of the target vehicle labeled with key points, and the key points include wheel touchdown points and specific attribute data of the wheel touchdown points, such as left front wheel, left rear wheel, right front wheel and right rear wheel, or visible wheel touchdown points, invisible wheel touchdown points, and the like.
Alternatively, in this embodiment, the wheel touchdown center may be, but is not limited to be, a center of at least two wheel touchdown points of the target vehicle, and since at least two wheel touchdown points may include a wheel touchdown point that is not currently visible, in this case, the position information of the wheel touchdown center may be, but is not limited to, predicted first by predicting the position information of the wheel touchdown point that is not currently visible, and then by predicting the position information of the wheel touchdown center by combining the position information of the wheel touchdown point that is visible.
Optionally, in this embodiment, the target image model may include, but is not limited to, at least two image substructures, where each of the at least two image substructures corresponds to at least one image region in the candidate identification image, or it may be understood that each image substructure corresponds to at least one image region in the candidate identification image, and thus the corresponding image substructures may be directly determined in a case where the image regions are determined in advance, or conversely, the corresponding image regions may also be directly determined in a case where the image substructures are determined in advance; in addition, the image substructure is used for identifying the image content information of the candidate identification image to obtain the position information of the wheel touchdown point, or it can be understood that the image substructure corresponds to the image area, but the image substructure is used for identifying the whole image content information of the candidate identification image, or is not limited to the image content information of the image area corresponding to the image substructure, but is not limited to having stronger correlation with the image content information of the corresponding image area, or having higher identification accuracy on the image content information of the corresponding image area, thereby improving the identification accuracy of the wheel touchdown point.
Optionally, in this embodiment, the target image substructure corresponding to each target vehicle is determined from the at least two image substructures according to the image region where the wheel touchdown center point is located, or it may be understood that the target image region where the wheel touchdown center point is located is determined from the plurality of image regions, and then the target image substructure corresponding to the target image region is determined from the at least two image substructures by using the correspondence between the image regions and the image substructures; in addition, since the target image area is determined by the wheel touchdown center point, which is clearly the target vehicle to which the wheel touchdown center point belongs, when the target image substructure is determined, the target vehicle corresponding to the target image substructure can be obtained, or it can be understood that the target image substructure and the target vehicle have a corresponding relationship.
Optionally, in this embodiment, the target image substructure is used to obtain the position information of the wheel touch point of each target vehicle, for example, a target image region in the candidate recognition image is determined by using a correspondence between the target image substructure and the image region, and then a region center point of the target image region is determined; and further utilizing an anchor point mechanism, calculating the position offset between the wheel touchdown point of each target vehicle and the central point of the area by adopting a target image substructure, and calculating the actual position information of the wheel touchdown point of each target vehicle according to the position offset.
It should be noted that, the position information of the wheel touch point of each target vehicle is acquired by using the target image substructure, the anchor point mechanism is used to determine the target image substructure corresponding to each vehicle through the wheel touch point, then the target image substructure is used to identify the wheel touch point with stronger pertinence to the corresponding vehicle, and the information association relationship is established between the identified wheel touch point and the corresponding vehicle, so that the position information of the wheel touch point becomes more comprehensive; in addition, the target image substructures corresponding to each target vehicle are determined from the plurality of image substructures by using the wheel touchdown center point, so that the application efficiency of the image substructures is improved, and the method for acquiring the wheel touchdown points can be realized in application scenes with low calculation force, such as application scenes of vehicle-mounted equipment.
By way of further example, alternatively, as shown in fig. 3, in a case where the candidate identification image 302 is acquired and at least one target vehicle (e.g., target vehicle 304-1, target vehicle 304-2) is identified from the candidate identification image 302, wheel contact-down center points (e.g., wheel contact-down center point 306-1, wheel contact-down center point 306-2) of each target vehicle are determined according to image features of the candidate identification image 302, where the wheel contact-down center points are used to represent position information of centers of at least two wheel contact-down points of the target vehicle in the candidate identification image 302;
furthermore, at least two image substructures (e.g., image substructures 308-1, 308-2, 308-3) corresponding to the candidate identification image 302 are obtained, wherein each image substructures of the at least two image substructures corresponds to at least one image area in the candidate identification image 302, and the image substructures are used for identifying image content information of the candidate identification image 302 to obtain position information of the wheel touch point; according to the image area where the wheel touchdown center point is located, determining a target image substructure corresponding to each target vehicle from at least two image substructures, such as an image substructure 308-1 corresponding to the wheel touchdown center point 306-2 and an image substructure 308-3 corresponding to the wheel touchdown center point 306-1;
further acquiring position information of the wheel contact point of each target vehicle using the target image substructure, such as acquiring position information of the wheel contact point 310-2 of the target vehicle 304-2 using the image substructure 308-1, and acquiring position information of the wheel contact point 310-1 of the target vehicle 304-1 using the image substructure 308-3; in addition, to improve the comprehensiveness of the position information of the wheel touchdown points, additional attribute information may also be provided, but not limited to, as shown in fig. 3 in which a solid-line circle represents a visible wheel touchdown point and a dashed-line circle represents an invisible wheel touchdown point.
By the embodiment provided by the application, under the condition that the candidate identification image is acquired and at least one target vehicle is identified from the candidate identification image, the wheel touchdown center point of each target vehicle is determined according to the image characteristics of the candidate identification image, wherein the wheel touchdown center point is used for representing the position information of the centers of at least two wheel touchdown points of the target vehicles in the candidate identification image; acquiring at least two image substructures corresponding to the candidate identification images, wherein each image substructure of the at least two image substructures corresponds to at least one image area in the candidate identification images, and the image substructures are used for identifying image content information of the candidate identification images to acquire position information of the wheel touchdown point; determining a target image substructure corresponding to each target vehicle from the at least two image substructures according to an image area where the wheel touchdown center point is located; acquiring position information of a wheel touch point of each target vehicle by using the target image substructure; according to the embodiment of the application, the position information of the wheel touch point of each target vehicle is obtained by using the target image substructure, the target image substructure corresponding to each vehicle is determined by using the wheel touch point through the anchor point mechanism, then the target image substructure is used for identifying the wheel touch point with stronger pertinence to the corresponding vehicle, and the information association relationship is established between the identified position information of the wheel touch point and the corresponding vehicle, so that the position information of the wheel touch point is more comprehensive; in addition, the wheel touchdown center point is used for determining the target image substructure corresponding to each target vehicle from the plurality of image substructures, and the application efficiency of the image substructures is improved.
As an alternative, determining a target image substructure corresponding to each target vehicle from the at least two image substructures according to an image region where the wheel contact center point is located includes:
s1, acquiring an image area set corresponding to at least two image substructures, wherein the image area set comprises at least two image areas in the candidate identification image;
s2, acquiring target image areas where wheel touchdown center points corresponding to each target vehicle are located, wherein at least two image areas comprise target image areas;
s3, determining a target image sub-structure corresponding to each target image area from the at least two image sub-structures.
Optionally, in this embodiment, the target image substructure corresponding to each target vehicle is determined from the at least two image substructures according to the image region where the wheel touchdown center point is located, or it may be understood that the target image region where the wheel touchdown center point is located is determined from the plurality of image regions, and then the target image substructure corresponding to the target image region is determined from the at least two image substructures by using the correspondence between the image regions and the image substructures; in addition, since the target image area is determined by the wheel touchdown center point, which is clearly the target vehicle to which the wheel touchdown center point belongs, when the target image substructure is determined, the target vehicle corresponding to the target image substructure can be obtained, or it can be understood that the target image substructure and the target vehicle have a corresponding relationship.
For further example, optionally, as shown in fig. 4, an image region set corresponding to at least two image substructures is obtained, where the image region set includes at least two image regions (for example, 12 grid-shaped structures) in the candidate recognition image 402; acquiring a target image area where a wheel touchdown center point corresponding to each target vehicle is located, for example, determining a target image area 408 where a vehicle touchdown center point 406 is located from at least two image areas; further, a target image substructure corresponding to each target image region 408 is determined from the at least two image substructures using the correspondence between the image regions and the image substructures.
By the embodiment provided by the application, the image region set corresponding to at least two image substructures is obtained, wherein the image region set comprises at least two image regions in the candidate identification image; acquiring target image areas where wheel touchdown center points corresponding to each target vehicle are located, wherein at least two image areas comprise the target image areas; and determining the target image substructure corresponding to each target image area from the at least two image substructures, thereby realizing the effect of improving the acquisition efficiency of the target image substructures.
As an alternative, acquiring a set of image regions corresponding to at least two image substructures includes:
s1, acquiring a first region set corresponding to at least two first image substructures, wherein the first region set comprises at least two first image regions in the candidate recognition image, and the first image regions correspond to the first image substructures;
s2, obtaining a second region set corresponding to at least two second image substructures, where the second region set includes at least two second image regions in the candidate recognition image, the second image regions correspond to the second image substructures, and a region range of the second image regions is greater than a region range of the first image regions.
Optionally, in this embodiment, the image sub-structures may be, but not limited to, image structures classified into multiple types, in which, in consideration of more types of target vehicles serving as recognition objects, the corresponding image structures are selected according to different types of the target vehicles to perform targeted processing, for example, if the target vehicle is recognized as a type a, the image sub-structure corresponding to the type a is used to process the target vehicle;
in addition, optionally, in order to improve the image processing efficiency, but not limited to, the image substructures may be differentiated by the area range of the image area, for example, the cart is subjected to image processing by using the image substructures corresponding to the image area with the smaller area range, and the cart is subjected to image processing by using the image substructures corresponding to the image area with the larger area range.
It should be noted that a first region set corresponding to at least two first image substructures is obtained, where the first region set includes at least two first image regions in the candidate recognition image, and the first image regions correspond to the first image substructures; and acquiring a second region set corresponding to at least two second image substructures, wherein the second region set comprises at least two second image regions in the candidate recognition image, the second image regions correspond to the second image substructures, and the region range of the second image regions is larger than that of the first image regions.
Further by way of example, optionally based on the scene shown in fig. 4, continuing to be shown in fig. 5, a first region set corresponding to at least two first image substructures is obtained, where the first region set includes at least two first image regions (a grid with a smaller region range and a number of 4 × 8) in the candidate recognition image, and the first image regions correspond to the first image substructures; further determining a target image area 502 in which the wheel contact center point 406 is located from the at least two first image areas;
optionally, for example, as shown in fig. 4, a second region set corresponding to at least two second image substructures is obtained, where the second region set includes at least two second image regions (a grid with a smaller region range and a number of 3 × 4) in the candidate recognition image, the second image regions correspond to the second image substructures, and a region range of the second image regions is greater than a region range of the first image regions; a target image region 408 in which the wheel contact center point 406 is located is further determined from the at least two first image regions.
By the embodiment provided by the application, a first area set corresponding to at least two first image substructures is obtained, wherein the first area set comprises at least two first image areas in the candidate recognition image, and the first image areas correspond to the first image substructures; and acquiring a second area set corresponding to at least two second image substructures, wherein the second area set comprises at least two second image areas in the candidate identification image, the second image areas correspond to the second image substructures, and the area range of the second image areas is larger than that of the first image areas, so that the effect of improving the accuracy of the position information of the wheel touchdown point is realized.
As an optional solution, determining a target image substructure corresponding to each target image area from at least two image substructures includes:
s1, determining a target first substructure and a target second substructure corresponding to each target image area, wherein the target image substructures comprise a target first substructure and a target second substructure; or the like, or a combination thereof,
s2, determining a first target substructure corresponding to each target image area respectively under the condition that the space occupation amount of the target vehicle is less than a first threshold value, and determining a second target substructure corresponding to each target image area respectively under the condition that the space occupation amount of the target vehicle is greater than or equal to the first threshold value.
Optionally, in this embodiment, to improve the image processing efficiency, but not limited to, the image substructures may be differentiated by the area range of the image area, for example, the image substructures corresponding to the image area with the smaller area range are used to perform image processing on the small vehicle (the space occupation amount of the small vehicle is less than the first threshold), and the image substructures corresponding to the image area with the larger area range are used to perform image processing on the large vehicle (the space occupation amount of the large vehicle is greater than or equal to the first threshold).
As an alternative, the obtaining of the position information of the wheel touch point of each target vehicle by using the target image substructure includes:
s1, acquiring the area center point of each target image area;
s2, calculating target offset between each area center point and the wheel touch point of the corresponding target vehicle by using the target image substructure;
and S3, acquiring the position information of the wheel touch point of each target vehicle according to each target offset and the area attribute information of the corresponding target image area.
Optionally, in this embodiment, the target image substructure is used to obtain the position information of the wheel touch point of each target vehicle, for example, a target image region in the candidate recognition image is determined by using a correspondence between the target image substructure and the image region, and then a region center point of the target image region is determined; and further utilizing an anchor point mechanism, calculating the position offset between the wheel touchdown point of each target vehicle and the central point of the area by adopting a target image substructure, and calculating the actual position information of the wheel touchdown point of each target vehicle according to the position offset.
Alternatively, in the present embodiment, the target offset may be, but is not limited to be, a position distance of a center point of each region relative to a wheel contact point of the corresponding target vehicle.
Alternatively, in the present embodiment, the region attribute information may be understood as, but not limited to, basic attribute information of the image region, such as length, width, area, image depth, image color, element distribution information, and the like.
It should be noted that, the area center point of each target image area is obtained; calculating target offset between each region center point and a wheel contact point of a corresponding target vehicle by using the target image substructure; and acquiring the position information of the wheel contact point of each target vehicle according to each target offset and the area attribute information of the corresponding target image area.
To further illustrate, optionally based on the scenario shown in fig. 4, continuing with, for example, fig. 6, a region center point 602 of the target image region 408 is obtained; calculating a target offset between the area center point 602 and the corresponding wheel touchdown point 406 of the target vehicle 404 using the target image substructure; the position information of the wheel contact point of the target vehicle 404 is acquired based on the target offset amount and the area attribute information of the corresponding target image area 408.
According to the embodiment provided by the application, the region center point of each target image region is obtained; calculating a target offset between each region center point and a wheel touch point of a corresponding target vehicle by using the target image substructure; and acquiring the position information of the wheel touch point of each target vehicle according to each target offset and the area attribute information of the corresponding target image area, thereby realizing the effect of improving the accuracy of the position information of the wheel touch point.
As an optional scheme, in order to improve the information acquisition efficiency of the wheel touchdown point, the wheel touchdown point acquisition method may also be, but is not limited to, completed in a model manner, such as inputting a candidate recognition image into an image recognition model, where the image recognition model is a neural network model for recognizing position information of the wheel touchdown point in the image, which is obtained after training by using a plurality of sample image data; and acquiring a recognition result output by the image recognition model, wherein the recognition result comprises position information of the wheel touch point of each target vehicle in the candidate recognition image.
As an alternative, before inputting the candidate recognition image into the image recognition model, the method includes:
s1, acquiring a plurality of sample image data;
s2, marking image data used for representing the wheel touchdown point in each sample image data to obtain a plurality of marked sample image data, wherein each marked sample image data comprises marked wheel touchdown point marks;
and S3, inputting the marked sample image data into an initial image recognition model to train and obtain the image recognition model.
Optionally, in this embodiment, the marked wheel contact point identifier is used as a sample input to the initial image recognition model to train the image recognition model capable of recognizing the wheel contact point.
As an alternative, inputting the marked multiple sample image data into an initial image recognition model to train to obtain an image recognition model, including:
repeatedly executing the following steps until an image recognition model is obtained:
s1, determining current sample image data from the marked sample image data, and determining a current image recognition model, wherein the current sample image data comprises marked current wheel touchdown point identification;
s2, identifying first result data which are used for representing the wheel touchdown center point in the current sample image data through the current image identification model;
s3, processing the first result data through the current image recognition model to obtain second result data used for representing the position information of the wheel touch point;
s4, acquiring next sample image data as the current sample image data under the condition that the second result data does not reach the identification convergence condition;
s5, when the second result data reaches the recognition convergence condition, the current image recognition model is determined to be the image recognition model.
As an alternative, before inputting the candidate recognition image into the image recognition model, the method includes:
s1, acquiring a plurality of sample image data;
s2, marking first image data used for representing a wheel touchdown point and second image data used for representing a wheel touchdown center point in each sample image data to obtain a plurality of marked sample image data, wherein each marked sample image data comprises a marked wheel touchdown point identifier and a marked wheel touchdown center point identifier;
and S3, inputting the marked sample image data into an initial image recognition model to train and obtain the image recognition model.
Optionally, in this embodiment, the marked wheel touchdown point identifier and the marked wheel touchdown center point identifier are used as samples to be input into the initial image recognition model, so as to train and obtain the image recognition model capable of recognizing the wheel touchdown point and the wheel touchdown center point.
As an alternative, inputting the marked multiple sample image data into an initial image recognition model to train to obtain an image recognition model, including:
repeatedly executing the following steps until an image recognition model is obtained:
s1, determining current sample image data from the marked sample image data, and determining a current image recognition model, wherein the current sample image data comprises marked current wheel touchdown point identification;
s2, identifying first result data which are used for representing the wheel touchdown center point in the current sample image data through the current image identification model;
s3, acquiring next sample image data as the current sample image data when the first result data does not reach the second convergence condition;
s4, processing the first result data by the current image recognition model when the first result data reaches the second convergence condition, to obtain second result data indicating position information of the wheel touchdown point;
s5, acquiring next sample image data as the current sample image data under the condition that the second result data does not reach the second convergence condition;
s6, when the second result data reaches the second convergence condition, the current image recognition model is determined to be the image recognition model.
As an alternative, the position information of the wheel touch point of each target vehicle is acquired by using the target image substructure, and the position information includes at least one of the following:
s1, acquiring first position information of the wheel touch point visible for each target vehicle by using the target image substructure;
and S2, acquiring second position information of the wheel touch point which is invisible to each target vehicle by using the target image substructure.
Optionally, in this embodiment, to improve the comprehensiveness of the wheel touchdown point information, the wheel touchdown point information may, but is not limited to, carry the wheel touchdown point and specific attribute data of the wheel touchdown point, such as a left front wheel, a left rear wheel, a right front wheel, and a right rear wheel, or a visible wheel touchdown point, an invisible wheel touchdown point, and the like.
It is noted that the target image substructure is used to obtain first position information of the wheel touch point visible to each target vehicle; second position information of the wheel touchdown point, which is invisible to each target vehicle, is acquired using the target image substructure.
For further illustration, alternatively, as shown in FIG. 3, for example, the solid line circles of wheel touchdown points 310-1 and 310-2 represent visible wheel touchdown points and the dashed line circles represent invisible wheel touchdown points.
According to the embodiment provided by the application, the target image substructure is used for acquiring the first position information of the wheel touch point visible for each target vehicle; and the second position information of the wheel touchdown point which is invisible for each target vehicle is obtained by utilizing the target image substructure, so that the effect of improving the comprehensiveness of the wheel touchdown point information is realized.
As an alternative, the obtaining of the position information of the wheel touch point of each target vehicle by using the target image substructure includes: acquiring a touch point set corresponding to each target vehicle by using a target image substructure, wherein the touch point set comprises a plurality of wheel touch points;
as an alternative, after acquiring the position information of the wheel touch point of each target vehicle by using the target image substructure, the method includes: determining a target touch point set from the touch point sets corresponding to the target vehicles respectively, wherein the number of wheel touch points in the target touch point set is greater than a second threshold value; and screening the wheel touch points in the target touch point set in a non-maximum value inhibition mode, wherein the number of the wheel touch points in the screened target touch point set is less than or equal to a second threshold value.
Optionally, in this embodiment, in order to improve the accuracy of the wheel contact point information, it may also be, but not limited to, after the position information of the wheel contact point of each target vehicle is acquired by using the target image substructure, some contact point instances with higher similarity are suppressed by non-maximum suppression, and the last contact point instances are detected.
According to the embodiment provided by the application, a contact point set corresponding to each target vehicle is obtained by using a target image substructure, wherein the contact point set comprises a plurality of wheel contact points; determining a target touch point set from the touch point sets corresponding to the target vehicles respectively, wherein the number of wheel touch points in the target touch point set is greater than a second threshold value; and screening the wheel touch points in the target touch point set by adopting a non-maximum value inhibition mode, wherein the number of the wheel touch points in the screened target touch point set is less than or equal to a second threshold value, and the effect of improving the accuracy of the wheel touch point information is realized.
As an alternative, for convenience of understanding, the wheel touchdown point acquisition method is applied to a real-time end-to-end wheel touchdown point detection scenario, for example, building a detection model (image sub-structure) based on ultra-lightweight convolutional neural networks, using a method based on key points (wheel touchdown center points) to detect wheel touchdown points, supporting occluded touchdown point prediction, and because the structure selection mode based on the anchor point mechanism is adopted to determine the detection model (target image substructure) corresponding to each target vehicle, the processing efficiency of the detection model is improved, for example a target vehicle only needs to be processed using a partial detection submodel of the overall detection model, therefore, the deployment (such as a vehicle machine) can be completed on the CPU equipment with low computing power, and the contact points of the wheels of other vehicles in the current driving environment can be further detected in real time.
Optionally, in this embodiment, the labeling is performed by using a method of a key point, and the left front wheel, the left rear wheel, the right front wheel and the right rear wheel are distinguished, for a visible wheel touchdown point, the labeling may be performed directly, for an invisible touchdown point, the labeling is performed according to experience, and at the same time, whether a visible attribute is attached. The marks are visualized as circles of different colors representing the marked touch points, and for invisible points, the points are displayed in white outer circles.
Optionally, in this embodiment, a detection network is built based on a depth separable convolution, and then based on a feature pyramid structure (different from the original feature pyramid scheme, where a feature pyramid is also built by using a depth separable convolution instead of a normal convolution, which has an advantage of low computation amount), the last layer of feature map of the network is used as a detection head to be responsible for predicting a touch location instance; the detailed configuration of the network shown in table (1) below, in which the linear bottleneck layer is the basic structure in mobilenetV2, the following expansion coefficient, the number of convolution kernels, the number of repetitions, and the stride are parameters of the linear bottleneck layer, and the representation of the feature pyramid as √ requires feature fusion through the feature pyramid. Further exemplifying, an optional network overall schematic diagram shown in fig. 7 includes a back bone and a feature pyramid structure, where an upsampling structure of the feature pyramid is obtained by depth-separable upsampling and bilinear upsampling stitching, and a right-side subgraph in fig. 7 describes a training overall flow diagram of the wheel contact point obtaining method, first, an original image is preprocessed and then input into a detection network for forward transmission calculation to obtain outputs of different detection heads, and then, a loss is constructed based on an output value of the network and a pre-labeled label value to calculate a new network in a gradient manner. The overall calculation amount of the scheme is 52MFLOPs, and the scheme belongs to a very low floating point calculation amount level, which is enough to achieve the aim of real-time operation on low-end CPU equipment.
Figure BDA0003538993500000231
Watch (1)
Figure BDA0003538993500000241
Continuation watch (1)
Optionally, in this embodiment, based on the regression prediction of the key points of the anchor point mechanism, specifically as shown in fig. 8, the feature map with fixed height and width is regarded as a grid and is uniformly distributed on the original image 802 (e.g., image areas included by white grid lines); for further example, it is first necessary to determine where there are touch points and where there are no touch points, such as on which grid the centers of the four touch points of an example of a car shown in the original image 802 are located, and the feature vector represented by that grid is responsible for predicting the four touch points of the car, or it can be understood that the feature vector at the grid is used to predict the four touch points of the car;
further to the foregoing network, in the present embodiment, feature maps of two detection heads are generated, with height and width of 9 × 16 and 18 × 32, respectively. The center of a specific vehicle touchdown point example falls into two layers of detection head feature maps at the same time, and at the moment, the wheel touchdown point of the vehicle is detected by which detection head, so that ambiguity can be generated, an anchor point frame mechanism is further introduced on the basis that the anchor point frames are virtually distributed in each grid of the detection head feature maps (the centers of the anchor point frames are aligned with the centers of the grids), anchor point frames with the width of 300 × 300 (the size is relative to the network input) are preset for the feature maps with the size of 9 × 16 (relative to the network input), anchor point frames with the width of 80 × 80 are preset for the feature maps with the size of 18 × 32, and the condition that the larger IOU of the maximum circumscribed rectangle formed by four touchdown point examples of the vehicle and the preset anchor point frames of different detection heads is responsible for predicting the touchdown point of the vehicle. Such a specification allows the dense detection heads (18 × 32) to predict the contact point of a small vehicle, and the sparse detection heads (9 × 16) to predict the contact point of a large vehicle.
Optionally, in this embodiment, regarding how the network predicts whether a certain point of the feature map contains the center of the wheel touch point of one vehicle instance, the prediction is performed by using two classifications based on cross entropy, as shown in the following formula (1):
Figure BDA0003538993500000251
wherein, clfpredConfidence score for whether there is a touchdown center predicted for the network, where index 0 represents the background and index 1 represents the presence of the touchdown center clfgtIs the touchdown point one-hot true value tag, if there is no touchdown point center, then is [1, 0%]If a touchdown point is present, then it is [0, 1 ]]And crossEntropy calculation function is expressed by crossEntrol.
Alternatively, in the present embodiment, for the feature vector responsible for predicting the vehicle touchdown point, the positions of four touchdown points, and the attributes of whether they are visible, need to be predicted. For the position of the points, a prediction is made in the form of a keypoint-based offset. For example, to use the offset based on the grid center to represent its position, and to balance the dimension of calculating the offset of the parking space key point by different detection heads, the offset is wide relative to the anchor point, so as to avoid the dimension problem that the calculated value of the small vehicle contact point is smaller than the calculated value of the large vehicle contact point, as shown in fig. 8, the contact point of the vehicle instance is predicted by the feature vector represented by the target grid 804, wherein, taking the right rear wheel point of the vehicle instance as an example, the coordinates of the right rear wheel point are represented by x _ offset and y _ offset, and the corresponding loss is shown in the following formula (2):
Figure BDA0003538993500000252
where a is the width and height of the anchor box, pgtIs the actual pixel coordinate of the touchdown point, g is the pixel coordinate of the grid center responsible for predicting the touchdown point, offsetpredIs the network predicted coordinate offset, with respect to the touchdown point coordinatesgtThe real value of the coordinate offset is obtained based on the actual pixel coordinates of the touch point and the pixel coordinates of the center of the grid.
Optionally, in this embodiment, for the attribute of whether the wheel touchdown point is visible, two-class cross entropy may be used for prediction, which may be specifically referred to the following formula (3):
Figure BDA0003538993500000261
wherein, vispredFor the prediction confidence score of whether the network is visible at this point, index 0 represents invisible, index 1 represents visible, visgtIs a one-hot true value label, if the point is not visible, then is [1, 0 ]]If visible, is [0, 1 ]]。
Optionally, in the detection stage of this embodiment, the acquired vehicle-end image may be, but is not limited to, sent into a network, the network outputs a feature map of two layers of detection heads, then, each feature vector (grid) on the feature map is sequentially traversed, if a confidence of the contact point center in the feature vector is greater than a preset threshold, the feature vector is considered to have four wheel contact points, and then, the specific positions of the four contact points are obtained by obtaining coordinate offsets of the four contact points. After traversing the two layers of feature maps, a series of contact point examples (4 contact point examples in a group comprising four contact points of the left front, the right front, the left back and the right back) can be obtained, some contact point examples with higher similarity are restrained through non-maximum value suppression, and the last contact point examples are detected. The nature of non-maxima suppression here is the same as in object detection, except that in object detection, the computation is based on rectangular boxes, and here is based on a closed graph of touch location instances.
By the embodiment provided by the application, the wheel touch point in the unit of the vehicle example is detected end to end; moreover, the algorithm of the wheel touch point acquisition method can be calculated only by means of one-time forward calculation of a network; the system can be deployed on CPU equipment to achieve real-time performance; in addition, the overall calculation amount of the wheel touchdown point acquisition method is low, the method can be operated on equipment with low calculation force, and most detection functions can be met in a single-core operation mode; furthermore, the accuracy and the recall rate of the wheel touch point acquisition method can be maintained at a considerable level while considering the calculation amount, and basically meet the accuracy and the recall rate of the use requirements of users.
It is understood that in the specific implementation of the present application, related data such as user information, when the above embodiments of the present application are applied to specific products or technologies, user permission or consent needs to be obtained, and the collection, use and processing of related data need to comply with related laws and regulations and standards of related countries and regions.
It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present application is not limited by the order of acts described, as some steps may occur in other orders or concurrently depending on the application. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required in this application.
According to another aspect of the embodiments of the present application, there is also provided an acquisition apparatus of a wheel touchdown point for implementing the above-described acquisition method of a wheel touchdown point. As shown in fig. 9, the apparatus includes:
a first determining unit 902, configured to, when a candidate identification image is acquired and at least one target vehicle is identified from the candidate identification image, determine a wheel touchdown center point of each target vehicle according to an image feature of the candidate identification image, where the wheel touchdown center point is used to represent position information of centers of at least two wheel touchdown points of the target vehicle in the candidate identification image;
a first obtaining unit 904, configured to obtain at least two image substructures corresponding to the candidate identification image, where each of the at least two image substructures corresponds to at least one image area in the candidate identification image, and the image substructures are configured to identify image content information of the candidate identification image to obtain position information of the wheel touch point;
a second determining unit 906, configured to determine, from the at least two image substructures, a target image substructure corresponding to each target vehicle according to the image region where the wheel touchdown center point is located;
a second acquisition unit 908 for acquiring position information of the wheel touchdown point of each target vehicle using the target image substructure.
Alternatively, in this embodiment, the wheel touch point obtaining device may be, but not limited to, applied in a real-time end-to-end wheel touch point detection scenario, for example, building a detection model (image sub-structure) based on ultra-lightweight convolutional neural networks, using a key point (wheel touchdown center) based device to detect wheel touchdown points, supporting occluded touchdown point prediction, and because the structure selection mode based on the anchor point mechanism is adopted to determine the detection model (target image substructure) corresponding to each target vehicle, the processing efficiency of the detection model is improved, for example a target vehicle only needs to be processed using a partial detection submodel of the overall detection model, therefore, the deployment (such as a vehicle machine) can be completed on the CPU equipment with low computing power, and the contact points of the wheels of other vehicles in the current driving environment can be further detected in real time.
Optionally, in this embodiment, the wheel grounding point may be, but is not limited to, helping the auto-driven vehicle to obtain a more accurate relative position, direction and size of the vehicle, and is of great significance to the improvement of safety during driving; for example, after obtaining the position information of the wheel contact point of each target vehicle by using the target image substructure, the spatial information of the target vehicle is determined by using the position information of the wheel contact point, wherein the spatial information may include, but is not limited to, at least one of the following: relative position of the vehicle, relative direction of the vehicle, size of the vehicle, travel path of the vehicle, etc.
Optionally, in this embodiment, the apparatus for acquiring a wheel touchdown point may also be applied, but not limited to, in a scene where a relatively accurate relative distance detection is performed on another vehicle, for example, a center (pixel position) corresponding to the obtained touchdown point is calculated based on the obtained touchdown point, and then the calculated touchdown point is converted into a world coordinate position under a vehicle body coordinate system through a camera parameter calibrated in advance and a ground assumption, so that the apparatus can help ADAS early warning, lane-level vehicle rendering, and the like to acquire accurate position information of the other vehicle.
Optionally, in this embodiment, the above-mentioned device for acquiring a wheel touchdown point may also be but not limited to be applied to a scene where a monocular camera performs vehicle 3D target detection, for example, based on the acquired touchdown point, in combination with 2D target detection and camera calibration parameters, 3D target detection information of a vehicle may be generated, which includes a center position (x, y, z), a length, a width, a height, a heading angle, and the like.
Optionally, in this embodiment, the candidate identification image may be acquired by, but not limited to, acquiring an image of a vehicle during operation by using a vehicle-mounted image acquisition device, or acquiring a road condition image by using a road side camera.
Optionally, in the present embodiment, the image feature may be understood as, but not limited to, a feature presented by an image pixel in the candidate recognition image, or may be understood as a feature recognized by an image recognition technology, such as a color feature, a shape feature, an image local feature, an image depth feature, an edge feature, a linear feature, a texture feature, and the like; determining the wheel touchdown center of each target vehicle according to the image features of the candidate recognition images can be, but is not limited to, accomplished by using an image recognition model, for example, an initial image recognition model is determined first, and then labeled sample image data is input for training to obtain a trained image recognition model, wherein the labeled sample image data includes image data of the target vehicle labeled with key points, and the key points include wheel touchdown points and specific attribute data of the wheel touchdown points, such as left front wheel, left rear wheel, right front wheel and right rear wheel, or visible wheel touchdown points, invisible wheel touchdown points, and the like.
Alternatively, in this embodiment, the wheel touchdown center may be, but is not limited to be, a center of at least two wheel touchdown points of the target vehicle, and since at least two wheel touchdown points may include a wheel touchdown point that is not currently visible, in this case, the position information of the wheel touchdown center may be, but is not limited to, predicted first by predicting the position information of the wheel touchdown point that is not currently visible, and then by predicting the position information of the wheel touchdown center by combining the position information of the wheel touchdown point that is visible.
Optionally, in this embodiment, the target image model may include, but is not limited to, at least two image substructures, where each of the at least two image substructures corresponds to at least one image region in the candidate recognition image, or it may be understood that each image substructure corresponds to at least one image region in the candidate recognition image, and then the corresponding image substructures may be directly determined in a case where the image regions are determined first, or conversely, the corresponding image regions may also be directly determined in a case where the image substructures are determined first; in addition, the image substructure is used for identifying the image content information of the candidate identification image to obtain the position information of the wheel touchdown point, or it can be understood that the image substructure corresponds to the image area, but the image substructure is used for identifying the whole image content information of the candidate identification image, or is not limited to the image content information of the image area corresponding to the image substructure, but is not limited to having stronger correlation with the image content information of the corresponding image area, or having higher identification accuracy on the image content information of the corresponding image area, thereby improving the identification accuracy of the wheel touchdown point.
Optionally, in this embodiment, the target image substructure corresponding to each target vehicle is determined from the at least two image substructures according to the image region where the wheel touchdown center point is located, or it may be understood that the target image region where the wheel touchdown center point is located is determined from the plurality of image regions, and then the target image substructure corresponding to the target image region is determined from the at least two image substructures by using the correspondence between the image regions and the image substructures; in addition, since the target image area is determined by the wheel contact center point, which is clearly the target vehicle, when the target image substructure is determined, the target vehicle corresponding to the target image substructure can be obtained, or it can be understood that the target image substructure and the target vehicle have a corresponding relationship.
Optionally, in this embodiment, the target image substructure is used to obtain the position information of the wheel touch point of each target vehicle, for example, a target image region in the candidate recognition image is determined by using a correspondence between the target image substructure and the image region, and then a region center point of the target image region is determined; and further utilizing an anchor point mechanism, calculating the position offset between the wheel touchdown point of each target vehicle and the central point of the area by adopting a target image substructure, and calculating the actual position information of the wheel touchdown point of each target vehicle according to the position offset.
It should be noted that, the position information of the wheel touch point of each target vehicle is acquired by using the target image substructure, the anchor point mechanism is used to determine the target image substructure corresponding to each vehicle through the wheel touch point, then the target image substructure is used to identify the wheel touch point with stronger pertinence to the corresponding vehicle, and the information association relationship is established between the identified wheel touch point and the corresponding vehicle, so that the position information of the wheel touch point becomes more comprehensive; in addition, the target image substructure corresponding to each target vehicle is determined from the plurality of image substructures by using the wheel touchdown center point, so that the application efficiency of the image substructures is improved, and the device for acquiring the wheel touchdown point can be realized in application scenes with low calculation force, such as application scenes of vehicle-mounted equipment.
For a specific embodiment, reference may be made to the example shown in the above-mentioned device for acquiring the wheel touch point, and details of this example are not repeated here.
According to the embodiment provided by the application, under the condition that the candidate identification image is acquired and at least one target vehicle is identified from the candidate identification image, the wheel touchdown center point of each target vehicle is determined according to the image characteristics of the candidate identification image, wherein the wheel touchdown center point is used for representing the position information of the centers of at least two wheel touchdown points of the target vehicles in the candidate identification image; acquiring at least two image substructures corresponding to the candidate identification images, wherein each image substructure of the at least two image substructures corresponds to at least one image area in the candidate identification images, and the image substructures are used for identifying image content information of the candidate identification images to acquire position information of the wheel touchdown point; determining a target image substructure corresponding to each target vehicle from the at least two image substructures according to an image region where the wheel touchdown center point is located; acquiring position information of a wheel touch point of each target vehicle by using the target image substructure; according to the embodiment of the application, the position information of the wheel touch point of each target vehicle is obtained by using the target image substructure, the target image substructure corresponding to each vehicle is determined by using the wheel touch point through the anchor point mechanism, then the target image substructure is used for identifying the wheel touch point with stronger pertinence to the corresponding vehicle, and the information association relationship is established between the identified position information of the wheel touch point and the corresponding vehicle, so that the position information of the wheel touch point is more comprehensive; in addition, the wheel touchdown center point is used for determining the target image substructure corresponding to each target vehicle from the plurality of image substructures, and the application efficiency of the image substructures is improved.
As an alternative, the second determining unit 906 includes:
the first acquisition module is used for acquiring an image area set corresponding to at least two image substructures, wherein the image area set comprises at least two image areas in the candidate identification image;
the second acquisition module is used for acquiring target image areas where the wheel touchdown central points corresponding to the target vehicles are located, wherein the at least two image areas comprise target image areas;
the first determining module is used for determining a target image substructure corresponding to each target image area from at least two image substructures.
For a specific embodiment, reference may be made to the example shown in the above method for acquiring the wheel touchdown point, and details in this example are not described herein again.
As an optional solution, the first obtaining module includes:
the first obtaining submodule is used for obtaining a first area set corresponding to at least two first image substructures, wherein the first area set comprises at least two first image areas in the candidate recognition image, and the first image areas correspond to the first image substructures;
and the second acquisition submodule is used for acquiring a second area set corresponding to at least two second image substructures, wherein the second area set comprises at least two second image areas in the candidate identification image, the second image areas correspond to the second image substructures, and the area range of the second image areas is larger than that of the first image areas.
For a specific embodiment, reference may be made to the example shown in the above method for acquiring the wheel touchdown point, and details in this example are not described herein again.
As an optional solution, the first determining module includes:
the first determining submodule is used for determining a first target substructure and a second target substructure corresponding to each target image area, wherein the target image substructures comprise a first target substructure and a second target substructure; or the like, or a combination thereof,
and the second determining sub-module is used for determining a target first sub-structure corresponding to each target image area under the condition that the space occupation amount of the target vehicle is smaller than the first threshold value, and determining a target second sub-structure corresponding to each target image area under the condition that the space occupation amount of the target vehicle is larger than or equal to the first threshold value.
For a specific embodiment, reference may be made to the example shown in the above method for acquiring the wheel touchdown point, and details in this example are not described herein again.
As an optional solution, the second obtaining unit 908 includes:
the third acquisition module is used for acquiring the area center point of each target image area;
the calculation module is used for calculating target offset between each area center point and a wheel touch point of each corresponding target vehicle by using the target image substructure;
and the fourth acquisition module is used for acquiring the position information of the wheel touch point of each target vehicle according to each target offset and the area attribute information of the corresponding target image area.
For a specific embodiment, reference may be made to the example shown in the above method for acquiring the wheel touchdown point, and details in this example are not described herein again.
As an optional solution, the apparatus further includes:
the input unit is used for inputting the candidate recognition image into an image recognition model, wherein the image recognition model is a neural network model which is obtained by training by utilizing a plurality of sample image data and is used for recognizing the position information of the wheel contact point in the image;
and the third acquisition unit is used for acquiring the identification result output by the image identification model, wherein the identification result comprises the position information of the wheel touch point of each target vehicle in the candidate identification image.
For a specific embodiment, reference may be made to the example shown in the above method for acquiring the wheel touchdown point, and details in this example are not described herein again.
As an alternative, the method comprises the following steps:
a fourth acquisition unit configured to acquire a plurality of sample image data before the candidate recognition image is input to the image recognition model;
the wheel contact point identification method comprises a first marking unit, a second marking unit and a third marking unit, wherein the first marking unit is used for marking image data used for representing wheel contact points in each sample image data before a candidate identification image is input into an image identification model to obtain a plurality of marked sample image data, and each marked sample image data comprises marked wheel contact point identification;
and the first training unit is used for inputting the marked sample image data into the initial image recognition model before the candidate recognition image is input into the image recognition model so as to train the image recognition model.
For a specific embodiment, reference may be made to the example shown in the above method for acquiring the wheel touchdown point, and details in this example are not described herein again.
As an alternative, the first training unit includes:
a first repeating module, configured to repeatedly perform the following steps until an image recognition model is obtained:
the second determining module is used for determining current sample image data from the marked sample image data and determining a current image recognition model, wherein the current sample image data comprises marked current wheel touchdown point identifiers;
the first identification module is used for identifying first result data which are used for representing the wheel touchdown center point in the current sample image data through a current image identification model;
the first processing module is used for processing the first result data through the current image recognition model to obtain second result data used for representing the position information of the wheel touch point;
the second processing module is used for acquiring next sample image data as the current sample image data under the condition that the second result data does not reach the identification convergence condition;
and the third processing module is used for determining the current image recognition model as the image recognition model under the condition that the second result data reaches the recognition convergence condition.
For a specific embodiment, reference may be made to the example shown in the above method for acquiring the wheel touchdown point, and details in this example are not described herein again.
As an alternative, the method comprises the following steps:
a fifth acquiring unit configured to acquire a plurality of sample image data before the candidate recognition image is input to the image recognition model;
the second marking unit is used for marking the first image data used for representing the wheel touchdown point and the second image data used for representing the wheel touchdown center point in each sample image data before the candidate identification image is input into the image identification model, so as to obtain a plurality of marked sample image data, wherein each marked sample image data comprises a marked wheel touchdown point identifier and a marked wheel touchdown center point identifier;
and the second training unit is used for inputting the marked sample image data into the initial image recognition model before the candidate recognition image is input into the image recognition model so as to train the image recognition model.
For a specific embodiment, reference may be made to the example shown in the above method for acquiring the wheel touch point, and details of this example are not described herein again.
As an alternative, the second training unit includes:
a second repeating module, configured to repeatedly perform the following steps until an image recognition model is obtained:
the third determining module is used for determining current sample image data from the marked sample image data and determining a current image recognition model, wherein the current sample image data comprises marked current wheel touchdown point identifiers;
the second identification module is used for identifying first result data which are used for representing the wheel touchdown center point in the current sample image data through the current image identification model;
the fourth processing module is used for acquiring next sample image data as the current sample image data under the condition that the first result data does not reach the second convergence condition;
the fifth processing module is used for processing the first result data through the current image recognition model under the condition that the first result data reaches a second convergence condition to obtain second result data used for representing the position information of the wheel touchdown point;
the sixth processing module is used for acquiring next sample image data as the current sample image data under the condition that the second result data does not reach the second convergence condition;
and the seventh processing module is used for determining the current image recognition model as the image recognition model under the condition that the second result data reaches the second convergence condition.
For a specific embodiment, reference may be made to the example shown in the above method for acquiring the wheel touchdown point, and details in this example are not described herein again.
As an optional solution, the second obtaining unit 908 includes at least one of:
a first location module for obtaining first location information of a wheel touchdown point visible to each target vehicle using the target image substructure;
a second location module to obtain second location information of the wheel touchdown point that is invisible to each of the target vehicles using the target image substructure.
For a specific embodiment, reference may be made to the example shown in the above method for acquiring the wheel touchdown point, and details in this example are not described herein again.
As an optional solution, the second obtaining unit 908 includes: a fifth obtaining module, configured to obtain, by using the target image substructure, a respective touch point set corresponding to each target vehicle, where the touch point set includes a plurality of wheel touch points;
an apparatus, comprising: after the position information of the wheel touch points of each target vehicle is obtained by using the target image substructure, determining a target touch point set from the touch point sets corresponding to the target vehicles, wherein the number of the wheel touch points in the target touch point set is greater than a second threshold value; and screening the wheel touch points in the target touch point set in a non-maximum value inhibition mode, wherein the number of the wheel touch points in the screened target touch point set is less than or equal to a second threshold value.
For a specific embodiment, reference may be made to the example shown in the above method for acquiring the wheel touchdown point, and details in this example are not described herein again.
According to another aspect of the embodiments of the present application, there is also provided an electronic device for implementing the method for acquiring a wheel touch point, as shown in fig. 10, the electronic device includes a memory 1002 and a processor 1004, the memory 1002 stores a computer program, and the processor 1004 is configured to execute the steps in any one of the method embodiments through the computer program.
Optionally, in this embodiment, the electronic device may be located in at least one network device of a plurality of network devices of a computer network.
Optionally, in this embodiment, the processor may be configured to execute the following steps by a computer program:
s1, under the condition that the candidate identification image is obtained and at least one target vehicle is identified from the candidate identification image, determining a wheel touchdown center point of each target vehicle according to the image characteristics of the candidate identification image, wherein the wheel touchdown center point is used for representing the position information of the centers of at least two wheel touchdown points of the target vehicle in the candidate identification image;
s2, acquiring at least two image substructures corresponding to the candidate identification images, wherein each image substructures in the at least two image substructures corresponds to at least one image area in the candidate identification images, and the image substructures are used for identifying the image content information of the candidate identification images to acquire the position information of the wheel touchdown points;
s3, determining a target image substructure corresponding to each target vehicle from the at least two image substructures according to the image area where the wheel touchdown center point is located;
and S4, acquiring the position information of the wheel touch point of each target vehicle by using the target image substructure.
Alternatively, it can be understood by those skilled in the art that the structure shown in fig. 10 is only an illustration, and the electronic device may also be a terminal device such as a smart phone (e.g., an Android phone, an iOS phone, etc.), a tablet computer, a palmtop computer, and a Mobile Internet Device (MID), a PAD, and the like. Fig. 10 is a diagram illustrating a structure of the electronic device. For example, the electronic device may also include more or fewer components (e.g., network interfaces, etc.) than shown in FIG. 10, or have a different configuration than shown in FIG. 10.
The memory 1002 may be used to store software programs and modules, such as program instructions/modules corresponding to the wheel contact point acquisition method and apparatus in the embodiment of the present application, and the processor 1004 executes various functional applications and data processing by running the software programs and modules stored in the memory 1002, so as to implement the wheel contact point acquisition method. The memory 1002 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 1002 can further include memory located remotely from the processor 1004, which can be coupled to the terminal over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof. The memory 1002 may be specifically, but not limited to, used for storing information such as candidate recognition images, target image substructures, and position information of wheel touchdown points. As an example, as shown in fig. 10, the memory 1002 may include, but is not limited to, a first determining unit 902, a first acquiring unit 904, a second determining unit 906, and a second acquiring unit 908 of the acquiring apparatus of the wheel contact point. In addition, other module units in the above-mentioned wheel touchdown point acquisition device may also be included, but are not limited to this, and are not described in detail in this example.
Optionally, the above-mentioned transmission device 1006 is used for receiving or sending data via a network. Examples of the network may include a wired network and a wireless network. In one example, the transmission device 1006 includes a Network adapter (NIC) that can be connected to a router via a Network cable and other Network devices so as to communicate with the internet or a local area Network. In one example, the transmission device 1006 is a Radio Frequency (RF) module, which is used for communicating with the internet in a wireless manner.
In addition, the electronic device further includes: a display 1008 for displaying information such as the candidate recognition image, the target image substructure, and the position information of the wheel touchdown point; and a connection bus 1010 for connecting the respective module parts in the above-described electronic apparatus.
In other embodiments, the terminal device or the server may be a node in a distributed system, where the distributed system may be a blockchain system, and the blockchain system may be a distributed system formed by connecting a plurality of nodes through a network communication. The nodes may form a Peer-To-Peer (P2P) network, and any type of computing device, such as a server, a terminal, and other electronic devices, may become a node in the blockchain system by joining the Peer-To-Peer network.
According to an aspect of the application, there is provided a computer program product comprising a computer program/instructions containing program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication section, and/or installed from a removable medium. When executed by the central processing unit, the computer program performs various functions provided by the embodiments of the present application.
The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments.
It should be noted that the computer system of the electronic device is only an example, and should not bring any limitation to the functions and the scope of the application of the embodiments.
The computer system includes a Central Processing Unit (CPU) that can perform various appropriate actions and processes according to a program stored in a Read-Only Memory (ROM) or a program loaded from a storage section into a Random Access Memory (RAM). In the random access memory, various programs and data necessary for the operation of the system are also stored. The central processor, the read-only memory and the random access memory are connected with each other through a bus. An Input/Output interface (i.e., I/O interface) is also connected to the bus.
The following components are connected to the input/output interface: an input section including a keyboard, a mouse, and the like; an output section including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, a speaker, and the like; a storage section including a hard disk and the like; and a communication section including a network interface card such as a local area network card, a modem, or the like. The communication section performs communication processing via a network such as the internet. The driver is also connected to the input/output interface as needed. A removable medium such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive as necessary, so that a computer program read out therefrom is mounted into the storage section as necessary.
In particular, according to embodiments of the present application, the processes described in the various method flowcharts may be implemented as computer software programs. For example, embodiments of the present application include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication section, and/or installed from a removable medium. When executed by the central processing unit, performs various functions defined in the system of the present application.
According to an aspect of the present application, there is provided a computer-readable storage medium from which a processor of a computer device reads computer instructions, the processor executing the computer instructions to cause the computer device to perform the method provided in the above-mentioned various alternative implementations.
Alternatively, in the present embodiment, the above-mentioned computer-readable storage medium may be configured to store a computer program for executing the steps of:
s1, under the condition that the candidate identification image is acquired and at least one target vehicle is identified from the candidate identification image, determining a wheel contact center point of each target vehicle according to the image characteristics of the candidate identification image, wherein the wheel contact center point is used for representing the position information of the centers of at least two wheel contact points of the target vehicle in the candidate identification image;
s2, acquiring at least two image substructures corresponding to the candidate identification images, wherein each image substructures in the at least two image substructures corresponds to at least one image area in the candidate identification images, and the image substructures are used for identifying the image content information of the candidate identification images to acquire the position information of the wheel touchdown points;
s3, determining a target image substructure corresponding to each target vehicle from the at least two image substructures according to the image area where the wheel touchdown center point is located;
and S4, acquiring the position information of the wheel touch point of each target vehicle by using the target image substructure.
Alternatively, in this embodiment, a person skilled in the art may understand that all or part of the steps in the methods of the foregoing embodiments may be implemented by a program instructing hardware associated with the terminal device, where the program may be stored in a computer-readable storage medium, and the storage medium may include: flash disks, Read-Only memories (ROMs), Random Access Memories (RAMs), magnetic or optical disks, and the like.
The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments.
The integrated unit in the above embodiments, if implemented in the form of a software functional unit and sold or used as a separate product, may be stored in the above computer-readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or a part of or all or part of the technical solution contributing to the prior art may be embodied in the form of a software product stored in a storage medium, and including instructions for causing one or more computer devices (which may be personal computers, servers, network devices, or the like) to execute all or part of the steps of the method described in the embodiments of the present application.
In the above embodiments of the present application, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the several embodiments provided in the present application, it should be understood that the disclosed client may be implemented in other manners. The above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one type of logical functional division, and other divisions may be implemented in practice, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The foregoing is only a preferred embodiment of the present application and it should be noted that those skilled in the art can make several improvements and modifications without departing from the principle of the present application, and these improvements and modifications should also be considered as the protection scope of the present application.

Claims (15)

1. A method of obtaining a wheel contact point, comprising:
when a candidate identification image is acquired and at least one target vehicle is identified from the candidate identification image, determining a wheel touchdown center point of each target vehicle according to the image characteristics of the candidate identification image, wherein the wheel touchdown center point is used for representing the position information of the centers of at least two wheel touchdown points of the target vehicle in the candidate identification image;
acquiring at least two image substructures corresponding to candidate identification images, wherein each image substructure in the at least two image substructures corresponds to at least one image area in the candidate identification images, and the image substructures are used for identifying image content information of the candidate identification images to acquire position information of the wheel touch points;
determining a target image substructure corresponding to each target vehicle from the at least two image substructures according to the image region where the wheel touchdown center point is located;
and acquiring the position information of the wheel touch point of each target vehicle by using the target image substructure.
2. The method of claim 1, wherein determining from the at least two image substructures a respective target image substructure for each of the target vehicles based on the image region in which the wheel contact center point is located comprises:
acquiring an image area set corresponding to the at least two image substructures, wherein the image area set comprises at least two image areas in the candidate identification image;
acquiring a target image area where the wheel touchdown center point corresponding to each target vehicle is located, wherein the at least two image areas comprise the target image area;
and determining the target image substructure corresponding to each target image area from the at least two image substructures.
3. The method of claim 2, wherein the obtaining the set of image regions corresponding to the at least two image substructures comprises:
acquiring a first region set corresponding to at least two first image substructures, wherein the first region set comprises at least two first image regions in the candidate identification image, and the first image regions correspond to the first image substructures;
and acquiring a second region set corresponding to at least two second image substructures, wherein the second region set comprises at least two second image regions in the candidate identification image, the second image regions correspond to the second image substructures, and the region range of the second image regions is larger than that of the first image regions.
4. The method according to claim 3, wherein said determining the target image substructure for each of the target image regions from the at least two image substructures comprises:
determining a target first substructure and a target second substructure corresponding to each target image area, wherein the target image substructure comprises the target first substructure and the target second substructure; or the like, or, alternatively,
determining the target first substructure corresponding to each target image region respectively under the condition that the space occupation amount of the target vehicle is smaller than a first threshold value, and determining the target second substructure corresponding to each target image region respectively under the condition that the space occupation amount of the target vehicle is larger than or equal to the first threshold value.
5. The method of claim 2, wherein said obtaining position information of the wheel touchdown point of each of the target vehicles using the target image substructure comprises:
acquiring a region central point of each target image region;
calculating a target offset between each of the region center points and the respective corresponding wheel contact points of the target vehicle using the target image substructure;
and acquiring the position information of the wheel touch point of each target vehicle according to each target offset and the area attribute information of the corresponding target image area.
6. The method of claim 1, further comprising:
inputting the candidate recognition image into an image recognition model, wherein the image recognition model is a neural network model which is obtained by training a plurality of sample image data and is used for recognizing the position information of the wheel touch point in the image;
and acquiring a recognition result output by the image recognition model, wherein the recognition result comprises the position information of the wheel touch point of each target vehicle in the candidate recognition image.
7. The method of claim 6, wherein prior to said inputting said candidate recognition images into an image recognition model, comprising:
acquiring the plurality of sample image data;
marking image data used for representing the wheel touchdown point in each sample image data to obtain a plurality of marked sample image data, wherein each marked sample image data comprises marked wheel touchdown point identifiers;
and inputting the marked sample image data into an initial image recognition model to train to obtain the image recognition model.
8. The method of claim 7, wherein the inputting the labeled sample image data into an initial image recognition model to train the image recognition model comprises:
repeatedly executing the following steps until the image recognition model is obtained:
determining current sample image data from the marked plurality of sample image data and determining a current image recognition model, wherein the current sample image data comprises marked current wheel touchdown point identifiers;
identifying, by the current image recognition model, first result data representing the wheel contact center point in the current sample image data;
processing the first result data through the current image recognition model to obtain second result data used for representing the position information of the wheel touchdown point;
under the condition that the second result data does not reach the identification convergence condition, acquiring next sample image data as the current sample image data;
determining the current image recognition model as the image recognition model when the second result data reaches the recognition convergence condition.
9. The method of claim 6, wherein prior to said inputting said candidate recognition images into an image recognition model, comprising:
acquiring the plurality of sample image data;
marking first image data used for representing the wheel touchdown point and second image data used for representing the wheel touchdown center point in each sample image data to obtain a plurality of marked sample image data, wherein each marked sample image data comprises a marked wheel touchdown point identifier and a marked wheel touchdown center point identifier;
and inputting the marked sample image data into an initial image recognition model to train to obtain the image recognition model.
10. The method of claim 9, wherein the inputting the labeled sample image data into an initial image recognition model to train the image recognition model comprises:
repeatedly executing the following steps until the image recognition model is obtained:
determining current sample image data from the marked plurality of sample image data and determining a current image recognition model, wherein the current sample image data comprises marked current wheel touchdown point identifiers;
identifying, by the current image recognition model, first result data representing the wheel touchdown center point in the current sample image data;
under the condition that the first result data does not reach a second convergence condition, acquiring next sample image data as the current sample image data;
under the condition that the first result data reach the second convergence condition, processing the first result data through the current image recognition model to obtain second result data used for representing position information of the wheel touchdown point;
under the condition that the second result data does not reach a second convergence condition, acquiring next sample image data as the current sample image data;
determining the current image recognition model as the image recognition model if the second result data reaches the second convergence condition.
11. The method of any one of claims 1 to 10, wherein said obtaining position information of said wheel touchdown point of each of said target vehicles using said target image substructure comprises at least one of:
obtaining first position information of the wheel touchdown point visible to each of the target vehicles using the target image substructure;
second position information of the wheel contact point, which is invisible to each of the target vehicles, is acquired using the target image substructure.
12. The method according to any one of claims 1 to 10,
the obtaining the position information of the wheel touchdown point of each of the target vehicles using the target image substructure includes: acquiring a respective corresponding touchdown point set of each target vehicle by using the target image substructure, wherein the touchdown point set comprises a plurality of wheel touchdown points;
after the obtaining of the position information of the wheel touchdown point of each of the target vehicles using the target image substructure, the method includes: determining a target touch point set from the touch point sets corresponding to the target vehicles respectively, wherein the number of the wheel touch points in the target touch point set is greater than a second threshold value; screening the wheel contact points in the target contact point set in a non-maximum suppression manner, wherein the number of the wheel contact points in the screened target contact point set is less than or equal to the second threshold.
13. A computer-readable storage medium, comprising a stored program, wherein the program when executed performs the method of any of claims 1 to 12.
14. A computer program product comprising computer program/instructions, characterized in that the computer program/instructions, when executed by a processor, implement the steps of the method as claimed in any one of claims 1 to 12.
15. An electronic device comprising a memory and a processor, characterized in that the memory has stored therein a computer program, the processor being arranged to execute the method of any of claims 1 to 12 by means of the computer program.
CN202210232479.6A 2022-03-09 2022-03-09 Method for acquiring wheel touchdown point, storage medium, and electronic apparatus Pending CN114627069A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210232479.6A CN114627069A (en) 2022-03-09 2022-03-09 Method for acquiring wheel touchdown point, storage medium, and electronic apparatus

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210232479.6A CN114627069A (en) 2022-03-09 2022-03-09 Method for acquiring wheel touchdown point, storage medium, and electronic apparatus

Publications (1)

Publication Number Publication Date
CN114627069A true CN114627069A (en) 2022-06-14

Family

ID=81900610

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210232479.6A Pending CN114627069A (en) 2022-03-09 2022-03-09 Method for acquiring wheel touchdown point, storage medium, and electronic apparatus

Country Status (1)

Country Link
CN (1) CN114627069A (en)

Similar Documents

Publication Publication Date Title
US20200401611A1 (en) Information processing device, map update method, program, and information processing system
CN109614935B (en) Vehicle damage assessment method and device, storage medium and electronic equipment
EP3042361B1 (en) Estimating depth from a single image
US9177381B2 (en) Depth estimate determination, systems and methods
CN111652050B (en) Traffic sign positioning method, device, equipment and medium
CN113034586B (en) Road inclination angle detection method and detection system
CN116740668B (en) Three-dimensional object detection method, three-dimensional object detection device, computer equipment and storage medium
US20240077331A1 (en) Method of predicting road attributers, data processing system and computer executable code
CN114140592A (en) High-precision map generation method, device, equipment, medium and automatic driving vehicle
CN113592015B (en) Method and device for positioning and training feature matching network
CN114299230A (en) Data generation method and device, electronic equipment and storage medium
CN116740669B (en) Multi-view image detection method, device, computer equipment and storage medium
CN112668675B (en) Image processing method and device, computer equipment and storage medium
CN116823966A (en) Internal reference calibration method and device for camera, computer equipment and storage medium
CN115880555B (en) Target detection method, model training method, device, equipment and medium
CN114882115B (en) Vehicle pose prediction method and device, electronic equipment and storage medium
CN114611635B (en) Object identification method and device, storage medium and electronic device
CN114627069A (en) Method for acquiring wheel touchdown point, storage medium, and electronic apparatus
CN115830073A (en) Map element reconstruction method, map element reconstruction device, computer equipment and storage medium
CN115375857A (en) Three-dimensional scene reconstruction method, device, equipment and storage medium
CN117274526A (en) Neural network model training method and image generating method
CN113362458A (en) Three-dimensional model interpretation method for simulating multi-view imaging, terminal and storage medium
CN108416305A (en) Position and orientation estimation method, device and the terminal of continuous type lane segmentation object
CN113240734B (en) Vehicle cross-position judging method, device, equipment and medium based on aerial view
CN110490066B (en) Target detection method and device based on picture analysis and computer equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 40071523

Country of ref document: HK