WO2023016082A1

WO2023016082A1 - Three-dimensional reconstruction method and apparatus, and electronic device and storage medium

Info

Publication number: WO2023016082A1
Application number: PCT/CN2022/098993
Authority: WO
Inventors: 张保成
Original assignee: 北京迈格威科技有限公司
Priority date: 2021-08-13
Filing date: 2022-06-15
Publication date: 2023-02-16
Also published as: CN113793413A

Abstract

Provided in the present disclosure are a three-dimensional reconstruction method and apparatus, and an electronic device and a storage medium. The method comprises: acquiring a plurality of pieces of monitoring image data, wherein the plurality of pieces of monitoring image data each comprise image data of a vehicle; determining a target image group of a vehicle of a target model according to the plurality of pieces of monitoring image data, wherein the target image group comprises images of the vehicle of the target model at different viewing angles; acquiring a calibration result of a camera corresponding to each image in the target image group; and obtaining a three-dimensional model of the vehicle of the target model according to the target image group and the calibration result of the camera corresponding to each image in the target image group. By means of the method, there is no need to manually hold a three-dimensional scanning device to scan a vehicle, and a three-dimensional vehicle model can be constructed on the basis of monitoring image data, such that the efficiency is higher; moreover, there is no need to use the three-dimensional scanning device and a physical vehicle, such that the cost is lower, and the implementation is easier.

Description

Three-dimensional reconstruction method, device, electronic equipment and storage medium

Cross References to Related Applications

This disclosure claims the priority of the Chinese patent application with application number 202110931999.1 and titled "Three-dimensional reconstruction method, device, electronic equipment and storage medium" filed with the State Intellectual Property Office of China on August 13, 2021, the entire contents of which are incorporated by reference incorporated in this disclosure.

technical field

The present disclosure relates to the technical field of image processing, and in particular, to a three-dimensional reconstruction method, device, electronic equipment, and storage medium.

Background technique

In the field of intelligent transportation, in order to accurately determine the pose of a real vehicle, it is necessary to perform 3D reconstruction of the real vehicle to obtain a 3D model of the vehicle.

In the vehicle 3D reconstruction technology in the related field, for each type of vehicle, it is necessary to place the real vehicle of this type at a certain position, and rely on manual hand-held 3D scanning equipment to scan the vehicle, in order to obtain the model of the vehicle. 3D model of the vehicle.

Since the above method requires manual labor and 3D scanning equipment, the efficiency of the above method is low, and the use of 3D scanning equipment increases the cost of 3D reconstruction of the vehicle to a certain extent; In many cases, the implementation of this method is difficult.

Contents of the invention

In view of this, the purpose of the embodiments of the present disclosure is to provide a three-dimensional reconstruction method, device, electronic equipment and storage medium to solve the above problems.

In a first aspect, an embodiment of the present disclosure provides a three-dimensional reconstruction method, the method comprising: acquiring a plurality of monitoring image data; each of the plurality of monitoring image data includes image data of a vehicle; according to the plurality of monitoring image data, Determine the target image group of the target model vehicle; the target image group includes: images of the target model vehicle at different viewing angles; obtain the calibration results of the cameras corresponding to each image in the target image group; according to the The target image group and the calibration results of the cameras corresponding to the images in the target image group obtain the three-dimensional model of the vehicle of the target model.

In the above implementation process, according to multiple monitoring image data, the images of the vehicle of the target model under different viewing angles are determined, and then the three-dimensional model of the vehicle of the target model is obtained according to the calibration results of each image and the corresponding camera. Manual hand-held 3D scanning equipment scans the vehicle, and the construction of the vehicle 3D model can be realized based on the monitoring image data, which is more efficient; secondly, since there is no need for 3D scanning equipment and physical vehicles, the cost is lower and the implementation is easier.

Based on the first aspect, in a possible design, the determining the target image group of the vehicle of the target model according to the plurality of monitoring image data includes: detecting the vehicle in the plurality of monitoring image data , to obtain a plurality of vehicle images; group the plurality of vehicle images according to the model of the vehicle to obtain at least one image group corresponding to at least one model of the vehicle, each image group includes the corresponding model of the vehicle under different viewing angles a plurality of vehicle images; determining an image group from the at least one image group as the target image group.

In the above-mentioned implementation process, multiple vehicle images are obtained by detecting vehicles in multiple monitoring image data, and then multiple vehicle images are grouped according to vehicle models to prevent subsequent use of A-type vehicle images from Model B vehicles perform 3D reconstruction to improve the accuracy of 3D reconstruction; secondly, when grouping, because only the images of each vehicle need to be processed, there is no need to process the images of other objects in the surveillance video, and avoid the distortion of other objects. The image interferes with the grouping, thereby reducing the complexity of the grouping, and improving the grouping efficiency and grouping accuracy.

Based on the first aspect, in a possible design, before determining an image group from the at least one image group as the target image group, the method further includes: for each of the at least one image group An image group, and the vehicle images belonging to the same viewing angle in the image group are deduplicated.

In the above implementation process, for each image group, the vehicle images belonging to the same viewing angle in the image group are deduplicated, so as to reduce the complexity of 3D reconstruction using the image group and improve the efficiency of 3D reconstruction.

Based on the first aspect, in a possible design, according to the target image group and the calibration results of the cameras corresponding to the images in the target image group, obtaining the 3D model of the vehicle of the target model includes: obtaining the The key point information group corresponding to each image in the target image group; the key point information group includes: the position of a plurality of two-dimensional key points representing the outline of the vehicle in the corresponding image in the image; according to the target image group The key point information group corresponding to each image in the image and the calibration result of the camera that captured each image are used to obtain the 3D model of the vehicle of the target model.

In the above implementation process, considering the position of multiple two-dimensional key points in the vehicle image that characterize the vehicle outline in the image, the camera calibration results corresponding to the image, and the three-dimensional model of the vehicle, there is a mapping relationship. Therefore, By obtaining the key point information group corresponding to each image in the target image group; the key point information group includes: the position of multiple two-dimensional key points representing the outline of the vehicle in the corresponding image in the image; and then according to each of the target image group The key point information group corresponding to the image and the calibration result of the camera that took each image can obtain the 3D model of the vehicle of the target model, without using the position information of all points in the target image group in the image, and then improve the efficiency of 3D reconstruction.

Based on the first aspect, in a possible design, the 3D model of the vehicle of the target model is obtained according to the key point information group corresponding to each image in the target image group and the calibration result of the camera that captures each image , comprising: determining the initial three-dimensional model of the vehicle of the target model, the initial three-dimensional model comprising: each three-dimensional key point constituting the three-dimensional model, and the initial coordinates of each three-dimensional key point in the model coordinate system; for the target image For each image in the group, determine the initial pose of the vehicle in the image in the world coordinate system when the image is taken; according to the key point information group corresponding to each image in the target image group and capture the respective images Based on the calibration result of the camera, the initial pose of the vehicle in each image and the initial coordinates of the three-dimensional key points of the initial three-dimensional model are optimized by using the bundle adjustment method to obtain the three-dimensional model of the vehicle of the target model.

In the above implementation process, after determining the initial 3D model of the vehicle of the target model and the initial position and posture of the vehicle in the world coordinate system in each image, according to the key point information group and the shooting data corresponding to each image in the target image group The camera calibration results of each image optimize the initial pose of the vehicle in each image and the initial coordinates of the 3D key points of the initial 3D model to reduce the impact of noise on the 3D reconstruction results and improve the accuracy of the 3D reconstruction.

Based on the first aspect, in a possible design, according to the key point information group corresponding to each image in the target image group and the calibration result of the camera that captured each image, the initial pose of the vehicle in each image , and the initial coordinates of the three-dimensional key points of the initial three-dimensional model are optimized to obtain the three-dimensional model of the vehicle of the target model, including: for each image in the target image group, according to the initial three-dimensional model, the The camera calibration result and initial pose corresponding to the image, determine the position of the initial projection point corresponding to the image in the image coordinate system; the initial projection point includes the initial three-dimensional model corresponding to the two-dimensional key point of the image The three-dimensional key point is projected to the point in the image coordinate system under the initial pose corresponding to the image; according to the position difference between each initial projection point corresponding to the image and the corresponding two-dimensional key point, determine the corresponding The first loss value of the image; according to the first loss value corresponding to each image, the initial coordinates and each initial pose of the 3D key point of the initial 3D model are optimized until the optimized 3D model and the optimized position are used The new loss value determined by the attitude satisfies the preset condition; the optimized three-dimensional model is the three-dimensional model of the vehicle of the target model.

In the above implementation process, for each image, according to the initial three-dimensional model, the camera calibration result corresponding to the image, and the initial pose, determine the position of the initial projection point corresponding to the image in the image coordinate system, and then according to the initial projection point and the position difference between the corresponding two-dimensional key points, determine the first loss value corresponding to the image, and according to the first loss value of each image, the initial coordinates of the three-dimensional key points of the initial three-dimensional model and corresponding to each image The initial pose is optimized until the new loss value determined by using the optimized 3D model and the optimized pose meets the preset conditions, then the optimization is stopped, and then the accuracy of the final 3D model can be guaranteed.

Based on the first aspect, in a possible design, obtaining the calibration result of the camera corresponding to each image in the target image group includes: determining each image in the target image group according to the plurality of surveillance image data The calibration results of the corresponding cameras.

In the above implementation process, the monitoring image data is used to determine the calibration result of the camera, so as to ensure that the subsequent three-dimensional reconstruction of the vehicle of the target model can be performed according to the calibration result of the camera.

In a second aspect, an embodiment of the present disclosure provides a three-dimensional reconstruction device, the device comprising: an acquisition unit configured to acquire a plurality of monitoring image data; the plurality of monitoring image data all include image data of a vehicle; An image group determining unit, configured to determine a target image group of a target model vehicle according to the plurality of monitoring image data; the target image group includes: images of the target model vehicle under different viewing angles; calibration result acquisition A unit configured to obtain the calibration result of the camera corresponding to each image in the target image group; a three-dimensional model obtaining unit configured to obtain the calibration result of the camera corresponding to each image in the target image group and the target image group A 3D model of the vehicle of the target model.

Based on the second aspect, in a possible design, the image group determination unit includes: a detection unit configured to detect vehicles in the plurality of monitoring image data to obtain a plurality of vehicle images The grouping unit may be configured to group the plurality of vehicle images according to the model of the vehicle to obtain at least one image group corresponding to at least one model of the vehicle, each image group including the vehicle of the corresponding model in A plurality of vehicle images under different viewing angles; the selecting unit may be configured to determine one image group from the at least one image group as the target image group.

Based on the second aspect, in a possible design, the device further includes: a deduplication unit configured to, for each image group in the at least one image group, The image of the vehicle under the perspective is deduplicated.

Based on the second aspect, in a possible design, the 3D model obtaining unit includes: an information group obtaining unit configured to obtain the key point information group corresponding to each image in the target image group; The key point information group includes: the position of multiple two-dimensional key points representing the outline of the vehicle in the corresponding image in the image; the three-dimensional model obtaining subunit can be configured to The corresponding key point information group and the calibration results of the cameras that capture the images are used to obtain the three-dimensional model of the vehicle of the target model.

Based on the second aspect, in a possible design, the 3D model obtaining subunit includes: an initial model determining unit configured to determine the initial 3D model of the vehicle of the target model, the initial 3D model The model includes: each 3D key point constituting the 3D model, and the initial coordinates of each 3D key point in the model coordinate system; the initial pose determination unit may be configured to, for each image in the target image group, Determining the initial pose of the vehicle in the image in the world coordinate system when the image is captured; the optimization unit may be configured to set and capture the key point information corresponding to each image in the target image group The camera calibration results of each image are optimized for the initial pose of the vehicle in each image and the initial coordinates of the 3D key points of the initial 3D model to obtain the 3D model of the vehicle of the target model.

Based on the second aspect, in a possible design, the optimization unit includes: a projection unit configured to, for each image in the target image group, according to the initial three-dimensional model, the image Corresponding to the camera calibration result and the initial pose, determine the position of the initial projection point corresponding to the image in the image coordinate system; the initial projection point includes the three-dimensional corresponding to the two-dimensional key point of the image in the initial three-dimensional model The key point is projected to a point in the image coordinate system under the initial pose corresponding to the image; the loss determination unit may be configured to be based on the relationship between each initial projected point corresponding to the image and the corresponding two-dimensional key point Determine the first loss value corresponding to the image according to the position difference between them; the optimization subunit may be configured to calculate the initial coordinates and Optimizing the initial poses corresponding to each image until a new loss value determined using the optimized three-dimensional model and the optimized pose meets the preset condition; the optimized three-dimensional model is the vehicle of the target model 3D model of .

Based on the second aspect, in a possible design, the calibration result acquisition unit may be configured to determine the calibration of the camera corresponding to each image in the target image group according to the plurality of surveillance image data result.

In a third aspect, an embodiment of the present disclosure provides an electronic device, including a processor and a memory connected to the processor, where a computer program is stored in the memory, and when the computer program is executed by the processor, the The electronic device executes the method described in the first aspect.

In a fourth aspect, an embodiment of the present disclosure provides a storage medium, where a computer program is stored in the storage medium, and when the computer program runs on a computer, the computer executes the method described in the first aspect.

Additional features and advantages of the disclosure will be set forth in the description which follows, and in part will be apparent from the description, or may be learned by practice of the disclosed embodiments.

Description of drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following will briefly introduce the accompanying drawings used in the embodiments. It should be understood that the following drawings only show some embodiments of the present disclosure, and therefore are not It should be regarded as a limitation on the scope, and those skilled in the art can also obtain other related drawings based on these drawings without creative work.

FIG. 1 is a schematic flowchart of a three-dimensional reconstruction method provided by an embodiment of the present disclosure.

FIG. 2 is a schematic structural diagram of a three-dimensional reconstruction device provided by an embodiment of the present disclosure.

FIG. 3 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure.

Icons: 200-three-dimensional reconstruction device; 210-acquisition unit; 220-image group determination unit; 230-calibration result acquisition unit; 240-three-dimensional model acquisition unit; 300-electronic equipment; 301-processor; 302-memory; 303- Communication Interface.

Detailed ways

The technical solutions in the embodiments of the present disclosure will be described below with reference to the drawings in the embodiments of the present disclosure.

It should be noted that like numerals and letters denote similar items in the following figures, therefore, once an item is defined in one figure, it does not require further definition and explanation in subsequent figures. Meanwhile, in the description of the present disclosure, the terms "first", "second", etc. are only used to distinguish descriptions, and cannot be understood as indicating or implying relative importance.

Please refer to FIG. 1 . FIG. 1 is a flow chart of a three-dimensional reconstruction method provided by an embodiment of the present disclosure. The process shown in FIG. 1 will be described in detail below, and the method includes steps: S11-S14.

S11: Acquire a plurality of monitoring image data; the plurality of monitoring image data all include image data of a vehicle.

S12: Determine a target image group of the target model vehicle according to the plurality of monitoring image data; the target image group includes: images of the target model vehicle under different viewing angles.

S13: Obtain a calibration result of the camera corresponding to each image in the target image group.

S14: According to the target image group and the calibration results of the cameras corresponding to the images in the target image group, obtain a three-dimensional model of the vehicle of the target model.

The above method will be described in detail below.

In an actual implementation process, S11 may be implemented in the following manner, acquiring a plurality of monitoring image data captured by at least one camera from a third party, wherein the plurality of monitoring image data all include vehicle image data.

In the above implementation process, multiple monitoring image data can be acquired without directly communicating with at least one camera, and the complexity of communication connection is low, especially when the number of at least one camera is large, the effect is more obvious.

As an implementation, S11 can be implemented in the following manner, acquiring multiple monitoring image data captured by at least one camera within a specified time period after the current moment, so as to ensure that the acquired multiple monitoring image data contains the latest vehicle model Image.

Wherein, in this embodiment, the latest vehicle model may be a model of a vehicle produced within one year before the current moment, and in other embodiments, the latest vehicle model may also be a model of a vehicle produced within half a year before the current moment.

As an implementation manner, S11 may be implemented in the following manner, acquiring a plurality of monitoring image data sent by at least one camera.

Wherein, the image may be a picture or a video. In each of the above-mentioned embodiments, if the number of cameras is at least two, each camera in the at least two cameras can be set at different geographical locations or different angles respectively, so as to ensure that images from different angles can be taken; in the above-mentioned In various embodiments, the camera may be a camera that is installed on roads inside the park, parking lots inside the company, etc., and can capture images of vehicles.

Wherein, the target model is the model of the vehicle whose three-dimensional model needs to be established.

In the actual implementation process, S12 can be implemented in the following manner, based on a predetermined target model, using a pre-trained vehicle model recognition model to identify a plurality of images of vehicles belonging to the target model from a plurality of monitoring image data, And divide the multiple images of the vehicle of the target model into the target image group.

Among them, in order to realize the three-dimensional reconstruction of the vehicle, the target image group needs to contain images of the vehicle under multiple viewing angles. Specifically, for example, the target image group needs to include images of the front surface of the target model vehicle, including the target model vehicle The image of the left side of the vehicle, the image of the right side of the vehicle including the target model, the image of the rear surface of the vehicle including the target model, and the image of the roof of the vehicle including the target model. It can be understood that, considering the symmetry of the vehicle, in the case that the target image group already includes images under some viewing angles, the target image group may not include images under the viewing angles corresponding to this part of the viewing angles. For example, if the target image group already includes the image of the left side of the vehicle, the image of the right side of the vehicle may not be included.

Specifically, for an image of a vehicle, the specific implementation manner of identifying which surface image of the vehicle the image belongs to may be: according to the image of the vehicle, determine the information of a plurality of two-dimensional key points that characterize the outline of the vehicle in the image, and then According to the information of the two-dimensional key points, it can be determined which surface image of the vehicle the image belongs to. For example, according to the image of the vehicle, it is determined that a plurality of two-dimensional key points representing the outline of the vehicle in the image include the key points of the left window, the key points of the left front wheel, the key points of the left rear wheel, etc., then it can be determined that the image is Left view.

Wherein, for each image, the origin of the image coordinate system corresponding to the image can be the center of the image or the vertex of the image, one of the u axis and the v axis of the image coordinate system is parallel to the upper edge of the image, and the image coordinate The v-axis of the system, the other of the v-axis is parallel to the lower edge of the image.

As an implementation, S12 can be implemented in the following manner. For the image of each vehicle in a plurality of surveillance video data, input the image of the vehicle into the pre-trained vehicle model recognition model to obtain the vehicle model, and use the same The images captured by the vehicle model under different viewing angles are divided into an image group, and then from at least one of the divided image groups, one image group is sequentially or randomly determined as the target image group; or, considering that in order to construct the target model For the three-dimensional model of the vehicle, it is necessary to ensure that the target image group includes images of each surface of the target model vehicle. Therefore, from at least one image group obtained by dividing, the number of images meets a certain requirement (for example, greater than or equal to 200) or The image group required by the angle distribution (eg, distribution at certain specific angles) is the target image group. It can be understood that, because the larger the number of images in the target image group, the more ensured that the images of the various surfaces of the vehicle of the target model can be determined from the target image group.

As an implementation manner, S12 includes steps: A1-A3.

A1: Detecting vehicles in the multiple monitoring image data to obtain multiple vehicle images.

In the actual implementation process, A1 can be implemented in the following manner. For each image in multiple monitoring image data, use a pre-trained vehicle detection model to detect the vehicle in the image to obtain the position of the vehicle detection frame. The vehicle image can be cropped according to the position of the vehicle detection frame.

After the vehicle images of each vehicle are acquired, step A2 is performed.

A2: Group the vehicle images of each vehicle according to the vehicle model to obtain at least one more image group corresponding to at least one model of vehicle, and each image group includes images of the corresponding model of the vehicle under different viewing angles.

In the actual implementation process, A2 can be implemented in the following manner. For each vehicle image, use the pre-trained vehicle brand recognition model to identify the vehicle brand in the vehicle image, wherein the vehicle image can include : The vehicle logo or logo of the vehicle; after determining the brand of the vehicle in the vehicle image, use the pre-trained vehicle model detection model corresponding to the brand of the vehicle to group the images of vehicles belonging to the same brand , dividing vehicle images of vehicles of the same model under various viewing angles into an image group to obtain at least one image group corresponding to at least one type of vehicle one-to-one.

It is worth mentioning that for each image group, since the larger the number of images in the image group, the more perspectives involved in the image group can be ensured. Therefore, the number of images in the image group needs to meet a certain Requirements (for example, the number of images in the group is greater than or equal to 200, or 3000), to ensure that the group of images includes vehicle images of vehicles of the corresponding model at various angles of view, and then ensure that the three-dimensionality of the vehicle can be accurately determined in the future Model.

As an implementation, A2 can be implemented in the following manner. For each vehicle image, input the vehicle image into a pre-trained vehicle model recognition model to obtain the vehicle model, and then divide the vehicle images of the same model into As an image group, at least one image group corresponding to at least one model of vehicle is obtained.

Wherein, the training methods of the vehicle brand model and the vehicle model detection model are well-known techniques in the art, so details will not be repeated here.

After obtaining at least one image group, step A3 may be performed.

A3: Determine an image group from the at least one image group as the target image group.

In an actual implementation process, A3 may be implemented in the following manner, one image group is sequentially determined from the at least one image group as the target image group.

As an implementation manner, A3 may be implemented in the following manner, randomly determining an image group from the at least one image group as the target image group.

As an implementation manner, A3 may be implemented in the following manner, using an image group whose number of images satisfies the requirement as a target image group.

As an implementation manner, A3 may be implemented in the following manner. First, the target model is determined, and then the image group corresponding to the target model is used as the target image group.

It is worth mentioning that after determining an image group from at least one image group as the target image group, and using the target image group to perform steps S13-S14, each image group in the remaining image groups can be sequentially As the target image group, S13-S14 is then executed using the image group as the target image group, so as to determine the three-dimensional models of vehicles of multiple models.

After the target image group is determined, step S13 is executed.

Wherein, the camera calibration result may include the internal reference matrix K of the camera, the external reference matrix P of the camera, or the matrix S obtained by multiplying the internal reference matrix K of the camera and the external reference matrix P of the camera.

The specific calibration method may use any camera calibration method, which is not limited here. There are three main types of camera calibration methods: the first type of camera calibration method needs to rely on the placed calibration object; the second type of camera calibration method mainly uses the camera’s motion information to calibrate the camera, this method does not need to rely on the calibration object, but needs to control the camera Do some special sports, and this method is not suitable for scenes where the motion information is unknown or the camera movement cannot be controlled (for example, security monitoring scenes). The third category is based on the target calibration captured by the camera.

A specific method of camera calibration based on the target captured by the camera is as follows:

Obtain the 3D model of the known target (the 3D model of the known target can be reconstructed by methods such as manual scanning, and the 3D model includes multiple 3D key points); obtain the pose of the known target in the world coordinate system (so it can be determined The position of multiple 3D key points in the world coordinate system and the attitude of the 3D model in the world coordinate system); use the camera to be calibrated to shoot the known target to obtain the calibration image; key the known target in the calibration image Point detection, to obtain each two-dimensional key point of the known target and the position (u, v) of the two-dimensional key point in the image coordinate system; according to the corresponding relationship between the two-dimensional key point and the three-dimensional key point in the three-dimensional model (for For any 2D key point of the target in any calibration image, it is bound to be able to find the 3D key point corresponding to the position of the 2D key point from the 3D model of the target. For any 3D key point in the 3D model of the target key point, there may not be a two-dimensional key point corresponding to the position of the three-dimensional key point in the calibration image; In the 3D model, the 3D key point corresponding to the 2D key point in the upper left corner of the front door is found, and the corresponding 3D key point is the point in the upper corner of the front door in the 3D model of the vehicle), and the known target can be determined The position (x _m , y _m , z _m ) of the 3D key point corresponding to the 2D key point in the 3D model of the The position (u, v) of the 2D key point in the image coordinate system, the position (x _m , y _m , z _m ) of the 3D key point corresponding to the 2D key point in the 3D model in the world coordinate system, and the position of the 3D key point in the world coordinate system The pose of the model in the world coordinate system when the calibration image is taken (the position is represented by (X, Y), and the attitude can be represented by the heading angle θ) satisfies the following relational expression:

Among them, λ is a constant,

Characterize the pose of a known object in the world coordinate system.

K and P or the product of the two can be determined by the above expressions.

Wherein, when the known 3D model is the 3D model of the vehicle, the xoy plane of the world coordinate system may overlap with the road where the vehicle is located. The origin of the world coordinate system may be a projection point corresponding to the center of the camera on the road where the vehicle is located.

In this way, a small amount of 3D models can be established through manual scanning to obtain camera calibration results, so that a large number of 3D models can be automatically reconstructed through the camera calibration results.

As an implementation manner, S13 includes: determining a calibration result of a camera corresponding to each image in the target image group according to the plurality of surveillance image data.

It can be understood that the multiple surveillance image data include multiple models of vehicles, some of which have unknown 3D models of vehicles, and some of which have known 3D models of vehicles. Therefore, automatic calibration of the camera can be performed based on the vehicle whose 3D model is known.

In the actual implementation process, S13 can be implemented in the following manner: use the cameras corresponding to each image in the target image group as the camera to be calibrated sequentially; find out the monitoring image data captured by the camera to be calibrated from a plurality of monitoring image data, From the monitoring image data captured by the camera to be calibrated, it is determined that there are multiple images of a target with a known 3D model (for example, if the 3D model of a certain type of vehicle is known, then the target of the known 3D model is the vehicle of this type) , determine the coordinates of the 3D key points representing the target according to the known 3D model, and detect the coordinates of the 2D key points representing the target in the multiple images according to the multiple images captured by the camera, and then according to the target’s The three-dimensional coordinates of each three-dimensional key point and the two-dimensional coordinates of the corresponding two-dimensional key point determine the calibration result of the camera to be calibrated.

In this way, after a small number of 3D models are determined by means of manual scanning, automatic camera calibration and automatic reconstruction of a large number of 3D models can be realized.

After the multiple cameras are calibrated, the corresponding relationship between the camera identification and the camera calibration result is stored.

As an implementation manner, S13 can be implemented in the following manner. Obtain the identifier of the camera corresponding to each image in the target image group, and then, for the identifier of the camera corresponding to each image, obtain the corresponding relationship between the camera identifier and the camera calibration result stored in advance. In , find out the camera calibration result corresponding to the identity of the camera corresponding to the image.

Wherein, the three-dimensional model of the vehicle of the target model includes: three-dimensional key points representing the profile of the vehicle of the target model, and the relative positional relationship of each three-dimensional key point (for example, a certain three-dimensional key point can be the origin, parallel to the plane where the chassis of the vehicle is located) The plane is the coordinate plane to establish a model coordinate system, and the coordinate values of each three-dimensional key point in the model coordinate system can represent the relative positional relationship between each three-dimensional key point); the three-dimensional key points contained in the three-dimensional model of each type of vehicle The type and quantity of the model can be consistent, and the 3D key points involved in each 3D model can include: each car light, each window, each door, each wheel, the front surface of the car, the rear surface of the car, the roof surface, etc. point and so on.

It can be understood that the model coordinate system can be a three-dimensional coordinate system, and the model coordinate system and the world coordinate system can only have a translation relationship without a rotation or scaling relationship; the origin of the model coordinate system can be a vehicle in an image in the target image group The projection point of the center point of the front of the vehicle on the road where the vehicle is located. The z-axis of the model coordinate system can be perpendicular to the road where the vehicle is located. The x-axis of the model coordinate system can be parallel to the central axis of the vehicle. axis is perpendicular to the z-axis.

As an implementation manner, S14 includes steps: B1-B2.

B1: Obtain the key point information group corresponding to each image in the target image group; the key point information group includes: the positions of multiple two-dimensional key points representing the outline of the vehicle in the corresponding image in the image.

In the actual implementation process, B1 can directly acquire multiple two-dimensional key points that characterize the outline of the vehicle in the image determined in the aforementioned step S12 for each image in the target image in the following manner: position; wherein, the positions of multiple two-dimensional key points in the image constitute the key point information group of the image. It can be understood that in some vehicle detection or vehicle model detection models, the vehicle key point detection is performed while the vehicle detection or vehicle model detection is performed, so that in step S12, multiple two-dimensional key points have been determined in the image position in .

As an implementation, B1 can be implemented in the following manner. For each image in the target image group, use the pre-trained vehicle key point extraction model to extract the key points of the image, so as to determine the representative vehicle from the image The positions of multiple 2D keypoints of the contour of the image in the image. It can be understood that in some vehicle detection or vehicle model detection models, vehicle key point detection is performed without vehicle detection or vehicle model detection, so additional steps are required to determine the positions of multiple two-dimensional key points in the image.

B2: Obtain the 3D model of the vehicle of the target model according to the key point information group corresponding to each image in the target image group and the calibration result of the camera that captures each image.

In the above implementation process, considering the position of multiple two-dimensional key points in the vehicle image that characterize the vehicle outline in the image, the camera calibration results corresponding to the image, and the three-dimensional model of the vehicle, there is a mapping relationship. Therefore, First obtain the key point information group corresponding to each image in the target image group; the key point information group includes: the positions of multiple two-dimensional key points representing the outline of the vehicle in the corresponding image in the image; then according to each of the target image group The key point information group corresponding to the image and the calibration result of the camera that captures each image can obtain the 3D model of the vehicle of the target model.

In this way, the three-dimensional reconstruction of the car models contained in the images can be automatically performed through a large number of multi-angle images, and then the efficiency and accuracy of the three-dimensional reconstruction can be improved.

As an implementation manner, B2 includes steps B21-B23.

B21: Determine the initial three-dimensional model of the vehicle of the target model, the initial three-dimensional model includes: each three-dimensional key point constituting the three-dimensional model, and the initial coordinates of each three-dimensional key point in the model coordinate system.

Among them, the model coordinate system is a three-dimensional coordinate system, which is established according to user requirements without limitation.

The types and quantities of 3D key points contained in the initial 3D models of vehicles of various models may be consistent, and the 3D key points involved in each 3D model may include: each lamp, each door, each window, each wheel, vehicle Points on the periphery of the front surface, rear surface, roof surface, etc.

A 3D model of a vehicle with a known model can be used as the initial 3D model, or a unified initial 3D model can be specified for a certain type of vehicle. For example, the initial 3D models of vehicles belonging to the same class can be the same by default, for example, cars belong to the same class, trucks belong to the same class, and non-motor vehicles belong to the same class.

B22: For each image in the target image group, determine the initial pose of the vehicle in the image in the world coordinate system when the image is captured.

Among them, the initial pose can be determined according to the two-dimensional key points detected in each image combined with camera calibration information, or can be determined according to the pose of the same vehicle in the image captured at the associated time, and can also be a default value determined based on experience.

For example, the two-dimensional key points detected in the first image are key points on the left rear wheel, left window, left front glass, left rear glass, etc., and the camera can be determined according to the camera calibration results of the camera that captured the first image. is erected parallel to the road, an initial pose can be estimated.

For example, if the pose of vehicle A in the first image is known, the second image also includes vehicle A, and the second image is captured by the same camera as the first image, and the shooting time interval is less than a certain length of time (for example, 3s), then the first The pose of vehicle A in the image is estimated for the pose of the vehicle in the second image, for example, the pose of vehicle A in the first image can be used as the initial pose of vehicle A in the second image.

For example, the initial pose of the vehicle in the world coordinate system when the image was taken in each image can be set to the same default value.

B23: According to the key point information group corresponding to each image in the target image group and the calibration result of the camera that captured each image, the initial pose of the vehicle in each image, and the 3D key points of the initial 3D model The initial coordinates are optimized to obtain the 3D model of the vehicle of the target model.

The method for optimizing the initial pose of the vehicle in each image and the initial coordinates of the 3D key points of the initial 3D model may be bundle adjustment.

After determining the initial 3D model of the vehicle of the target model and the initial pose of the vehicle in the world coordinate system in each image, it is necessary to determine the initial pose of the vehicle in each image and the 3D key points of the initial 3D model. The initial coordinates are optimized, and the initial pose and initial model can be corrected to obtain an accurate 3D model.

As an implementation manner, B23 includes steps: B231-B233.

B231: For each image in the target image group, according to the initial three-dimensional model, the camera calibration result corresponding to the image, and the initial pose, determine the position of the initial projection point corresponding to the image in the image coordinate system; The initial projection point includes a point where a 3D key point corresponding to a 2D key point of the image in the initial 3D model is projected into the image coordinate system at an initial pose corresponding to the image.

It can be understood that the initial pose of the vehicle is the initial pose of the 3D model corresponding to the vehicle in the world coordinate system.

In the actual implementation process, B231 can be implemented in the following manner. For each image, according to the corresponding two-dimensional key point information group of the image, from the initial three-dimensional model, determine the corresponding two-dimensional key points in the image The initial coordinates of the three-dimensional key points, for each determined three-dimensional key point, the initial coordinates (x, y, z) of the three-dimensional key point in the model, the camera calibration result S and the initial pose T corresponding to the image, Input to projection expression

In , the position (u', v') of the initial projection point corresponding to the 3D key point in the image coordinate system corresponding to the image is obtained.

B232: Determine the first loss value corresponding to the image for the position difference between each initial projection point corresponding to the image and the corresponding dimensional key point.

In the actual implementation process, B232 can be implemented in the following manner, for each initial projection point corresponding to the image, determine the position (u', v') of the initial projection point in the image coordinate system corresponding to the image and the The distance between the positions (u, v) of the two-dimensional key points corresponding to the three-dimensional key points corresponding to the initial projection point is determined as the first loss value as the sum of the respective distances corresponding to the image.

B233: According to the first loss value corresponding to each image, optimize the initial coordinates of the 3D key points of the initial 3D model and the initial pose corresponding to each image, until the optimized 3D model and the optimized pose are used The determined new loss value satisfies a preset condition; the optimized three-dimensional model is the three-dimensional model of the vehicle of the target model.

According to the first loss value corresponding to each image, the initial coordinates of the 3D key points of the initial 3D model and the initial pose corresponding to each image are optimized, which can be based on the sum of the first loss values corresponding to each image, for The initial coordinates of the 3D key points of the initial 3D model and the initial poses corresponding to each image are optimized.

The preset condition can be one of the following: the sum of the loss values corresponding to each image converges; the sum of the loss values corresponding to each image is the minimum value in previous iterations; the loss value corresponding to each image is less than the target loss value; the corresponding loss value of each image The sum of the loss values of is less than the preset value; the number of iterations reaches the preset number.

In the actual implementation process, B233 can be implemented in the following manner. According to the first loss value corresponding to each image in the target image group, the initial coordinates of the 3D key points of the initial 3D model and the initial poses corresponding to each image The parameters are optimized to obtain the optimized 3D model and the optimized pose; for each image in the target image group, according to the corresponding 2D key point information group of the image, from the optimized 3D model, determine Get the coordinates of the three-dimensional key points corresponding to the two-dimensional key points in the image, and for each three-dimensional key point, the coordinates (x, y, z) of the three-dimensional key points, the camera calibration result S corresponding to the image and The optimized pose T is input to the projection expression

, the position (u', v') of the optimized projection point corresponding to the 3D key point in the image coordinate system corresponding to the image is obtained, according to each optimized projection point corresponding to the image and the corresponding 2D The position difference between the key points, determine the second loss value corresponding to the image, and stop the optimization when the sum of the second loss values corresponding to each image is determined to be less than or equal to the preset value, wherein the optimized three-dimensional model is the target A 3D model of the vehicle.

In the above implementation process, for each image, according to the initial three-dimensional model, the camera calibration result corresponding to the image, and the initial pose, determine the position of the initial projection point corresponding to the image in the image coordinate system, and then according to the initial projection point and the position difference between the corresponding two-dimensional key points, determine the first loss value corresponding to the image, and according to the first loss value of each image, the initial coordinates and initial pose of the three-dimensional key points of the initial three-dimensional model Optimization, until the new loss value determined by using the optimized 3D model and the optimized pose meets the preset condition, the optimization is stopped, and then the accuracy of the final 3D model can be guaranteed.

As an implementation manner, before step A3, the method further includes: for each image group in the at least one image group, deduplicating the vehicle images belonging to the same viewing angle in the image group.

Specifically, for each image group, calculate the similarity of the vehicle outline in any two vehicle images in the image group; when it is determined that the similarity is greater than the preset threshold, it is determined that the two vehicle images belong to the same viewing angle Otherwise, it is determined that the two images belong to vehicle images under different viewing angles; and then the vehicle images belonging to the same viewing angle in the image group are deduplicated. Wherein; in this embodiment, the preset threshold may be any value in 79%-90%.

Please refer to FIG. 2 . FIG. 2 is a structural block diagram of a three-dimensional reconstruction apparatus 200 provided by an embodiment of the present disclosure. The structural block diagram shown in Figure 2 will be described below, and the shown devices include:

The obtaining unit 210 may be configured to obtain a plurality of monitoring image data; the plurality of monitoring image data all include image data of a vehicle.

The image group determining unit 220 may be configured to determine the target image group of the target model vehicle according to the plurality of surveillance image data; the target image group includes: the target model vehicle under different viewing angles image.

The calibration result obtaining unit 230 may be configured to obtain a calibration result of the camera corresponding to each image in the target image group.

The 3D model obtaining unit 240 may be configured to obtain the 3D model of the vehicle of the target model according to the target image group and the calibration results of the cameras corresponding to the images in the target image group.

As an implementation manner, the image group determination unit 220 includes: a detection unit configured to detect vehicles in the plurality of monitoring image data to obtain a plurality of vehicle images; a grouping unit may be It is configured to group the plurality of vehicle images according to the model of the vehicle to obtain at least one image group corresponding to at least one model of the vehicle, and each image group includes a plurality of images of the corresponding model of the vehicle under different viewing angles. The vehicle image; selecting unit may be configured to determine one image group from the at least one image group as the target image group.

As an implementation manner, the device further includes: a deduplication unit, which may be configured to, for each image group in the at least one image group, deduplicate the vehicle images belonging to the same viewing angle in the image group. Heavy.

As an implementation manner, the 3D model obtaining unit 240 includes: an information group obtaining unit configured to obtain a key point information group corresponding to each image in the target image group; the key point information group includes : The position of multiple two-dimensional key points representing the outline of the vehicle in the corresponding image in the image; the three-dimensional model obtaining subunit can be configured to set the key point information corresponding to each image in the target image group and the calibration results of the cameras that capture the respective images to obtain a three-dimensional model of the vehicle of the target model.

As an implementation manner, the 3D model obtaining subunit includes: an initial model determining unit configured to determine the initial 3D model of the vehicle of the target model, and the initial 3D model includes: Each three-dimensional key point, and the initial coordinates of each three-dimensional key point in the model coordinate system; the initial pose determination unit may be configured to determine the position of the vehicle in the image for each image in the target image group The initial pose of the image in the world coordinate system when the image is taken; the optimization unit may be configured to be based on the key point information group corresponding to each image in the target image group and the calibration result of the camera that captures each image , optimizing the initial pose of the vehicle in each image and the initial coordinates of the 3D key points of the initial 3D model to obtain the 3D model of the vehicle of the target model.

As an implementation manner, the optimization unit includes: a projection unit, configured to, for each image in the target image group, according to the initial 3D model, the camera calibration result corresponding to the image, and the initial Pose, determine the position of the initial projection point corresponding to the image in the image coordinate system; the initial projection point includes the three-dimensional key point corresponding to the two-dimensional key point of the image in the initial three-dimensional model in the corresponding position of the image Points projected into the image coordinate system under the initial pose; the loss determination unit may be configured to determine the corresponding The first loss value of the image; the optimization subunit is used to optimize the initial coordinates of the three-dimensional key points of the initial three-dimensional model and the initial pose corresponding to each image according to the first loss value corresponding to each image, Until the new loss value determined by using the optimized three-dimensional model and the optimized pose satisfies the preset condition; the optimized three-dimensional model is the three-dimensional model of the vehicle of the target model.

As an implementation manner, the calibration result acquisition unit 230 may be configured to determine, according to the plurality of monitoring image data, the calibration results of the cameras corresponding to the images in the target image group.

Refer to the content described in the embodiment shown in FIG. 1 above for the process of realizing the respective functions of each functional unit in this embodiment, and details are not repeated here.

Please refer to FIG. 3. FIG. 3 is a schematic structural diagram of an electronic device 300 provided by an embodiment of the present disclosure. The electronic device 300 may be a personal computer, a tablet computer, a smart phone, a personal digital assistant (personal digital assistant, PDA) and the like.

The electronic device 300 may include: a memory 302, a processor 301, a communication interface 303, and a communication bus, and the communication bus is used to implement connection and communication of these components.

The memory 302 is used to store various data such as calculation program instructions corresponding to the three-dimensional reconstruction method and device provided by the embodiments of the present disclosure, wherein the memory 302 may be, but not limited to, random access memory, read only memory (Read Only Memory, ROM), Programmable Read-Only Memory (Programmable Read-Only Memory, PROM), Erasable Programmable Read-Only Memory (EPROM), Electric Erasable Programmable Read-Only Memory (Electric Erasable Programmable Read-Only Memory, Only Memory, EEPROM), etc.

The processor 301 is used to read and run the computer program instructions corresponding to the three-dimensional reconstruction method and device stored in the memory to obtain a plurality of monitoring image data; the plurality of monitoring image data includes image data of vehicles; according to the A plurality of monitoring image data, determine the target image group of the target model vehicle; the target image group includes: the images of the target model vehicle at different angles of view; obtain the corresponding camera of each image in the target image group The calibration result of the target image group and the calibration results of the cameras corresponding to the images in the target image group to obtain a three-dimensional model of the vehicle of the target model.

Wherein, the processor 301 may be an integrated circuit chip, which has a signal processing capability. Above-mentioned processor 301 can be general purpose processor, comprises CPU, network processor (Network Processor, NP) etc.; Can also be digital signal processor (DSP), application specific integrated circuit (ASIC), field programmable gate array (FPGA) ) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. Various methods, steps and logic block diagrams disclosed in the embodiments of the present disclosure may be implemented or executed. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like.

The communication interface 303 is used for receiving or sending data.

In addition, an embodiment of the present disclosure also provides a storage medium, in which a computer program is stored, and when the computer program is run on a computer, the computer is made to execute the method provided by any one of the embodiments of the present disclosure. method.

To sum up, the 3D reconstruction method, device, electronic device, and storage medium proposed by the various embodiments of the present disclosure determine the images of the target model vehicle under different viewing angles based on a plurality of monitoring image data, and then, according to each image and The 3D model of the vehicle of the target model is obtained from the calibration result of the corresponding camera. This method does not need to manually scan the vehicle with a 3D scanning device, and the construction of the 3D model of the vehicle can be realized based on the monitoring image data, which is more efficient. Since there is no need for 3D scanning equipment and physical vehicles, the cost is lower and the implementation is easier.

In the embodiments provided in the present disclosure, it should be understood that the disclosed devices and methods may also be implemented in other ways. The device embodiments described above are only illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the architecture, functions and possible implementations of devices, methods and computer program products according to multiple embodiments of the present disclosure. operate. In this regard, each block in a flowchart or block diagram may represent a module, program segment, or part of code that includes one or more Executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by a dedicated hardware-based device that performs the specified function or action , or may be implemented by a combination of dedicated hardware and computer instructions.

In addition, each functional module in each embodiment of the present disclosure may be integrated together to form an independent part, each module may exist independently, or two or more modules may be integrated to form an independent part.

Industrial Applicability

The present disclosure provides a three-dimensional reconstruction method, device, electronic equipment, and storage medium. The method includes: acquiring a plurality of monitoring image data; the plurality of monitoring image data includes image data of a vehicle; The target image group of the target model vehicle; the target image group includes the images of the target model vehicle under different viewing angles; obtain the calibration results of the cameras corresponding to each image in the target image group; according to the target image group and each image in the target image group The calibration result of the camera corresponding to the image is used to obtain the 3D model of the vehicle of the target model. This method does not need to scan the vehicle with a hand-held 3D scanning device, and can realize the construction of a 3D model of the vehicle based on the monitoring image data, which is more efficient; easy.

In addition, it can be understood that the three-dimensional reconstruction method, device, electronic device and storage medium of the present disclosure are reproducible and can be used in various industrial applications. For example, the three-dimensional reconstruction method, device, electronic equipment, and storage medium of the present disclosure can be used in the technical field of image processing.

Claims

A three-dimensional reconstruction method, characterized in that the method comprises:

Obtaining a plurality of monitoring image data; the plurality of monitoring image data all include image data of the vehicle;

According to the plurality of monitoring image data, a target image group of the target model vehicle is determined; the target image group includes: images of the target model vehicle under different viewing angles;

Acquiring calibration results of cameras corresponding to each image in the target image group;

According to the target image group and the calibration results of the cameras corresponding to the images in the target image group, a three-dimensional model of the vehicle of the target model is obtained.
The method according to claim 1, wherein the determining the target image group of the vehicle of the target model according to the plurality of monitoring image data includes:

Detecting vehicles in the plurality of monitoring image data to obtain a plurality of vehicle images;

Grouping the plurality of vehicle images according to the vehicle type to obtain at least one image group corresponding to at least one type of vehicle, each image group including a plurality of vehicle images of the corresponding type of vehicle under different viewing angles;

A group of images is determined from the at least one group of images as the target group of images.
The method according to claim 2, wherein before determining an image group from the at least one image group as the target image group, the method further comprises:

For each image group in the at least one image group, the vehicle images belonging to the same viewing angle in the image group are deduplicated.
The method according to any one of claims 1-3, characterized in that, according to the target image group and the calibration results of the cameras corresponding to each image in the target image group, the vehicle of the target model is obtained. 3D models, including:

Obtain the key point information group corresponding to each image in the target image group; the key point information group includes: the positions of multiple two-dimensional key points representing the outline of the vehicle in the corresponding image in the image;

According to the key point information group corresponding to each image in the target image group and the calibration result of the camera that captures each image, a three-dimensional model of the vehicle of the target model is obtained.
The method according to claim 4, wherein the three-dimensional model of the vehicle of the target model is obtained according to the key point information group corresponding to each image in the target image group and the calibration result of the camera that captures each image ,include:

determining the initial three-dimensional model of the vehicle of the target model, the initial three-dimensional model including: each three-dimensional key point constituting the three-dimensional model, and the initial coordinates of each three-dimensional key point in the model coordinate system;

For each image in the target image group, determine the initial pose of the vehicle in the image in the world coordinate system when the image is taken;

According to the key point information group corresponding to each image in the target image group and the calibration result of the camera that captured each image, the initial pose of the vehicle in each image and the initial three-dimensional key point of the initial three-dimensional model The coordinates are optimized to obtain a three-dimensional model of the vehicle of the target model.
The method according to claim 5, characterized in that, according to the key point information group corresponding to each image in the target image group and the calibration result of the camera that took the each image, the initial pose of the vehicle in each image , and the initial coordinates of the three-dimensional key points of the initial three-dimensional model are optimized to obtain the three-dimensional model of the vehicle of the target model, including:

For each image in the target image group, according to the initial three-dimensional model, the camera calibration result corresponding to the image, and the initial pose, determine the position of the initial projection point corresponding to the image in the image coordinate system; The initial projected point includes a point in the initial three-dimensional model where the three-dimensional key point corresponding to the two-dimensional key point of the image is projected into the image coordinate system at the initial pose corresponding to the image;

Determining a first loss value corresponding to the image according to the position difference between each initial projection point corresponding to the image and the corresponding two-dimensional key point;

According to the first loss value corresponding to each image, the initial coordinates of the 3D key points of the initial 3D model and the initial pose corresponding to each image are optimized until the optimized 3D model and the optimized pose are used to determine The new loss value satisfies the preset condition; the optimized three-dimensional model is the three-dimensional model of the vehicle of the target model.
The method according to any one of claims 1 to 6, wherein obtaining the calibration result of the camera corresponding to each image in the target image group includes:

According to the plurality of monitoring image data, the calibration result of the camera corresponding to each image in the target image group is determined.
A three-dimensional reconstruction device, characterized in that the device comprises:

an acquisition unit configured to acquire a plurality of monitoring image data; the plurality of monitoring image data includes image data of a vehicle;

The image group determining unit is configured to determine a target image group of a target model vehicle according to the plurality of monitoring image data; the target image group includes: images of the target model vehicle under different viewing angles;

A calibration result acquisition unit configured to acquire a calibration result of the camera corresponding to each image in the target image group;

The three-dimensional model obtaining unit is configured to obtain the three-dimensional model of the vehicle of the target model according to the target image group and the calibration result of the camera corresponding to each image in the target image group.
The three-dimensional reconstruction device according to claim 8, wherein the image group determination unit comprises: a detection unit, a grouping unit, and a selection unit, wherein,

The detection unit is configured to detect vehicles in the plurality of monitoring image data to obtain a plurality of vehicle images;

The grouping unit is configured to group the plurality of vehicle images according to the model of the vehicle to obtain at least one image group corresponding to at least one model of the vehicle, and each image group includes the corresponding model of the vehicle in different Multiple vehicle images from perspective;

The selecting unit is configured to determine one image group from the at least one image group as the target image group.
The three-dimensional reconstruction device according to claim 9, characterized in that the three-dimensional reconstruction device further comprises: a deduplication unit configured to: for each image in the at least one image group group, deduplicating the vehicle images belonging to the same viewing angle in the image group.
The 3D reconstruction device according to any one of claims 8 to 10, wherein the 3D model obtaining unit includes an information group obtaining unit and a 3D model obtaining subunit,

Wherein, the information group acquisition unit is configured to: acquire the key point information group corresponding to each image in the target image group; the key point information group includes: a plurality of two-dimensional information representing the outline of the vehicle in the corresponding image The position of the key point in the image;

Wherein, the three-dimensional model obtaining subunit is configured to: obtain the vehicle of the target model according to the key point information group corresponding to each image in the target image group and the calibration result of the camera that captures each image. 3D model.
The 3D reconstruction device according to claim 11, wherein the 3D model obtaining subunit comprises: an initial model determination unit, an initial pose determination unit and an optimization unit,

Wherein, the initial model determination unit is configured to: determine the initial three-dimensional model of the vehicle of the target model, the initial three-dimensional model includes: each three-dimensional key point constituting the three-dimensional model, and each three-dimensional key point in the model coordinates initial coordinates in the system;

Wherein, the initial pose determining unit is configured to: for each image in the target image group, determine the initial pose of the vehicle in the image in the world coordinate system when the image is captured;

Wherein, the optimization unit is configured to: according to the key point information group corresponding to each image in the target image group and the calibration result of the camera that captures each image, for the initial pose of the vehicle in each image, And the initial coordinates of the 3D key points of the initial 3D model are optimized to obtain the 3D model of the vehicle of the target model.
The three-dimensional reconstruction device according to claim 12, wherein the optimization unit comprises: a projection unit, a loss determination unit, and an optimization subunit,

Wherein, the projection unit is configured to: for each image in the target image group, according to the initial three-dimensional model, the camera calibration result corresponding to the image and the initial pose, determine the initial The position of the projection point in the image coordinate system; the initial projection point includes the projection of the three-dimensional key points corresponding to the two-dimensional key points of the image in the initial three-dimensional model to the image coordinates under the initial pose corresponding to the image points in the system;

Wherein, the loss determination unit is configured to: determine the first loss value corresponding to the image according to the position difference between each initial projection point corresponding to the image and the corresponding two-dimensional key point;

Wherein, the optimization subunit is configured to: optimize the initial coordinates of the 3D key points of the initial 3D model and the initial poses corresponding to each image according to the first loss value corresponding to each image, until using The optimized three-dimensional model and the new loss value determined by the optimized pose satisfy a preset condition; the optimized three-dimensional model is the three-dimensional model of the vehicle of the target model.
The three-dimensional reconstruction device according to any one of claims 8 to 13, wherein the calibration result acquisition unit is configured to: determine the target image group according to the plurality of monitoring image data The calibration results of the cameras corresponding to each image in .
An electronic device, characterized by comprising a memory and a processor, wherein computer program instructions are stored in the memory, and when the computer program instructions are read and executed by the processor, any one of the methods described.
A storage medium, wherein computer program instructions are stored on the storage medium, and when the computer program instructions are read and executed by a computer, the method according to any one of claims 1 to 7 is executed.