WO2022121283A1 - Détection d'informations de point clé de véhicule et commande de véhicule - Google Patents

Détection d'informations de point clé de véhicule et commande de véhicule Download PDF

Info

Publication number
WO2022121283A1
WO2022121283A1 PCT/CN2021/102179 CN2021102179W WO2022121283A1 WO 2022121283 A1 WO2022121283 A1 WO 2022121283A1 CN 2021102179 W CN2021102179 W CN 2021102179W WO 2022121283 A1 WO2022121283 A1 WO 2022121283A1
Authority
WO
WIPO (PCT)
Prior art keywords
target vehicle
information
dimensional coordinate
coordinate system
dimensional
Prior art date
Application number
PCT/CN2021/102179
Other languages
English (en)
Chinese (zh)
Inventor
刘诗男
韩志伟
曾星宇
闫俊杰
王晓刚
Original Assignee
浙江商汤科技开发有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 浙江商汤科技开发有限公司 filed Critical 浙江商汤科技开发有限公司
Priority to KR1020227008917A priority Critical patent/KR20220084021A/ko
Publication of WO2022121283A1 publication Critical patent/WO2022121283A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G06T7/75—Determining position or orientation of objects or cameras using feature-based methods involving models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/06—Topological mapping of higher dimensional structures onto lower dimensional surfaces
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30244—Camera pose
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle

Definitions

  • the present disclosure relates to the technical field of information processing, and in particular, to a vehicle key point information detection method, a vehicle control method, and a corresponding device and vehicle.
  • the driven vehicle it is necessary to automatically identify the driving state of the vehicle around the driven vehicle, so as to control the driven vehicle.
  • pictures of surrounding vehicles can be taken by the driven vehicle, and then the driving states of the surrounding vehicles can be determined according to the recognition results of the pictures, and the driving states of the driven vehicles can be controlled.
  • identifying the surrounding vehicles in the picture taken by the driving vehicle it is necessary to use the key point information of the surrounding vehicles in the picture.
  • the key point information of the vehicle is generally marked on the sample image by manual method, and the marking efficiency and accuracy are relatively low.
  • the recognition accuracy of the driving state of the surrounding vehicles in the picture affects the control strategy of the driven vehicle.
  • the embodiments of the present disclosure provide at least a vehicle key point information detection method, a vehicle control method, and a corresponding device and vehicle.
  • an embodiment of the present disclosure provides a method for detecting vehicle key point information, including: acquiring three-dimensional scan data for a target vehicle and a to-be-detected picture containing the target vehicle; and determining, based on the three-dimensional scan data, all three-dimensional coordinate information of multiple key points on the target vehicle; according to the coordinate conversion relationship between the three-dimensional coordinate system of the target vehicle and the two-dimensional coordinate system of the picture to be detected, and the three-dimensional coordinate system of the multiple key points Coordinate information, to determine the two-dimensional coordinate information of the multiple key points in the to-be-detected picture.
  • the three-dimensional coordinate information of multiple key points on the target vehicle is determined based on the three-dimensional scanning data of the target vehicle, and then based on the coordinate conversion between the three-dimensional coordinate system of the target vehicle and the two-dimensional coordinate system of the picture to be detected
  • the three-dimensional coordinate information of multiple key points is converted into the two-dimensional coordinate information in the image to be detected. Since the three-dimensional coordinate information of the key points can be pre-marked on the three-dimensional vehicle coordinate system corresponding to the target vehicle, it is relatively accurate, so that The two-dimensional coordinate information determined by mapping the three-dimensional coordinate information to the image to be detected is also relatively accurate.
  • determining the three-dimensional coordinate information of multiple key points on the target vehicle based on the three-dimensional scan data includes: determining parameter information of the target vehicle based on the three-dimensional scan data; According to the parameter information, a target vehicle model corresponding to the target vehicle is determined from a variety of pre-built three-dimensional vehicle models; three-dimensional coordinate information of multiple key points pre-marked on the target vehicle model is obtained.
  • determine the The two-dimensional coordinate information of the multiple key points in the picture to be detected includes: according to the coordinate conversion relationship between the three-dimensional coordinate system of the target vehicle model and the two-dimensional coordinate system of the to-be-detected picture, and a pre- The three-dimensional coordinate information of multiple key points marked on the target vehicle model is determined, and the two-dimensional coordinate information of the multiple key points in the to-be-detected picture is determined.
  • the key points are pre-marked on the target vehicle model, and the target vehicle model is a three-dimensional model, so the obtained coordinate information of the key points is three-dimensional coordinate information and is relatively accurate, and since the target vehicle model and the real target vehicle are in equal proportions Therefore, the three-dimensional coordinate information of the corresponding key points on the target vehicle is also relatively accurate.
  • the coordinate transformation relationship between the three-dimensional coordinate system of the target vehicle model and the two-dimensional coordinate system of the to-be-detected picture, and a plurality of pre-marked on the target vehicle model The three-dimensional coordinate information of the key points, and determining the two-dimensional coordinate information of the plurality of key points in the picture to be detected, including: based on the parameter information, determining the relationship between the three-dimensional coordinate system of the target vehicle model and the camera coordinate system
  • the camera coordinate system is a three-dimensional coordinate system with the camera that collects the picture to be detected as the coordinate origin; for each key point in the multiple key points, according to the three-dimensional coordinate system of the target vehicle model
  • the conversion relationship between the coordinate system and the camera coordinate system, and the three-dimensional coordinate information of the key points pre-marked on the target vehicle model determine that the corresponding key points on the target vehicle are in the Three-dimensional coordinate information in the camera coordinate system; for each key point in the plurality of key points, based on the camera internal parameter information of the camera, the key
  • determining the transformation relationship between the three-dimensional coordinate system of the target vehicle model and the camera coordinate system based on the parameter information includes: The orthogonal rotation matrix of the angle change between the three-dimensional coordinate system of the target vehicle model and the camera coordinate system, and the offset between the three-dimensional coordinate system used to characterize the target vehicle model and the camera coordinate system the translation matrix; generate a first transformation matrix according to the orthogonal rotation matrix and the offset matrix, and the first transformation matrix is used to represent the relationship between the three-dimensional coordinate system of the target vehicle model and the camera coordinate system conversion relationship.
  • the corresponding target vehicle is determined.
  • the three-dimensional coordinate information of the key points on the camera in the camera coordinate system including: based on the first transformation matrix, converting the pre-marked three-dimensional coordinate information of the key points on the target vehicle model into corresponding The three-dimensional coordinate information of the key points on the target vehicle in the camera coordinate system.
  • the three-dimensional coordinate information of the key point on the target vehicle in the camera coordinate system is converted into the image to be detected in the image.
  • Two-dimensional coordinate information including: based on the camera internal parameter information, determining a second transformation matrix for representing the transformation relationship between the camera coordinate system and the two-dimensional coordinate system corresponding to the picture to be detected;
  • the second conversion matrix is to convert the three-dimensional coordinate information of the key points on the target vehicle in the camera coordinate system into the two-dimensional coordinate information in the to-be-detected picture.
  • the plurality of key points include at least one of the following: key points visible in the picture to be detected, and key points not visible in the picture to be detected.
  • the determining the parameter information of the target vehicle based on the three-dimensional scan data includes: based on a relationship between a device that collects the three-dimensional scan data and a camera that collects the image to be detected Relative positional relationship, convert the three-dimensional scan data to the camera coordinate system with the camera as the coordinate origin; input the three-dimensional scan data in the camera coordinate system to a pre-trained neural network for processing to obtain the target
  • the parameter information of the vehicle, the neural network is obtained by training based on the sample 3D scanning data marked with the parameter information of the involved vehicle.
  • the determining the parameter information of the target vehicle based on the 3D scan data includes: acquiring label information corresponding to the 3D scan data, where the label information includes the parameter information of the target vehicle. Parameter information.
  • the parameter information of the target vehicle includes vehicle model information.
  • determining a target vehicle model corresponding to the target vehicle from a variety of pre-built three-dimensional vehicle models according to the parameter information includes: according to the vehicle model information, selecting from a variety of pre-built three-dimensional vehicle models The target vehicle model that matches the vehicle model information is filtered out.
  • the parameter information of the target vehicle includes at least any one or more of the following: size information of the target vehicle; Euler angle information of the target vehicle; center of the target vehicle The coordinates of the point in the camera coordinate system.
  • an embodiment of the present disclosure provides a vehicle control method, including: acquiring a picture to be detected collected by a vehicle; inputting the to-be-detected picture into a pre-trained key point detection model to obtain surrounding vehicles of the vehicle
  • the two-dimensional coordinate information of the key points on the vehicle, the key point detection model is obtained based on a plurality of sample images containing the target vehicle, and each sample image carries the first aspect or any possibility based on the first aspect.
  • Two-dimensional coordinate information of a plurality of key points on the target vehicle determined by the method of the embodiment of ; based on the two-dimensional coordinate information of the key points on the surrounding vehicles, identify the pose information of the surrounding vehicles; Based on the identified pose information of the surrounding vehicles, the driving state of the vehicle is controlled.
  • the key point information carried by the sample image is determined based on the method described in the first aspect or any possible implementation manner of the first aspect. Since the key point information of the sample image does not need to be manually marked, it is used for The training of the key point detection model will make the training more efficient, and the recognition accuracy of the trained key point detection model will also be higher. In addition, after identifying the images collected during the driving process of the driven vehicle based on the key point detection model, the pose information of the surrounding vehicles of the driven vehicle can be quickly determined according to the identification result, and then the driving state of the driven vehicle can be controlled in time. The safety of the vehicle being driven is improved.
  • an embodiment of the present disclosure further provides a vehicle key point information detection device, including: a first acquisition module configured to acquire three-dimensional scan data for a target vehicle and a to-be-detected picture containing the target vehicle; a first acquisition module a determination module for determining the three-dimensional coordinate information of multiple key points on the target vehicle based on the three-dimensional scan data; a second determination module for determining the three-dimensional coordinate system of the target vehicle and the image to be detected according to the three-dimensional coordinate system of the target vehicle The coordinate conversion relationship between the two-dimensional coordinate systems and the three-dimensional coordinate information of multiple key points on the target vehicle, determine the two-dimensional coordinate information of the multiple key points in the to-be-detected picture.
  • the first determining module when determining, based on the three-dimensional scan data, the three-dimensional coordinate information of multiple key points on the target vehicle, is configured to: based on the three-dimensional scan data, Determine the parameter information of the target vehicle; according to the parameter information, determine the target vehicle model corresponding to the target vehicle from a variety of pre-built three-dimensional vehicle models; obtain a plurality of key pre-marked on the target vehicle model 3D coordinate information of the point.
  • the second determination module is based on the coordinate conversion relationship between the three-dimensional coordinate system of the target vehicle and the two-dimensional coordinate system of the to-be-detected picture, and the three-dimensional coordinates of multiple key points on the target vehicle.
  • Coordinate information when determining the two-dimensional coordinate information of the plurality of key points in the to-be-detected picture, used for: according to the difference between the three-dimensional coordinate system of the target vehicle model and the two-dimensional coordinate system of the to-be-detected picture and the three-dimensional coordinate information of multiple key points pre-marked on the target vehicle model, to determine the two-dimensional coordinate information of the multiple key points in the to-be-detected picture.
  • the second determination module is based on the coordinate conversion relationship between the three-dimensional coordinate system of the target vehicle model and the two-dimensional coordinate system of the to-be-detected picture, and pre-marked on the
  • the three-dimensional coordinate information of multiple key points on the target vehicle model when determining the two-dimensional coordinate information of the multiple key points in the to-be-detected picture, is used to: determine the target vehicle model based on the parameter information
  • the conversion relationship between the three-dimensional coordinate system and the camera coordinate system, the camera coordinate system is a three-dimensional coordinate system with the camera that collects the picture to be detected as the coordinate origin; for each key point in the multiple key points , according to the conversion relationship between the three-dimensional coordinate system of the target vehicle model and the camera coordinate system, and the three-dimensional coordinate information of the key points pre-marked on the target vehicle model, determine the corresponding target vehicle
  • the second determining module when determining the transformation relationship between the three-dimensional coordinate system of the target vehicle model and the camera coordinate system based on the parameter information, is configured to:
  • the parameter information is used to determine the orthogonal rotation matrix used to characterize the angle change between the three-dimensional coordinate system of the target vehicle model and the camera coordinate system, and the three-dimensional coordinate system used to characterize the target vehicle model and all The translation matrix of the offset between the camera coordinate systems;
  • the first transformation matrix is generated according to the orthogonal rotation matrix and the offset matrix, and the first transformation matrix is used to represent the three-dimensional coordinate system of the target vehicle model and the transformation relationship between the camera coordinate system.
  • the second determining module is based on the transformation relationship between the three-dimensional coordinate system of the target vehicle model and the camera coordinate system, and the three-dimensional coordinates of the key points pre-marked on the target vehicle model.
  • it is used to: based on the first transformation matrix, convert the pre-labeled target vehicle model to the The three-dimensional coordinate information of the key point is converted into the corresponding three-dimensional coordinate information of the key point on the target vehicle in the camera coordinate system.
  • the second determining module converts, based on camera intrinsic parameter information of the camera, the three-dimensional coordinate information of the key point on the target vehicle in the camera coordinate system into a
  • the two-dimensional coordinate information in the picture to be detected is used to: determine the conversion relationship between the camera coordinate system and the two-dimensional coordinate system corresponding to the to-be-detected picture based on the camera internal parameter information; a second transformation matrix; based on the second transformation matrix, transform the three-dimensional coordinate information of the key point on the target vehicle in the camera coordinate system into two-dimensional coordinate information in the to-be-detected picture.
  • the plurality of key points include: key points visible in the picture to be detected and/or key points not visible in the picture to be detected.
  • the first determination module when determining the parameter information of the target vehicle based on the three-dimensional scan data, is configured to: based on the device for collecting the three-dimensional scan data and the acquisition of the to-be-to-be-scan data Detect the relative positional relationship between the cameras of the picture, and convert the three-dimensional scan data to a camera coordinate system with the camera as the coordinate origin; input the three-dimensional scan data in the camera coordinate system into a pre-trained neural network Processing is performed to obtain parameter information of the target vehicle.
  • the neural network is obtained by training based on the sample 3D scanning data marked with the parameter information of the involved vehicle.
  • the first determination module when determining the parameter information of the target vehicle based on the three-dimensional scan data, is configured to: acquire label information corresponding to the three-dimensional scan data, and the label The information includes parameter information of the target vehicle.
  • the parameter information of the target vehicle includes vehicle model information.
  • the first determining module is used for: selecting from the pre-built various three-dimensional vehicle models , filter out a target vehicle model that matches the vehicle model information of the target vehicle.
  • the parameter information of the target vehicle includes at least any one or more of the following: size information of the target vehicle; Euler angle information of the target vehicle; center of the target vehicle The coordinates of the point in the camera coordinate system.
  • an embodiment of the present disclosure further provides a vehicle control device, comprising: a second acquisition module for acquiring a picture to be detected collected by the vehicle; a detection module for inputting the to-be-detected picture into a pre-trained key In the point detection model, the two-dimensional coordinate information of the key points on the surrounding vehicles of the vehicle is obtained, and the key point detection model is trained based on a plurality of sample images containing the target vehicle, and each sample image carries a The two-dimensional coordinate information of multiple key points on the target vehicle determined based on the first aspect or any possible implementation manner of the first aspect; The coordinate information is used to identify the pose information of the surrounding vehicles; the control module is used to control the driving state of the vehicle based on the recognized pose information of the surrounding vehicles.
  • embodiments of the present disclosure further provide an electronic device, including a processor, a memory, and a bus, where the memory stores machine-readable instructions executable by the processor, and when the electronic device runs, the The processor communicates with the memory through a bus, and when the machine-readable instructions are executed by the processor, the steps in the first aspect or any possible implementation manner of the first aspect are performed.
  • an embodiment of the present disclosure further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and the computer program is executed by a processor to execute the first aspect or any one of the first aspects. steps in a possible implementation.
  • an embodiment of the present disclosure further provides a vehicle, including an image acquisition device, and a computing device.
  • the image collection device is used to collect pictures to be detected.
  • the computing device includes a processor, a memory, and a bus, the memory stores machine-readable instructions executable by the processor, and when the computing device is running, the processor and the memory communicate through the bus, so When the machine-readable instruction is executed by the processor, the following steps are performed: acquiring a picture to be detected collected by the image acquisition device; inputting the to-be-detected picture into a pre-trained key point detection model to obtain the vehicle's The two-dimensional coordinate information of the key points on the surrounding vehicles, the key point detection model is obtained based on a plurality of sample images including the target vehicle, and each of the sample images carries the first aspect or any one of the first aspect.
  • two-dimensional coordinate information of multiple key points on the target vehicle determined by the method in a possible implementation manner; based on the two-dimensional coordinate information of the key points on the surrounding vehicles, identify the pose of the surrounding vehicles information; controlling the driving state of the vehicle based on the identified pose information of the surrounding vehicles.
  • vehicle key point information detection device vehicle control device, electronic equipment, computer-readable storage medium, and vehicle effect description
  • vehicle control device electronic equipment, computer-readable storage medium, and vehicle effect description
  • FIG. 1 shows a flowchart of a method for detecting vehicle key point information provided by an embodiment of the present disclosure
  • FIG. 2 shows a flowchart of a method for determining two-dimensional coordinate information of multiple key points in a picture to be detected provided by an embodiment of the present disclosure
  • FIG. 3 shows a schematic flowchart of a vehicle control method provided by an embodiment of the present disclosure
  • FIG. 4 shows a schematic structural diagram of a vehicle key point information detection device provided by an embodiment of the present disclosure
  • FIG. 5 shows a schematic structural diagram of a vehicle control device provided by an embodiment of the present disclosure
  • FIG. 6 shows a schematic structural diagram of an electronic device 600 for detecting vehicle key point information provided by an embodiment of the present disclosure
  • FIG. 7 shows a schematic structural diagram of an electronic device 700 for vehicle control provided by an embodiment of the present disclosure.
  • the three-dimensional coordinate information of multiple key points on the target vehicle is determined based on the three-dimensional scan data of the target vehicle, and then based on The coordinate conversion relationship between the three-dimensional coordinate system of the target vehicle and the two-dimensional coordinate system of the picture to be detected converts the three-dimensional coordinate information of multiple key points into two-dimensional coordinate information in the picture to be detected, because the three-dimensional coordinates of the key points
  • the information is pre-marked on the three-dimensional vehicle coordinate system corresponding to the target vehicle, which is relatively accurate, so that by mapping the three-dimensional coordinate information of the key point to the two-dimensional coordinate information of the key point in the to-be-detected picture determined in the to-be-detected picture Also more precise.
  • the execution subject of the vehicle key point information detection method provided by the embodiment of the present disclosure generally has a certain computing ability.
  • the electronic equipment for example includes terminal equipment or server or other processing equipment, the terminal equipment can be user equipment (User Equipment, UE), mobile equipment, user terminal, terminal, Personal Digital Assistant (Personal Digital Assistant, PDA), Handheld devices, computing devices, etc.
  • the method for detecting vehicle key point information may be implemented by the processor calling computer-readable instructions stored in the memory.
  • FIG. 1 is a flowchart of a method for detecting vehicle key point information provided by an embodiment of the present disclosure, the method includes the following steps:
  • Step 101 Acquire three-dimensional scan data for a target vehicle and a to-be-detected picture including the target vehicle.
  • Step 102 Based on the three-dimensional scan data, determine three-dimensional coordinate information of multiple key points on the target vehicle.
  • Step 103 According to the coordinate conversion relationship between the three-dimensional coordinate system of the target vehicle and the two-dimensional coordinate system of the picture to be detected, and the three-dimensional coordinate information of multiple key points on the target vehicle, determine the multiple Two-dimensional coordinate information of a key point in the to-be-detected picture.
  • the three-dimensional coordinate information of multiple key points on the target vehicle is determined based on the three-dimensional scanning data of the target vehicle, and then based on the coordinate conversion between the three-dimensional coordinate system of the target vehicle and the two-dimensional coordinate system of the picture to be detected
  • the three-dimensional coordinate information of multiple key points is converted into the two-dimensional coordinate information in the image to be detected. Since the three-dimensional coordinate information of the key points can be pre-marked on the three-dimensional vehicle coordinate system corresponding to the target vehicle, it is more accurate. Therefore, the two-dimensional coordinate information of the key points in the to-be-detected picture determined by mapping the three-dimensional coordinate information of the key point to the to-be-detected picture is also relatively accurate.
  • the three-dimensional scan data for the target vehicle may be point cloud data for the target vehicle.
  • the 3D scan data and the pictures to be detected may be collected in real time by a collection device, or may be historically collected 3D scan data and pictures to be detected from a database. It is necessary to ensure that the vehicle targeted by the 3D scan data and the target vehicle in the image to be detected are the same vehicle with the same pose.
  • the computer when obtaining the historically collected 3D scanning data and the pictures to be detected from the database, it can be read from the local database of the computer or from the cloud server. This disclosure is not limited to this.
  • the three-dimensional scan data may be point cloud data acquired by a lidar sensor, and the lidar sensor and the camera may both be installed on the same acquisition device, which may be a driven vehicle or a data acquisition robot.
  • the point cloud data collected by the lidar sensor contains the point cloud data corresponding to the target vehicle.
  • the target vehicle is a vehicle in a picture collected by a collection device during driving, and the collection device can collect 3D scanning data and a picture to be detected once every preset period of time during driving.
  • the three-dimensional scan data and the pictures to be detected collected by the collection device during the driving process may be screened first. Screen out the to-be-detected pictures and three-dimensional scan data containing the target vehicle, and then perform steps 102 to 103 . For images to be detected and 3D scan data that do not include the target vehicle, they can be discarded directly.
  • the three-dimensional scan data is taken as point cloud data as an example, and the acquisition device of the three-dimensional scan data may be a lidar sensor, and the execution steps of step 102 and step 103 will be described in detail.
  • the parameter information of the target vehicle when determining the three-dimensional coordinate information of multiple key points on the target vehicle based on the three-dimensional scan data, the parameter information of the target vehicle may be determined based on the three-dimensional scan data, and then the parameter information may be determined based on the three-dimensional scan data. , determine the target vehicle model corresponding to the target vehicle from a variety of pre-built three-dimensional vehicle models, and then obtain the three-dimensional coordinate information of multiple key points pre-marked on the target vehicle model.
  • the parameter information of the target vehicle may include at least any one or more of the following: size information of the target vehicle (such as the length, width, and height of the target vehicle); Euler angle information of the target vehicle (such as pitch angle, rollover, etc.) angle, yaw angle, etc.); the coordinates of the center point of the target vehicle in the camera coordinate system.
  • the center point of the target vehicle may refer to the intersection of the body diagonals of the smallest rectangular parallelepiped covering the target vehicle.
  • the three-dimensional scan data is data in the coordinate system of the three-dimensional scan data acquisition device with the device for collecting the three-dimensional scan data as the coordinate origin
  • the parameter information of the target vehicle refers to the parameter information of the target vehicle in the camera coordinate system. Therefore, when determining the parameter information of the target vehicle, the three-dimensional scan data can be first converted into a camera coordinate system, and the camera coordinate system is a coordinate system with the camera that collects the picture to be detected as the coordinate origin.
  • the lidar sensor collects point cloud data
  • the recorded coordinates of the radar point are the coordinates in the radar coordinate system (that is, the 3D coordinates constructed with the lidar sensor as the coordinate origin). Therefore, when determining the parameter information of the target vehicle based on the point cloud data, the point cloud data collected by the lidar sensor can be converted from the radar coordinate system with the lidar as the coordinate origin to the camera coordinate system.
  • the point cloud data in the radar coordinate system collected by the lidar sensor can be converted into point cloud data in the camera coordinate system with the camera as the coordinate origin; Then, based on the point cloud data in the camera coordinate system, the parameter information of the target vehicle is determined.
  • the point cloud data in the camera coordinate system can be input into a pre-trained neural network to obtain the parameter information of the target vehicle.
  • the neural network may be obtained by training based on sample point cloud data marked with parameter information of the involved vehicle, and the marked vehicle parameter information carried by the sample point cloud data may be manually marked.
  • the sample point cloud data related to the vehicle can be input into the neural network to be trained, the vehicle parameter information predicted by the neural network can be output, and then based on the predicted vehicle parameter information and the labeled vehicle parameter information carried by the sample point cloud data itself , determine the model loss value in this training process, and if the determined model loss value does not meet the preset conditions, adjust the model parameters of the neural network based on the model loss value and perform training again until the model loss of a certain training The value satisfies the preset condition, and thus it is determined that the neural network training is complete.
  • the 3D scanning data can also be manually annotated, and then the annotation information corresponding to the 3D scanning data is obtained, and the annotation information includes the target vehicle parameter information.
  • this implementation manner since it is necessary to manually mark the three-dimensional scan data, this implementation manner is not suitable for the application scenario of determining the two-dimensional coordinate information of the key points in the picture to be detected in real time.
  • Multiple key points on the target vehicle can correspond to preset positions on the target vehicle model, such as rearview mirrors, wheels, lights, etc.
  • the key points on the specific vehicle model can be set according to user needs.
  • the number of key points and the positions of the key points on the three-dimensional vehicle model may be the same.
  • three-dimensional vehicle models of various types of vehicles may be constructed in advance, and the three-dimensional vehicle models may be CAD models, and then the three-dimensional coordinate information of each key point on each three-dimensional vehicle model is determined.
  • the three-dimensional coordinate information is the coordinate information in the vehicle model coordinate system
  • the vehicle model coordinate system may be a three-dimensional coordinate system constructed with any position point on the three-dimensional vehicle model as the coordinate origin.
  • the coordinate origins of the three-dimensional vehicle model coordinate systems of different models may correspond to the same position on the vehicle, for example, the rearview mirror of the vehicle is used as the coordinate origin.
  • the parameter information of the target vehicle may further include vehicle model information of the target vehicle, and the vehicle model information may include, for example, a vehicle brand name, a vehicle model name, and the like.
  • the vehicle model information in the parameter information of the target vehicle can be determined first, and then screened from the pre-built various 3D vehicle models. A target vehicle model that matches the vehicle model information of the target vehicle is obtained.
  • the vehicle model information of the target vehicle in the to-be-detected picture may also be determined only according to the to-be-detected picture.
  • semantic recognition may be performed on the picture to be detected, or vehicle model information in the picture to be detected may be identified through a pre-trained vehicle identification network.
  • the target vehicle model is screened out, since the three-dimensional coordinate information of each key point on the vehicle model has been determined when the vehicle model is constructed, the three-dimensional coordinates of multiple key points on the target vehicle model can be determined directly according to the screened target vehicle model. Coordinate information.
  • step 103 according to the coordinate conversion relationship between the three-dimensional coordinate system of the target vehicle model and the two-dimensional coordinate system of the picture to be detected, and the three-dimensional coordinate information of multiple key points on the target vehicle, determine When the multiple key points are in the two-dimensional coordinate information of the picture to be detected, it may be specifically implemented as follows: according to the coordinates between the three-dimensional coordinate system of the target vehicle model and the two-dimensional coordinate system of the to-be-detected picture The conversion relationship and the three-dimensional coordinate information of multiple key points pre-marked on the target vehicle model determine the two-dimensional coordinate information of the multiple key points in the to-be-detected picture. And, with reference to the method described in FIG. 2, the specific implementation may include the following three steps:
  • Step 201 based on the parameter information, determine the conversion relationship between the three-dimensional coordinate system of the target vehicle model and the camera coordinate system.
  • Step 202 For each key point in the plurality of key points, according to the conversion relationship between the three-dimensional coordinate system of the target vehicle model and the camera coordinate system, and the pre-marked on the target vehicle model.
  • the three-dimensional coordinate information of the key point determines the three-dimensional coordinate information of the corresponding key point on the target vehicle in the camera coordinate system.
  • Step 203 For each key point in the plurality of key points, based on the camera internal parameter information of the camera, convert the three-dimensional coordinate information of the key point on the target vehicle in the camera coordinate system into Two-dimensional coordinate information in the picture to be detected.
  • each key point on the target vehicle model corresponds to a key point on the target vehicle in the real world.
  • the keypoints on the rearview mirror of the target vehicle model correspond to the keypoints on the rearview mirror of the target vehicle in the real world.
  • each key point on the target vehicle model corresponds to a real key point on the target vehicle.
  • the key point is the three-dimensional coordinate information in the camera coordinate system, it may be: based on the first transformation matrix, convert the pre-marked three-dimensional coordinate information of each key point on the target vehicle model into the corresponding The three-dimensional coordinate information of the key points on the target vehicle in the camera coordinate system.
  • the 3D coordinate system of the target vehicle model is a 3D coordinate system constructed with any point on the target vehicle model as the coordinate origin
  • the camera coordinate system is a 3D coordinate system constructed with the camera as the coordinate origin.
  • the first conversion matrix can be determined by the following formula:
  • R is the orthogonal rotation matrix
  • T is the offset matrix
  • R can be expressed as:
  • yaw represents the yaw angle in the parameter information of the target vehicle
  • pitch represents the pitch angle in the parameter information of the target vehicle
  • roll represents the roll angle in the parameter information of the target vehicle.
  • T can be expressed as:
  • x c , y c , and z c represent the coordinates of the center point of the target vehicle in the camera coordinate system in the parameter information of the target vehicle.
  • the three-dimensional coordinate information of each of the multiple key points on the target vehicle model is converted into the three-dimensional coordinate information of the corresponding key points on the target vehicle in the camera coordinate system Coordinate information can be calculated by the following formula:
  • P ci represents the three-dimensional coordinate information of the ith key point in the camera coordinate system
  • P ki represents the three-dimensional coordinate information of the ith key point in the three-dimensional coordinate system of the target vehicle model.
  • the three-dimensional coordinate information of the key points on the target vehicle in the camera coordinate system is converted into the two-dimensional coordinate information in the to-be-detected picture.
  • dimensional coordinate information it may include: first determining a second transformation matrix based on the camera internal parameter information, and the second transformation matrix is used to represent the transformation relationship between the camera coordinate system and the two-dimensional coordinate system corresponding to the picture to be detected; and then based on the second transformation matrix, which converts the three-dimensional coordinate information of the key points on the target vehicle in the camera coordinate system into the two-dimensional coordinate information in the image to be detected.
  • the camera intrinsic parameter information may include the camera focal length and the actual coordinates of the principal point.
  • the camera internal parameter information can be obtained when the camera leaves the factory, and the camera internal parameter information will not change in subsequent applications.
  • the second transformation matrix can be determined according to the following formula:
  • f x represents the focal length of the horizontal axis
  • f y represents the focal length of the vertical axis
  • (u 0 , v 0 ) represents the actual coordinates of the principal point.
  • the conversion can be performed by the following formula:
  • (u i , v i ) represents the two-dimensional coordinate information of the converted i-th key point in the image to be detected
  • Z c represents the Z-axis coordinate of the center point of the target vehicle in the camera's internal reference information in the camera coordinate system
  • K represents the second transformation matrix
  • P ci represents the three-dimensional coordinate information of the i-th key point on the target vehicle in the camera coordinate system.
  • the determined multiple key points include the visible key points in the image to be detected. Points and/or keypoints that are not visible in the image to be detected.
  • the visible key points in the picture to be detected are the key points that can be seen in the picture to be detected (with the naked eye); the invisible key points in the picture to be detected are affected by the shooting angle, Keypoints that are not visible (with the naked eye) in the image to be detected.
  • the two-dimensional coordinate information of multiple key points on multiple target vehicles in the image to be detected may also be based on , to train a keypoint detection model.
  • the key point detection model can be used to detect the two-dimensional coordinate information of the key points on the surrounding vehicles included in the to-be-detected pictures collected during the driving process of the driven vehicle.
  • a schematic flowchart of a vehicle control method provided by the present disclosure includes the following steps:
  • Step 301 acquiring a picture to be detected collected by the vehicle.
  • Step 302 Input the to-be-detected picture into a pre-trained key point detection model to obtain two-dimensional coordinate information of key points on vehicles surrounding the vehicle.
  • the key point detection model is obtained by training based on a plurality of sample images containing the target vehicle, and each sample image carries a plurality of key points on the target vehicle determined based on the method provided by the present disclosure 2D coordinate information.
  • Step 303 Identify the pose information of the surrounding vehicles based on the two-dimensional coordinate information of the key points on the surrounding vehicles.
  • Step 304 Control the driving state of the vehicle based on the identified pose information of the surrounding vehicles.
  • each sample picture containing a vehicle may be input into the keypoint detection model to obtain the keypoints on the vehicle predicted by the keypoint detection model 2D coordinate information. Then, the two-dimensional coordinate information of the key points on the vehicle included in the sample picture determined in advance based on the vehicle key point detection method of the present disclosure can be compared with the two-dimensional coordinate information of the key points predicted by the key point detection model, Determine the loss value in this training process, and if the determined loss value does not meet the preset conditions, adjust the model parameters of the key point detection model and perform training again until the loss value determined in a certain training meets the The preset conditions, and thus determine that the keypoint detection model training is complete.
  • step 303 the pose information of the surrounding vehicles is identified based on the two-dimensional coordinate information of the key points on the surrounding vehicles, which can be implemented by any pose recognition method well known to those skilled in the art, which is not limited here.
  • the controlling of the driving state of the vehicle may include controlling the vehicle to move forward, backward, turn, increase speed, decrease speed, brake, and the like.
  • the key point information carried by the sample image is determined based on the vehicle key point information detection method provided by the embodiment of the present application. Since the key point information of the sample image does not need to be manually marked, it is used for the key point detection model. The training will make the training more efficient, and the recognition accuracy of the trained keypoint detection model will also be higher. In addition, after identifying the images collected during the driving process of the driven vehicle based on the key point detection model, the pose information of the surrounding vehicles of the driven vehicle can be quickly determined according to the identification result, and then the driving state of the driven vehicle can be controlled in time. The safety of the vehicle being driven is improved.
  • the embodiment of the present disclosure also provides a vehicle key point information detection device corresponding to the vehicle key point information detection method.
  • the information detection methods are similar, so the implementation of the apparatus may refer to the implementation of the method, and the repeated parts will not be repeated.
  • the device includes a first acquisition module 401, a first determination module 402, and a second determination module 403, specifically:
  • the first acquisition module 401 is used to acquire the 3D scan data for the target vehicle and the pictures to be detected including the target vehicle;
  • the first determination module 402 is used to determine the 3D scan data on the target vehicle based on the 3D scan data.
  • the three-dimensional coordinate information of multiple key points; the second determination module 403 is used for the coordinate conversion relationship between the three-dimensional coordinate system of the target vehicle and the two-dimensional coordinate system of the to-be-detected picture, and the The three-dimensional coordinate information of the multiple key points, and the two-dimensional coordinate information of the multiple key points in the to-be-detected picture is determined.
  • the first determining module 402 when determining the three-dimensional coordinate information of multiple key points on the target vehicle based on the three-dimensional scan data, is configured to: based on the three-dimensional scan data , determine the parameter information of the target vehicle; according to the parameter information, determine the target vehicle model corresponding to the target vehicle from a variety of pre-built three-dimensional vehicle models; obtain multiple pre-marked on the target vehicle model 3D coordinate information of key points.
  • the second determination module 403 is based on the coordinate conversion relationship between the three-dimensional coordinate system of the target vehicle and the two-dimensional coordinate system of the image to be detected, and the transformation of multiple key points on the target vehicle.
  • Three-dimensional coordinate information when determining the two-dimensional coordinate information of the plurality of key points in the to-be-detected picture, for: according to the relationship between the three-dimensional coordinate system of the target vehicle model and the two-dimensional coordinate system of the to-be-detected picture and the three-dimensional coordinate information of multiple key points pre-marked on the target vehicle model, to determine the two-dimensional coordinate information of the multiple key points in the to-be-detected picture.
  • the second determining module 403 determines the coordinate conversion relationship between the three-dimensional coordinate system of the target vehicle model and the two-dimensional coordinate system of the to-be-detected picture, and pre-marks the The three-dimensional coordinate information of multiple key points on the target vehicle model is determined, and when the two-dimensional coordinate information of the multiple key points in the to-be-detected picture is determined, it is used to: determine the target vehicle based on the parameter information.
  • the transformation relationship between the three-dimensional coordinate system of the model and the camera coordinate system, the camera coordinate system is a three-dimensional coordinate system with the camera that collects the picture to be detected as the coordinate origin; for each key point in the multiple key points The corresponding target is determined according to the conversion relationship between the three-dimensional coordinate system of the target vehicle model and the camera coordinate system, and the three-dimensional coordinate information of the key points pre-marked on the target vehicle model
  • the three-dimensional coordinate information of the key point on the vehicle in the camera coordinate system for each key point in the plurality of key points, based on the camera intrinsic parameter information of the camera, the The three-dimensional coordinate information of the key point in the camera coordinate system is converted into two-dimensional coordinate information in the picture to be detected.
  • the second determining module 403 determines the transformation relationship between the three-dimensional coordinate system of the target vehicle model and the camera coordinate system based on the parameter information
  • the second determining module 403 is configured to:
  • the parameter information determines an orthogonal rotation matrix used to characterize the angular change between the three-dimensional coordinate system of the target vehicle model and the camera coordinate system, and the three-dimensional coordinate system used to characterize the target vehicle model and the camera coordinates
  • the translation matrix of the offset between the systems; the first transformation matrix is generated according to the orthogonal rotation matrix and the offset matrix, and the first transformation matrix is used to characterize the three-dimensional coordinate system of the target vehicle model and the The transformation relationship between camera coordinate systems.
  • the second determining module 403 is based on the transformation relationship between the three-dimensional coordinate system of the target vehicle model and the camera coordinate system, and the three-dimensional Coordinate information, when determining the three-dimensional coordinate information of the corresponding key points on the target vehicle in the camera coordinate system, it is used to: based on the first transformation matrix, convert all pre-marked target vehicle models The three-dimensional coordinate information of the key point is converted into the corresponding three-dimensional coordinate information of the key point on the target vehicle in the camera coordinate system.
  • the second determining module 403 converts the three-dimensional coordinate information of the key point on the target vehicle in the camera coordinate system based on the camera intrinsic parameter information of the camera into In the case of the two-dimensional coordinate information in the picture to be detected, for: determining the conversion relationship between the camera coordinate system and the two-dimensional coordinate system corresponding to the to-be-detected picture based on the camera internal parameter information based on the second transformation matrix, convert the three-dimensional coordinate information of the key point on the target vehicle in the camera coordinate system into the two-dimensional coordinate information in the to-be-detected picture .
  • the plurality of key points include visible key points in the picture to be detected and/or invisible key points in the picture to be detected.
  • the first determining module 402 when determining the parameter information of the target vehicle based on the three-dimensional scan data, is configured to: based on the device for acquiring the three-dimensional scan data and acquiring the The relative positional relationship between the cameras of the picture to be detected, the three-dimensional scan data is converted into a camera coordinate system with the camera as the coordinate origin; the three-dimensional scan data in the camera coordinate system is input into the pre-trained neural The network performs processing to obtain the parameter information of the target vehicle.
  • the neural network is obtained by training based on the sample 3D scanning data marked with the parameter information of the involved vehicle.
  • the first determining module 402 when determining the parameter information of the target vehicle based on the three-dimensional scan data, is configured to: acquire label information corresponding to the three-dimensional scan data, and the The annotation information includes parameter information of the target vehicle.
  • the parameter information of the target vehicle includes vehicle model information.
  • the first determining module 402 determines a target vehicle model corresponding to the target vehicle from a variety of pre-built three-dimensional vehicle models according to the parameter information, the first determining module 402 is configured to: In the model, a target vehicle model matching the vehicle model information of the target vehicle is screened out.
  • the parameter information of the target vehicle includes at least any one or more of the following: size information of the target vehicle; Euler angle information of the target vehicle; center of the target vehicle The coordinates of the point in the camera coordinate system.
  • the device includes a second acquisition module 501 , a detection module 502 , an identification module 503 , and a control module 504 , specifically: a second acquisition module 501 , a detection module 502 , an identification module 503 , and a control module 504
  • the acquisition module 501 is used to acquire the pictures to be detected collected by the vehicle;
  • the detection module 502 is used to input the to-be-detected pictures into the pre-trained key point detection model, and obtain two data of the key points on the surrounding vehicles of the vehicle.
  • the key point detection model is obtained by training based on a plurality of sample images containing the target vehicle, and each sample image carries a plurality of The two-dimensional coordinate information of the key points;
  • the identification module 503 is used to identify the pose information of the surrounding vehicles based on the two-dimensional coordinate information of the key points on the surrounding vehicles;
  • the control module 504 is used to identify the The pose information of surrounding vehicles controls the driving state of the vehicles.
  • a schematic structural diagram of an electronic device 600 provided in an embodiment of the present application includes a processor 601 , a memory 602 , and a bus 603 .
  • the memory 602 is used for storing execution instructions, including a memory 6021 and an external memory 6022.
  • the memory 6021 here is also called an internal memory, and is used to temporarily store operation data in the processor 601 and data exchanged with an external memory 6022 such as a hard disk.
  • the processor 601 exchanges data with the external memory 6022 through the memory 6021 .
  • the processor 601 communicates with the memory 602 through the bus 603, so that the processor 601 executes the following instructions: acquiring the three-dimensional scan data for the target vehicle and the to-be-detected picture containing the target vehicle; based on the the three-dimensional scan data, determine three-dimensional coordinate information of a plurality of key points on the target vehicle; according to the coordinate conversion relationship between the three-dimensional coordinate system of the target vehicle and the two-dimensional coordinate system of the to-be-detected picture, and The three-dimensional coordinate information of the multiple key points on the target vehicle determines the two-dimensional coordinate information of the multiple key points in the to-be-detected picture.
  • a schematic structural diagram of an electronic device 700 provided in an embodiment of the present application includes a processor 701 , a memory 702 , and a bus 703 .
  • the memory 702 is used for storing execution instructions, including a memory 7021 and an external memory 7022.
  • the memory 7021 here is also called an internal memory, and is used to temporarily store operation data in the processor 701 and data exchanged with an external memory 7022 such as a hard disk.
  • the processor 701 exchanges data with the external memory 7022 through the memory 7021 .
  • the processor 701 communicates with the memory 702 through the bus 703, so that the processor 701 executes the following instructions: acquire the image to be detected collected by the vehicle; input the image to be detected into the pre-trained key points In the detection model, the two-dimensional coordinate information of the key points on the surrounding vehicles of the vehicle is obtained, and the key point detection model is obtained based on a plurality of sample images containing the target vehicle, and each sample image carries a Based on the two-dimensional coordinate information of multiple key points on the target vehicle determined by the method described in the above embodiment; based on the two-dimensional coordinate information of the key points on the surrounding vehicles, identify the pose information of the surrounding vehicles ; Control the driving state of the vehicle based on the identified pose information of the surrounding vehicles.
  • Embodiments of the present disclosure also provide a vehicle, including an image acquisition device, and a computing device.
  • the image acquisition device is used for acquiring pictures to be detected;
  • the computing device includes a processor, a memory and a bus, and the memory stores machine-readable instructions executable by the processor.
  • the processor and the memory communicate through a bus, and when the machine-readable instruction is executed by the processor, the following steps are performed: acquiring the image to be detected collected by the image capturing device;
  • the to-be-detected picture is input into a pre-trained key point detection model to obtain two-dimensional coordinate information of key points on the surrounding vehicles of the vehicle.
  • the key point detection model is based on multiple sample images containing the target vehicle.
  • each of the sample images carries the two-dimensional coordinate information of multiple key points on the target vehicle determined based on the method provided in the above embodiment; based on the two-dimensional coordinate information of the key points on the surrounding vehicles
  • the coordinate information is used to identify the pose information of the surrounding vehicles; based on the identified pose information of the surrounding vehicles, the driving state of the vehicle is controlled.
  • the vehicle provided by the embodiment of the present disclosure may be an automatic driving vehicle, and may also be an artificial driving vehicle with some intelligent control functions.
  • Embodiments of the present disclosure further provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the method for detecting vehicle key point information described in the foregoing method embodiments, Steps of a vehicle control method.
  • the storage medium may be a volatile or non-volatile computer-readable storage medium.
  • the computer program products of the vehicle key point information detection method and the vehicle control method provided by the embodiments of the present disclosure include a computer-readable storage medium storing program codes, and the instructions included in the program codes can be used to execute the above method embodiments.
  • the steps of the above-mentioned method for detecting vehicle key point information reference may be made to the above method embodiments, which will not be repeated here.
  • the computer program product can be specifically implemented by hardware, software or a combination thereof.
  • the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK), etc. Wait.
  • the units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.
  • each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.
  • the functions, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a processor-executable non-volatile computer-readable storage medium.
  • the computer software products are stored in a storage medium, including Several instructions are used to cause an electronic device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present disclosure.
  • the aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes .

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Traffic Control Systems (AREA)

Abstract

L'invention concerne un procédé de détection d'informations de point clé de véhicule, un procédé de commande de véhicule et un dispositif correspondant, ainsi qu'un véhicule. Selon un exemple du procédé de détection d'informations de point clé de véhicule, à la suite de l'acquisition de données de balayage tridimensionnel pour un véhicule cible et une image, contenant le véhicule cible, à détecter, des informations de coordonnées tridimensionnelles d'une pluralité de points clés sur le véhicule cible sont déterminées sur la base des données de balayage tridimensionnel, et en fonction d'une relation de conversion de coordonnées entre un système de coordonnées tridimensionnelles du véhicule cible et un système de coordonnées bidimensionnelles de ladite image, et les informations de coordonnées tridimensionnelles de la pluralité de points clés sur le véhicule cible, des informations de coordonnées bidimensionnelles de la pluralité de points clés dans ladite image sont déterminées.
PCT/CN2021/102179 2020-12-10 2021-06-24 Détection d'informations de point clé de véhicule et commande de véhicule WO2022121283A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020227008917A KR20220084021A (ko) 2020-12-10 2021-06-24 차량 키포인트 정보 검출 및 차량 제어

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011454859.1A CN112489126B (zh) 2020-12-10 2020-12-10 车辆关键点信息检测方法、车辆控制方法及装置、车辆
CN202011454859.1 2020-12-10

Publications (1)

Publication Number Publication Date
WO2022121283A1 true WO2022121283A1 (fr) 2022-06-16

Family

ID=74916656

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/102179 WO2022121283A1 (fr) 2020-12-10 2021-06-24 Détection d'informations de point clé de véhicule et commande de véhicule

Country Status (3)

Country Link
KR (1) KR20220084021A (fr)
CN (1) CN112489126B (fr)
WO (1) WO2022121283A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116883496A (zh) * 2023-06-26 2023-10-13 小米汽车科技有限公司 交通元素的坐标重建方法、装置、电子设备及存储介质
CN117423109A (zh) * 2023-10-31 2024-01-19 北京代码空间科技有限公司 一种图像关键点标注方法及其相关设备

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112489126B (zh) * 2020-12-10 2023-09-19 浙江商汤科技开发有限公司 车辆关键点信息检测方法、车辆控制方法及装置、车辆
CN113327192B (zh) * 2021-05-11 2022-07-08 武汉唯理科技有限公司 一种通过三维测量技术测算汽车行驶速度的方法
CN113793413A (zh) * 2021-08-13 2021-12-14 北京迈格威科技有限公司 三维重建方法、装置、电子设备及存储介质
CN114821497A (zh) * 2022-02-24 2022-07-29 广州文远知行科技有限公司 目标物位置的确定方法、装置、设备及存储介质
CN114705121B (zh) * 2022-03-29 2024-05-14 智道网联科技(北京)有限公司 车辆位姿测量方法、装置及电子设备、存储介质
CN115661577B (zh) * 2022-11-01 2024-04-16 吉咖智能机器人有限公司 用于对象检测的方法、设备和计算机可读存储介质
CN117078752B (zh) * 2023-07-19 2024-05-28 苏州魔视智能科技有限公司 车辆位姿估计方法、装置、车辆及存储介质
CN117274956B (zh) * 2023-11-17 2024-05-24 深圳市航盛电子股份有限公司 车辆侧视图生成方法、装置、终端设备以及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170193311A1 (en) * 2015-12-30 2017-07-06 Texas Instruments Incorporated Vehicle control with efficient iterative traingulation
CN109214980A (zh) * 2017-07-04 2019-01-15 百度在线网络技术（北京）有限公司 一种三维姿态估计方法、装置、设备和计算机存储介质
CN109903337A (zh) * 2019-02-28 2019-06-18 北京百度网讯科技有限公司 用于确定挖掘机的铲斗的位姿的方法和装置
CN111462249A (zh) * 2020-04-02 2020-07-28 北京迈格威科技有限公司 一种交通摄像头的标定数据获取方法、标定方法及装置
CN112489126A (zh) * 2020-12-10 2021-03-12 浙江商汤科技开发有限公司 车辆关键点信息检测方法、车辆控制方法及装置、车辆

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111951326A (zh) * 2019-05-15 2020-11-17 北京地平线机器人技术研发有限公司 基于多摄像装置的目标对象骨骼关键点定位方法和装置
US10600210B1 (en) * 2019-07-25 2020-03-24 Second Spectrum, Inc. Data processing systems for real-time camera parameter estimation
CN110370286B (zh) * 2019-08-13 2022-04-12 西北工业大学 基于工业机器人和单目相机的定轴运动刚体空间位置识别方法
CN110427917B (zh) * 2019-08-14 2022-03-22 北京百度网讯科技有限公司 用于检测关键点的方法和装置
CN111340877B (zh) * 2020-03-25 2023-10-27 北京爱笔科技有限公司 一种车辆定位方法及装置
CN111652928B (zh) * 2020-05-11 2023-12-15 上海交通大学 三维点云中物体抓取位姿检测方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170193311A1 (en) * 2015-12-30 2017-07-06 Texas Instruments Incorporated Vehicle control with efficient iterative traingulation
CN109214980A (zh) * 2017-07-04 2019-01-15 百度在线网络技术（北京）有限公司 一种三维姿态估计方法、装置、设备和计算机存储介质
CN109903337A (zh) * 2019-02-28 2019-06-18 北京百度网讯科技有限公司 用于确定挖掘机的铲斗的位姿的方法和装置
CN111462249A (zh) * 2020-04-02 2020-07-28 北京迈格威科技有限公司 一种交通摄像头的标定数据获取方法、标定方法及装置
CN112489126A (zh) * 2020-12-10 2021-03-12 浙江商汤科技开发有限公司 车辆关键点信息检测方法、车辆控制方法及装置、车辆

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116883496A (zh) * 2023-06-26 2023-10-13 小米汽车科技有限公司 交通元素的坐标重建方法、装置、电子设备及存储介质
CN116883496B (zh) * 2023-06-26 2024-03-12 小米汽车科技有限公司 交通元素的坐标重建方法、装置、电子设备及存储介质
CN117423109A (zh) * 2023-10-31 2024-01-19 北京代码空间科技有限公司 一种图像关键点标注方法及其相关设备

Also Published As

Publication number Publication date
CN112489126B (zh) 2023-09-19
CN112489126A (zh) 2021-03-12
KR20220084021A (ko) 2022-06-21

Similar Documents

Publication Publication Date Title
WO2022121283A1 (fr) Détection d'informations de point clé de véhicule et commande de véhicule
CN111340797B (zh) 一种激光雷达与双目相机数据融合检测方法及系统
JP6452186B2 (ja) 複数の写真の一致性に基づく保険賠償詐欺防止方法、システム、装置及び読み取り可能な記録媒体
JP6011102B2 (ja) 物体姿勢推定方法
CN103870798B (zh) 对象检测方法、对象检测设备以及图像拾取设备
CN110490936A (zh) 车辆摄像头的标定方法、装置、设备及可读存储介质
JP2011096135A (ja) 画像処理装置、画像処理方法
WO2022183685A1 (fr) Procédé de détection de cible, support électronique et support de stockage informatique
CN111598065A (zh) 深度图像获取方法及活体识别方法、设备、电路和介质
CN110007764B (zh) 一种手势骨架识别方法、装置、系统及存储介质
WO2023284358A1 (fr) Procédé et appareil d'étalonnage de caméra, dispositif électronique et support de stockage
CN113281780B (zh) 对图像数据进行标注的方法、装置及电子设备
JP2018195070A (ja) 情報処理装置、情報処理方法、及びプログラム
CN110197104B (zh) 基于车辆的测距方法及装置
CN111898552B (zh) 一种判别人员关注目标物的方法、装置及计算机设备
CN112699748B (zh) 基于yolo及rgb图像的人车距离估计方法
CN116012712A (zh) 基于物体通用特征的目标检测方法、装置、设备及介质
CN113658274B (zh) 用于灵长类动物种群行为分析的个体间距自动计算方法
CN112364693B (zh) 基于双目视觉的障碍识别方法、装置、设备及存储介质
WO2023179390A1 (fr) Procédé et appareil de reconnaissance d'objet, dispositif électronique, support d'enregistrement lisible par ordinateur et produit programme d'ordinateur
JP7493545B2 (ja) 位置推定装置、位置推定方法、及びプログラム
JP2019045993A (ja) 画像処理装置、画像処理方法及びプログラム
JP6855207B2 (ja) 画像処理装置、画像処理方法及びプログラム
Shanesazan et al. Human Detection and Its Distance Measurement in Agricultural Fields by Stereo Image Processing
JP2023146044A (ja) 位置推定装置、位置推定方法、及びプログラム

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2022517776

Country of ref document: JP

Kind code of ref document: A

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21901995

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21901995

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 21901995

Country of ref document: EP

Kind code of ref document: A1