CN114972694A - Object labeling method and device, equipment and storage medium - Google Patents

Object labeling method and device, equipment and storage medium Download PDF

Info

Publication number
CN114972694A
CN114972694A CN202210528073.2A CN202210528073A CN114972694A CN 114972694 A CN114972694 A CN 114972694A CN 202210528073 A CN202210528073 A CN 202210528073A CN 114972694 A CN114972694 A CN 114972694A
Authority
CN
China
Prior art keywords
coordinate system
coordinate
image acquisition
acquisition device
plane image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210528073.2A
Other languages
Chinese (zh)
Inventor
纪大胜
刘世林
杜霖
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Depth Search Technology Co ltd
Original Assignee
Beijing Depth Search Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Depth Search Technology Co ltd filed Critical Beijing Depth Search Technology Co ltd
Priority to CN202210528073.2A priority Critical patent/CN114972694A/en
Publication of CN114972694A publication Critical patent/CN114972694A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30204—Marker
    • G06T2207/30208—Marker matrix
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle
    • G06T2207/30256—Lane; Road marking
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2219/00—Indexing scheme for manipulating 3D models or images for computer graphics
    • G06T2219/012—Dimensioning, tolerancing

Abstract

The disclosure relates to an object labeling method, comprising: and acquiring a planar image to be subjected to object labeling and space data corresponding to the planar image. Marking the current object to be marked in the plane image, and acquiring the pixel coordinates of each pixel point of the object in the plane image. And converting the pixel coordinates of each pixel point of the object in the plane image based on the acquisition parameters of the plane image acquisition equipment to obtain the coordinate data of each pixel point of the object in the coordinate system of the plane image acquisition equipment. And converting the coordinate data of each pixel point of the object under the coordinate system of the plane acquisition equipment according to a coordinate conversion matrix from the coordinate system of the plane image acquisition equipment to the coordinate system of the space data acquisition equipment to obtain the coordinate data of each pixel point of the object under the coordinate system of the space data acquisition equipment. And correspondingly labeling the object to the spatial data.

Description

Object labeling method and device, equipment and storage medium
Technical Field
The present disclosure relates to the field of machine learning technologies, and in particular, to an object labeling method and apparatus, a device, and a storage medium.
Background
In the field of autopilot technology, cameras and lidar are widely used sensors. The laser radar outputs 3D point cloud data, which is processed by a machine learning algorithm to identify various objects therein, such as vehicles, people, lane lines, and the like.
Training of a machine learning algorithm requires a large amount of labeling data, and for labeling of lane lines, the current labeling methods label point clouds in a 3D space, such as: each point on the lane line is labeled (point cloud segmentation), or the line is directly drawn in the 3D space, so that the lane line information in the 3D space is obtained.
However, the above marking methods for lane lines all need to mark in a 3D space, are high in complexity and low in efficiency, and how to directly mark a 3D object on a 2D picture to avoid complex operations in the 3D space and improve marking efficiency, which is a problem to be solved urgently by those skilled in the art.
Disclosure of Invention
In view of this, the present disclosure provides an object labeling method to realize labeling of a 3D object through a 2D picture.
According to an aspect of the present disclosure, there is provided an object labeling method, including:
acquiring a planar image to be subjected to object labeling at present and spatial data corresponding to the planar image;
marking an object to be marked currently in the plane image, and acquiring pixel coordinates of each pixel point of the object in the plane image;
converting the pixel coordinates of each pixel point of the object in the plane image based on the acquisition parameters of the plane image acquisition equipment to obtain the coordinate data of each pixel point of the object in the coordinate system of the plane image acquisition equipment;
converting the coordinate data of each pixel point of the object under the coordinate system of the plane acquisition equipment according to a coordinate conversion matrix from the coordinate system of the plane image acquisition equipment to the coordinate system of the space data acquisition equipment to obtain the coordinate data of each pixel point of the object under the coordinate system of the space data acquisition equipment;
and correspondingly marking the object into the spatial data according to the coordinate data of each pixel point of the object in the coordinate system of the spatial data acquisition equipment.
In one possible implementation, the acquisition parameters of the planar image acquisition device include at least one of a focal length of the planar image acquisition device, the acquired planar image resolution, and a sensor size of the planar image acquisition device.
In a possible implementation manner, according to the focal length of the planar image acquisition device, the acquired resolution of the planar image and the size of a sensor of the planar image acquisition device, respectively obtaining the focal length of the planar image acquisition device in units of horizontal pixels and the focal length of the planar image acquisition device in units of vertical pixels;
and respectively obtaining the horizontal offset and the vertical offset of the principal point of the plane image acquisition equipment on the plane image according to the resolution of the plane image acquired by the plane image acquisition equipment.
In a possible implementation manner, when converting pixel coordinates of each pixel point of the object in the planar image based on an acquisition parameter of a planar image acquisition device to obtain coordinate data of each pixel point of the object in a coordinate system of the planar image acquisition device, the method includes:
and substituting the pixel coordinates of the two-dimensional object pixel points in the plane image into the mapping relation between the plane image coordinate system and the image acquisition equipment coordinate system to obtain the coordinate data of the three-dimensional object pixel points in the plane image acquisition equipment coordinate system.
In one possible implementation, the mapping relationship between the plane image coordinate system and the image capturing device coordinate system is:
Figure BDA0003645391180000021
wherein (u, v) is the pixel coordinate of the pixel point of the object in the planar image, (x ', y' 0 Z ') is the coordinate, y ' of the converted pixel point of the object in the plane acquisition device coordinate system ' 0 For the measured distance between the planar image acquisition device and the plane of the object, f u And f v Focal distance in horizontal pixels and focal distance in vertical pixels of the planar image acquisition device, c u And c v Respectively a horizontal offset and a vertical offset of a principal point of the planar image acquisition device on the planar image.
In one possible implementation manner, the coordinate transformation matrix is obtained based on a relationship between a coordinate axis orientation of the planar image acquisition device coordinate system and a coordinate axis orientation of the spatial data acquisition device coordinate system.
In a possible implementation manner, a rotation matrix is obtained according to the X-axis unit vector, the Y-axis unit vector and the Z-axis unit vector of the coordinate system of the planar image acquisition device and the X-axis unit vector, the Y-axis unit vector and the Z-axis unit vector of the coordinate system of the spatial data acquisition device;
obtaining a translation matrix according to coordinate data of the origin of the spatial data acquisition equipment coordinate system under the plane image acquisition equipment coordinate system;
and merging the translation matrix to the right side of the rotation matrix to obtain a merged matrix, and adding a matrix [0, 0, 0, 1] below the merged matrix to obtain the coordinate conversion matrix.
According to another aspect of the present disclosure, there is provided an object labeling apparatus including: the system comprises an acquisition module, a marking module, a first conversion module and a second conversion module;
the acquisition module is configured to acquire a planar image to be subjected to object labeling currently and spatial data corresponding to the planar image;
the marking module is configured to mark the object in the planar graph and acquire pixel coordinates of each pixel point of the object in the planar graph;
the first conversion module is configured to convert pixel coordinates of each pixel point of the object in the planar image based on an acquisition parameter of a planar image acquisition device to obtain coordinate data of each pixel point of the object in a coordinate system of the planar image acquisition device;
the second conversion module is configured to convert, according to a coordinate conversion matrix from a plane image acquisition device coordinate system to a spatial data acquisition device coordinate system, coordinate data of each pixel point of the object in the plane image acquisition device coordinate system to obtain coordinate data of each pixel point of the object in the spatial data acquisition device coordinate system, and label the object into the spatial data correspondingly according to the coordinate data of each pixel point of the object in the spatial data acquisition device coordinate system.
According to another aspect of the present disclosure, there is provided an object labeling apparatus including:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to carry out any of the above methods when executing the executable instructions.
According to another aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, implement the method of any one of the above.
The object labeling method of the embodiment of the disclosure acquires a planar image and corresponding spatial data through a planar image acquisition device and a spatial data acquisition device, labels an object to be labeled currently in the acquired planar image, acquires pixel coordinates of the labeled object in the planar image, and converts the pixel coordinates of each pixel point of the object in the planar image based on acquisition parameters of the planar image acquisition device to obtain coordinate data of each pixel point of the object in a coordinate system of the planar image acquisition device. And converting coordinate data of the marked object in a plane image acquisition equipment coordinate system into coordinate data of space data acquisition equipment coordinates by using a coordinate conversion matrix, and realizing marking of the marked object in the space data by using the coordinate data of the space data acquisition equipment coordinates. The coordinate transformation matrix is constructed based on the relative positions of the plane image acquisition equipment coordinate system and the spatial data acquisition equipment coordinate system.
Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments, which proceeds with reference to the accompanying drawings.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.
FIG. 1 shows a flow chart of an object labeling method of an embodiment of the present disclosure;
fig. 2 is a schematic diagram illustrating a coordinate system of a spatial data acquisition device and a coordinate system of a planar image acquisition device in an object labeling method according to an embodiment of the present disclosure;
fig. 3 is a schematic diagram illustrating a plane image coordinate system and a plane image acquisition device coordinate system in the object labeling method according to the embodiment of the present disclosure;
fig. 4 shows a main body structure diagram of a subject apparatus of the embodiment of the present disclosure;
fig. 5 shows a main body structure diagram of the object apparatus of the embodiment of the present disclosure.
Detailed Description
Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numbers in the drawings identify functionally the same or similar elements. While the various aspects of the embodiments are presented in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.
The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.
Furthermore, in the following detailed description, numerous specific details are set forth in order to provide a better understanding of the present disclosure. It will be understood by those skilled in the art that the present disclosure may be practiced without some of these specific details. In some instances, methods, means, elements and circuits that are well known to those skilled in the art have not been described in detail so as not to obscure the subject matter of the present disclosure. Fig. 1 shows a flowchart of an object labeling method according to an embodiment of the present disclosure. Fig. 2 shows a schematic diagram of a spatial data acquisition device coordinate system and a planar image acquisition device coordinate system in an object labeling method according to an embodiment of the present disclosure. Fig. 3 is a schematic diagram illustrating a plane image coordinate system and a plane image capturing device coordinate system in an object labeling method according to an embodiment of the disclosure. Fig. 4 illustrates a main body structure diagram of a target device according to an embodiment of the present disclosure. Fig. 5 illustrates a body structure diagram of a subject apparatus according to an embodiment of the present disclosure. As shown in fig. 1, the object labeling method includes: s100: and acquiring a planar image to be subjected to object labeling and space data corresponding to the planar image. S200: and marking the current object to be marked in the plane image, and acquiring the pixel coordinates of each pixel point of the object in the plane image. S300: and converting the pixel coordinates of each pixel point of the object in the plane image based on the acquisition parameters of the plane image acquisition equipment to obtain the coordinate data of each pixel point of the object in the coordinate system of the plane image acquisition equipment. S400: and converting the coordinate data of each pixel point of the object under the coordinate system of the plane acquisition equipment according to a coordinate conversion matrix from the coordinate system of the plane image acquisition equipment to the coordinate system of the space data acquisition equipment to obtain the coordinate data of each pixel point of the object under the coordinate system of the space data acquisition equipment. S500: and correspondingly marking the object into the spatial data according to the coordinate data of each pixel point of the object in the coordinate system of the spatial data acquisition equipment.
The object labeling method of the embodiment of the disclosure acquires a planar image and corresponding spatial data through a planar image acquisition device and a spatial data acquisition device, labels an object to be labeled currently in the acquired planar image, acquires pixel coordinates of the labeled object in the planar image, and converts the pixel coordinates of each pixel point of the object in the planar image based on acquisition parameters of the planar image acquisition device to obtain coordinate data of each pixel point of the object in a coordinate system of the planar image acquisition device. And converting coordinate data of the marked object in a plane image acquisition equipment coordinate system into coordinate data of space data acquisition equipment coordinates by using a coordinate conversion matrix, and realizing marking of the marked object in the space data by using the coordinate data of the space data acquisition equipment coordinates. The coordinate transformation matrix is constructed based on the relative positions of the plane image acquisition equipment coordinate system and the spatial data acquisition equipment coordinate system.
Here, it should be noted that the data acquired by the planar image acquisition device is a two-dimensional pixel image, and the data acquired by the spatial data acquisition device is three-dimensional point cloud data. Correspondingly, the plane image acquisition equipment can be a camera and also can be other devices capable of realizing image acquisition. The spatial data acquisition equipment can be directly realized by adopting a laser radar. In the method of the embodiment of the present disclosure, the types or models of the planar image capturing device and the spatial data capturing device are not particularly limited.
In the method of the disclosed embodiment, after the planar image to be currently subjected to object labeling and the spatial data corresponding to the planar image are acquired, the processing operation of coordinate conversion is performed on each pixel point of the labeled object in the acquired planar image, so that a corresponding coordinate system needs to be constructed to facilitate the pixel point coordinate conversion.
Specifically, the constructed coordinate system includes: the method comprises the steps of constructing a three-dimensional plane drawing plane acquisition equipment coordinate system based on plane image acquisition equipment, constructing a three-dimensional space data acquisition equipment coordinate system based on space data acquisition equipment, and constructing a two-dimensional plane drawing coordinate system based on plane images acquired by the plane image acquisition equipment.
The plane image coordinate system is mapping of two of three coordinate axes of the plane image acquisition equipment coordinate system, the other coordinate axis of the three coordinate axes of the plane image acquisition equipment coordinate system is a lens optical axis of the plane image acquisition equipment, and an origin of the plane image acquisition equipment coordinate system is a lens optical center of the plane image acquisition equipment.
For example, when the planar image acquisition device is a camera and the spatial data acquisition device is a laser radar, the constructed coordinate system of the planar image acquisition device refers to a coordinate system constructed by taking the optical center of the lens of the camera as the origin. The constructed coordinate system of the image acquisition equipment refers to a coordinate system which is constructed by taking a radio frequency signal sending point of the laser radar as an origin. Correspondingly, the plane image coordinate system refers to a coordinate system constructed by respectively taking the long side and the wide side of the plane image as two coordinate axes.
Further, as shown in fig. 2, a lens optical center of the planar image capturing device is used as an origin of a coordinate system of the planar image capturing device, a lens optical axis of the planar image capturing device is used as a z-axis of the coordinate system of the planar image capturing device, and a long side and a wide side of a rectangular sensor of the planar image capturing device are respectively used as an x-axis and a y-axis of the coordinate system of the planar image capturing device, so as to construct the coordinate system of the planar image capturing device. As shown in fig. 3, the plane image acquired by the plane image acquisition device is a mapping of data acquired by the rectangular sensor of the plane image acquisition device, so that a plane image coordinate system is constructed by respectively using the long side and the wide side of the plane image as a u-axis and a v-axis and using the top left vertex of the plane image as an origin, wherein the positive direction of the u-axis is the same as the positive direction of the x-axis of the plane image acquisition device coordinate system, and the positive direction of the v-axis is the same as the positive direction of the y-axis of the plane image acquisition device coordinate system.
Referring to fig. 3, the origin of the coordinate system of the plane image pickup device is located at the center of the plane figure. Marking an object to be marked in a plane image, wherein points marked on the plane image take pixels as units, all marked points acquire pixel coordinates in a plane image coordinate system, and converting the pixel coordinates in the plane image coordinate system into coordinate data in the plane image acquisition equipment coordinate system by utilizing a parameter matrix constructed by acquisition parameters of the plane image acquisition equipment.
The acquisition parameters of the plane image acquisition equipment comprise at least one of the focal length of the plane image acquisition equipment, the acquired resolution of the plane image and the size of a sensor of the plane image acquisition equipment.
Further, according to the focal length of the plane image acquisition equipment, the resolution of the acquired plane image and the size of the sensor of the plane image acquisition equipment, respectively obtaining the focal length f of the plane image acquisition equipment by taking a horizontal pixel as a unit u And a focal length f in vertical pixels v I.e. the scale factors of the plane image acquisition equipment in the directions of the u-axis and the v-axis of the plane image coordinate system. Wherein f is u ＝ f/dx,f v F/dy. f is the focal length of the camera, and dx and dy correspond to the Pixel Size (Pixel Size), which is generally directly noted in the camera specification. Respectively obtaining the horizontal offset c of the principal point of the plane image acquisition equipment on the plane image according to the plane image resolution acquired by the plane image acquisition equipment u And a vertical offset c v Namely the distance between the origin of the coordinate system of the plane image acquisition equipment and the v axis of the plane image coordinate system and the distance between the origin of the coordinate system of the plane image acquisition equipment and the u axis of the plane image coordinate system, and the optical axis of the lens of the plane image acquisition equipment is positioned at the center of the plane image, so the value of the optical axis is half of the resolution ratio.
For example, the focal length f of the planar image capture device is 35mm, the highest resolution: 4256 × 2832, sensor size: 36.0X 23.9mm, according to the method described above, to give c u ＝4256/2＝2128， c v ＝2832/2＝1416，dx＝36.0/4256，dy＝23.9/2832，f u ＝f/dx＝4137.8，f v ＝f/dy＝4147.3。
Here, c is to be explained u 、c v 、f u And f v With the parameters c introduced in the pinhole camera model u 、c v 、f u And f v Similarly, the pinhole camera model is a simple theoretical model used to simulate conventional phasesThe imaging process of the machine will not be described in detail here. Wherein, the above c u 、c v 、f u And f v The calculation method can be used for explaining the principle, the actual parameters can be obtained by measuring and calculating by other existing methods, and the method is not particularly limited and only needs to enable the parameter c u 、c v 、f u And f v Can be used for constructing the parameter matrix K.
Focal length f in horizontal pixels according to a planar image capture device u And a focal length f in vertical pixels v And a horizontal offset c of a principal point of the planar image acquisition device on the planar image u And a vertical offset c v And constructing a parameter matrix K.
Figure BDA0003645391180000081
Then, the pixel coordinates (u, v) in the plane image coordinate system, and the coordinate data (x ', y ') mapped thereto in the plane image acquisition apparatus coordinate system ' 0 Z') has the following relationship:
Figure BDA0003645391180000082
then
Figure BDA0003645391180000091
Then
Figure BDA0003645391180000092
Wherein, y' 0 For the measured distance between the planar image acquisition device and the plane of the object, such as: in the field of automatic driving, if the object to be marked is a driving road, the distance between the plane image acquisition equipment and the plane where the object is located is the plane image acquisitionDistance between the optical center of the lens of the device and the ground. Further obtaining a mapping relation:
Figure BDA0003645391180000101
in this way, the pixel coordinates (u, v) of the two-dimensional object pixel points in the plane image are substituted into the mapping relation between the plane image coordinate system and the plane image acquisition equipment coordinate system to obtain the coordinate data (x ', y ') of the three-dimensional object pixel points in the plane image acquisition equipment coordinate system ' 0 ，z′)。
Here, it should be noted that, if the original planar image acquired by the planar image acquisition device is edited before the annotation, the parameters in the parameter demonstration need to be adjusted correspondingly, for example: c, enlarging or reducing the original plane image by a u 、c v 、f u And f v Are correspondingly enlarged or reduced by a times. C, cutting off b pixels from the edge of the original plane image v Correspondingly reducing b, cutting c pixels from the left edge of the original plane image, and then c u The correspondence is decreased by c.
Here, it should be noted that the mapping relationship between the plane image coordinate system and the plane image capturing device coordinate system can also be obtained by measurement, for example: the method comprises the steps of selecting more than two reference objects in a shooting range of the plane image acquisition equipment, respectively measuring coordinate data of the more than two reference objects in a plane image coordinate system and a plane image acquisition equipment coordinate system, and then obtaining a mapping relation between the plane image coordinate system and the plane image acquisition equipment coordinate system through mathematical methods such as equation solving and the like.
Obtaining coordinate data (x ', y ') of each pixel point of the object in the plane image acquisition equipment coordinate system ' 0 Z'), converting the coordinate data of the object pixel points in the coordinate system of the planar image acquisition device into coordinate data (x, y,z). And the spatial data acquisition equipment coordinate system is constructed according to the spatial data acquisition equipment and the plane figure acquisition equipment coordinate system.
Preferably, the space data acquisition device selects a laser radar, refer to fig. 2, and constructs a space data acquisition device coordinate system by using a radio frequency emission point of the laser radar as an origin of a space data acquisition device coordinate system, using a positive direction of a z-axis of the plane image acquisition device coordinate system as a positive direction of an x-axis of the space data acquisition device coordinate system, using a positive direction of the x-axis of the plane image acquisition device coordinate system as a negative direction of a y-axis of the space data acquisition device coordinate system, and using a positive direction of the y-axis of the plane image acquisition device coordinate system as a negative direction of the z-axis of the space data acquisition device coordinate system.
In one possible implementation, the coordinate transformation matrix is obtained based on a relationship between the orientation of the coordinate axes of the coordinate system of the planar image acquisition device and the orientation of the coordinate axes of the coordinate system of the spatial data acquisition device. And obtaining a mapping relation between a coordinate system of the plane image acquisition equipment and a coordinate system of the space data acquisition equipment by using a coordinate conversion matrix, bringing coordinate data of the object under the coordinate system of the plane image data acquisition equipment into the mapping relation to obtain coordinate data of each pixel point of the object under the coordinate system of the space data acquisition equipment, and correspondingly marking the object into the space data according to the object to finish marking.
Wherein the coordinate transformation matrix is 4 × 4 matrix, and calculating inverse matrix T of the coordinate transformation matrix ′ The following equation is obtained:
Figure BDA0003645391180000111
further obtaining a mapping relation:
Figure BDA0003645391180000112
coordinate data (x) of a plane image acquisition device coordinate system ′ ，y 0 ′ ，z ′ ) And substituting the formula into the above formula to obtain coordinate data (x, y, z) under the corresponding spatial data acquisition equipment coordinate system.
The coordinate transformation matrix is obtained based on the relationship between the coordinate axis orientation of the plane image acquisition equipment coordinate system and the coordinate axis orientation of the space data acquisition equipment coordinate system, namely the mapping relationship between the X-axis unit vector, the Y-axis unit vector and the Z-axis unit vector of the plane image acquisition equipment coordinate system and the X-axis unit vector, the Y-axis unit vector and the Z-axis unit vector of the space data acquisition equipment coordinate system.
Further, referring to fig. 2, a unit vector (1, 0, 0) of an x-axis of a coordinate system of the spatial data acquisition device corresponds to a unit vector (0, 0, 1) of a z-axis of a coordinate system of the planar image acquisition device, a unit vector (0, 1, 0) of a y-axis of a coordinate system of the spatial data acquisition device corresponds to a unit vector (-1, 0, 0) of an x-axis of a coordinate system of the planar image acquisition device, and a unit vector (0, 0, 1) of a z-axis of a coordinate system of the spatial data acquisition device corresponds to a unit vector (0, -1, 0) of a y-axis of a coordinate system of the planar image acquisition device, thereby obtaining a rotation matrix
Figure BDA0003645391180000121
The method is used for converting coordinate data between a coordinate system of the space data acquisition equipment and a coordinate system of the plane image acquisition equipment.
However, in the actual use process, the plane image acquisition device and the spatial data acquisition device cannot be arranged in a superposition manner, so that the origin of the coordinate system of the plane image acquisition device and the origin of the coordinate system of the spatial data acquisition device are superposed. Therefore, a translation matrix is added to compensate for the position difference between the two coordinate systems. And obtaining a translation matrix according to the coordinate data of the origin of the coordinate system of the space data acquisition equipment under the coordinate system of the plane image acquisition equipment.
Further, the coordinates of the origin of the coordinate system of the spatial data acquisition equipment under the coordinate system of the plane image acquisition equipment are (a, b, c), and then a translation matrix is constructed
Figure BDA0003645391180000122
Merging the translation matrix and the rotation matrix to the right side to obtain a merged matrix, and adding a matrix [0, 0, 0, 1] below the merged matrix to obtain the coordinate conversion matrix, namely the coordinate conversion matrix
Figure BDA0003645391180000123
Here, it should be noted that the coordinates of the origin of the coordinate system of the spatial data acquisition device in the coordinate system of the planar image acquisition device are (a, b, c), and are obtained by measuring the transverse distance, the longitudinal distance and the height distance between the planar image acquisition device and the spatial data acquisition device.
Still further, according to another aspect of the present disclosure, there is also provided an object labeling apparatus 100. Referring to fig. 4, an object labeling apparatus 100 according to an embodiment of the present disclosure includes: an acquisition module 110, an annotation module 120, a first transformation module 130, and a second transformation module 140. The acquisition module 110 is configured to acquire a planar image to be currently subjected to object labeling and spatial data corresponding to the planar image. The marking module 120 is configured to mark the object in the planar image and obtain pixel coordinates of each pixel point of the object in the planar image. The first conversion module 130 is configured to convert the pixel coordinates of each pixel point of the object in the planar image based on the acquisition parameters of the planar image acquisition device, so as to obtain the coordinate data of each pixel point of the object in the planar image acquisition device coordinate system. The second conversion module 140 is configured to convert, according to a coordinate conversion matrix from a planar image acquisition device coordinate system to a spatial data acquisition device coordinate system, coordinate data of each pixel point of the object in the planar image acquisition device coordinate system to obtain coordinate data of each pixel point of the object in the spatial data acquisition device coordinate system, and label the object in the spatial data correspondingly according to the coordinate data of each pixel point of the object in the spatial data acquisition device coordinate system.
Still further, according to another aspect of the present disclosure, there is also provided an object labeling apparatus 200. Referring to fig. 5, an object labeling apparatus 200 according to an embodiment of the present disclosure includes a processor 210 and a memory 220 for storing instructions executable by the processor 210. Wherein the processor 210 is configured to implement any of the object labeling methods described above when executing the executable instructions.
Here, it should be noted that the number of the processors 210 may be one or more. Meanwhile, in the object labeling apparatus 200 of the embodiment of the present disclosure, an input device 230 and an output device 240 may also be included. The processor 210, the memory 220, the input device 230, and the output device 240 may be connected via a bus, or may be connected via other methods, which is not limited in this embodiment.
The memory 220, which is a computer-readable storage medium, may be used to store software programs, computer-executable programs, and various modules, such as: the object labeling method of the embodiment of the present disclosure corresponds to a program or a module. The processor 210 performs various functional applications and data processing of the object labeling apparatus 200 by executing software programs or modules stored in the memory 220.
The input device 230 may be used to receive an input number or signal. Wherein the signal may be a key signal generated in connection with user settings and function control of the device/terminal/server. The output device 240 may include a display device such as a display screen.
According to another aspect of the present disclosure, there is also provided a non-transitory computer readable storage medium having stored thereon computer program instructions which, when executed by the processor 210, implement the object annotation method as described in any of the preceding.
Having described embodiments of the present disclosure, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen in order to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims (10)

1. An object labeling method, comprising:
acquiring a planar image to be subjected to object labeling at present and spatial data corresponding to the planar image;
marking an object to be marked currently in the plane image, and acquiring pixel coordinates of each pixel point of the object in the plane image;
converting the pixel coordinates of each pixel point of the object in the plane image based on the acquisition parameters of the plane image acquisition equipment to obtain the coordinate data of each pixel point of the object in the coordinate system of the plane image acquisition equipment;
converting the coordinate data of each pixel point of the object under the coordinate system of the plane acquisition equipment according to a coordinate conversion matrix from the coordinate system of the plane image acquisition equipment to the coordinate system of the space data acquisition equipment to obtain the coordinate data of each pixel point of the object under the coordinate system of the space data acquisition equipment;
and correspondingly marking the object into the spatial data according to the coordinate data of each pixel point of the object in the coordinate system of the spatial data acquisition equipment.
2. The method of claim 1, wherein the acquisition parameters of the planar image acquisition device comprise at least one of a focal length of the planar image acquisition device, the resolution of the acquired planar image, and a sensor size of the planar image acquisition device.
3. The method according to claim 2, wherein the focal length in horizontal pixels and the focal length in vertical pixels of the planar image acquisition device are respectively obtained according to the focal length of the planar image acquisition device, the acquired planar image resolution and the sensor size of the planar image acquisition device;
and respectively obtaining the horizontal offset and the vertical offset of the principal point of the plane image acquisition equipment on the plane image according to the resolution of the plane image acquired by the plane image acquisition equipment.
4. The method according to claim 3, wherein when converting pixel coordinates of each pixel point of the object in the plane image based on an acquisition parameter of a plane image acquisition device to obtain coordinate data of each pixel point of the object in a coordinate system of the plane image acquisition device, the method comprises:
and substituting the pixel coordinates of each two-dimensional pixel point of the object in the plane image into the mapping relation between the plane image coordinate system and the image acquisition equipment coordinate system to obtain the coordinate data of each three-dimensional pixel point of the object in the plane image acquisition equipment coordinate system.
5. The method of claim 4, wherein the mapping relationship between a planar image coordinate system and an image acquisition device coordinate system is:
Figure FDA0003645391170000021
wherein (u, v) is the pixel coordinate of the pixel point of the object in the planar image, (x ', y' 0 Z ') is the coordinate of the pixel point of the converted object in the plane acquisition equipment coordinate system, y' 0 For the measured distance between the planar image acquisition device and the plane of the object, f u And f v Focal lengths in units of horizontal pixels and of the planar image pickup device, respectivelyFocal length in vertical pixels, c u And c v Respectively a horizontal offset and a vertical offset of a principal point of the planar image acquisition device on the planar image.
6. The method of claim 1, wherein the coordinate transformation matrix is derived based on a relationship between orientation of coordinate axes of the planar image acquisition device coordinate system and orientation of coordinate axes of the spatial data acquisition device coordinate system.
7. The method according to claim 6, wherein a rotation matrix is obtained according to the X-axis unit vector, the Y-axis unit vector and the Z-axis unit vector of the plane image acquisition device coordinate system and the X-axis unit vector, the Y-axis unit vector and the Z-axis unit vector of the space data acquisition device coordinate system;
obtaining a translation matrix according to coordinate data of the origin of the coordinate system of the spatial data acquisition equipment in the coordinate system of the plane image acquisition equipment;
and merging the translation matrix to the right side of the rotation matrix to obtain a merged matrix, and adding a matrix [0, 0, 0, 1] below the merged matrix to obtain the coordinate conversion matrix.
8. An object labeling apparatus, comprising: the system comprises an acquisition module, a labeling module, a first conversion module and a second conversion module;
the acquisition module is configured to acquire a planar image to be subjected to object labeling currently and spatial data corresponding to the planar image;
the marking module is configured to mark the object in the planar graph and acquire pixel coordinates of each pixel point of the object in the planar graph;
the first conversion module is configured to convert pixel coordinates of each pixel point of the object in the planar image based on an acquisition parameter of a planar image acquisition device to obtain coordinate data of each pixel point of the object in a coordinate system of the planar image acquisition device;
the second conversion module is configured to convert, according to a coordinate conversion matrix from a plane image acquisition device coordinate system to a spatial data acquisition device coordinate system, coordinate data of each pixel point of the object in the plane image acquisition device coordinate system to obtain coordinate data of each pixel point of the object in the spatial data acquisition device coordinate system, and label the object in the spatial data correspondingly according to the coordinate data of each pixel point of the object in the spatial data acquisition device coordinate system.
9. An object labeling apparatus, comprising:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to carry out the executable instructions when implementing the method of any one of claims 1 to 7.
10. A non-transitory computer readable storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, implement the method of any of claims 1 to 7.
CN202210528073.2A 2022-05-16 2022-05-16 Object labeling method and device, equipment and storage medium Pending CN114972694A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210528073.2A CN114972694A (en) 2022-05-16 2022-05-16 Object labeling method and device, equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210528073.2A CN114972694A (en) 2022-05-16 2022-05-16 Object labeling method and device, equipment and storage medium

Publications (1)

Publication Number Publication Date
CN114972694A true CN114972694A (en) 2022-08-30

Family

ID=82983100

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210528073.2A Pending CN114972694A (en) 2022-05-16 2022-05-16 Object labeling method and device, equipment and storage medium

Country Status (1)

Country Link
CN (1) CN114972694A (en)

Similar Documents

Publication Publication Date Title
CN112894832B (en) Three-dimensional modeling method, three-dimensional modeling device, electronic equipment and storage medium
KR101666959B1 (en) Image processing apparatus having a function for automatically correcting image acquired from the camera and method therefor
CN111062873B (en) Parallax image splicing and visualization method based on multiple pairs of binocular cameras
CN107223269B (en) Three-dimensional scene positioning method and device
JP5739584B2 (en) 3D image synthesizing apparatus and method for visualizing vehicle periphery
CN111028155B (en) Parallax image splicing method based on multiple pairs of binocular cameras
CN112444242B (en) Pose optimization method and device
JPWO2018235163A1 (en) Calibration apparatus, calibration chart, chart pattern generation apparatus, and calibration method
SG189284A1 (en) Rapid 3d modeling
CN109741241B (en) Fisheye image processing method, device, equipment and storage medium
GB2580691A (en) Depth estimation
CN108362205B (en) Space distance measuring method based on fringe projection
CN114494388B (en) Three-dimensional image reconstruction method, device, equipment and medium in large-view-field environment
US20230351625A1 (en) A method for measuring the topography of an environment
CN114761997A (en) Target detection method, terminal device and medium
CN113379815A (en) Three-dimensional reconstruction method and device based on RGB camera and laser sensor and server
CN114616586A (en) Image annotation method and device, electronic equipment and computer-readable storage medium
CN115423863B (en) Camera pose estimation method and device and computer readable storage medium
CN114972694A (en) Object labeling method and device, equipment and storage medium
CN109089100B (en) Method for synthesizing binocular stereo video
CN112995641A (en) 3D module imaging device and method and electronic equipment
CN115578466A (en) Camera calibration method and device, computer readable storage medium and electronic equipment
CN114766039A (en) Object detection method, object detection device, terminal device, and medium
WO2021014538A1 (en) Template creation device, object recognition processing device, template creation method, object recognition processing method, and program
CN117095131B (en) Three-dimensional reconstruction method, equipment and storage medium for object motion key points

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination