CN114611635A - Object identification method and device, storage medium and electronic device - Google Patents
Object identification method and device, storage medium and electronic device Download PDFInfo
- Publication number
- CN114611635A CN114611635A CN202210506905.0A CN202210506905A CN114611635A CN 114611635 A CN114611635 A CN 114611635A CN 202210506905 A CN202210506905 A CN 202210506905A CN 114611635 A CN114611635 A CN 114611635A
- Authority
- CN
- China
- Prior art keywords
- target
- point cloud
- data
- point
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Abstract
The embodiment of the invention provides an object identification method, an object identification device, a storage medium and an electronic device, wherein the method comprises the following steps: determining target point cloud obtained by shooting a target area by first equipment at a target moment; determining target subdata included in target data acquired by the second equipment at a target moment and determining data characteristics of the target subdata; mapping each pixel point included in the target subdata to a point cloud coordinate system where the target point cloud is located to obtain a plurality of target pixel points; determining point cloud characteristics based on the plurality of target pixel points and the target point cloud; fusing the data characteristics and the point cloud characteristics based on the dimension parameters of the data characteristics to obtain fused characteristics; the target object is identified based on the fused features. By the method and the device, the problem of inaccurate identification of the object in the related technology is solved, and the effect of improving the accuracy of the identification of the object is achieved.
Description
Technical Field
The embodiment of the invention relates to the field of computers, in particular to an object identification method, an object identification device, a storage medium and an electronic device.
Background
In the related art, image recognition technology under monocular vision is generally used for recognizing an object. However, monocular vision techniques also have some limitations, such as insufficient capability for distance measurement, object size, and performance in low light, resulting in inaccurate recognition results.
Therefore, the problem that the identification object is inaccurate exists in the related art.
In view of the above problems in the related art, no effective solution has been proposed.
Disclosure of Invention
The embodiment of the invention provides an object identification method, an object identification device, a storage medium and an electronic device, which are used for at least solving the problem of inaccurate object identification in the related art.
According to an embodiment of the present invention, there is provided an object recognition method including: determining a target point cloud obtained by shooting a target area by first equipment at a target moment; determining target subdata included in target data acquired by second equipment at the target moment, and determining data characteristics of the target subdata, wherein the target data are data obtained by shooting the target area by the second equipment, the shooting angle of the target area by the second equipment is the same as the shooting angle of the target area by the first equipment, and the target subdata is data of a target object included in the target data; mapping each pixel point included in the target subdata to a point cloud coordinate system where the target point cloud is located to obtain a plurality of target pixel points; determining a point cloud feature based on a plurality of the target pixel points and the target point cloud; fusing the data features and the point cloud features based on the dimension parameters of the data features to obtain fused features; identifying the target object based on the fused features.
According to another embodiment of the present invention, there is provided an identification apparatus of an object, including: the first determining module is used for determining a target point cloud obtained by shooting a target area by first equipment at a target moment; a second determining module, configured to determine target sub-data included in target data acquired by a second device at the target time, and determine data characteristics of the target sub-data, where the target data is obtained by shooting the target area by the second device, an angle of shooting the target area by the second device is the same as an angle of shooting the target area by the first device, and the target sub-data is data of a target object included in the target data; the mapping module is used for mapping each pixel point included in the target subdata to a point cloud coordinate system where the target point cloud is located to obtain a plurality of target pixel points; a third determining module for determining point cloud characteristics based on the plurality of target pixel points and the target point cloud; the fusion module is used for fusing the data features and the point cloud features based on the dimension parameters of the data features to obtain fusion features; an identification module to identify the target object based on the fused feature.
According to yet another embodiment of the invention, there is also provided a computer-readable storage medium having a computer program stored therein, wherein the computer program, when executed by a processor, implements the steps of the method as set forth in any of the above.
According to yet another embodiment of the present invention, there is also provided an electronic device, including a memory in which a computer program is stored and a processor configured to execute the computer program to perform the steps in any of the above method embodiments.
According to the method, the target point cloud obtained by shooting the target area by the first equipment at the target moment is determined, the target subdata contained in the target data acquired by the target equipment by the second equipment is determined, and the target characteristics of the target subdata are determined, wherein the target data are the data obtained by shooting the target area by the second equipment, the shooting angle of the target area by the first equipment to the target area is the same as the shooting angle of the target area by the second equipment, the target subdata is the data of the target object contained in the target data, and each pixel point contained in the target data is mapped to the point cloud coordinate system where the target point cloud is located to obtain a plurality of target pixel points; determining point cloud characteristics according to the target pixel points and the target point cloud, fusing the data characteristics and the point cloud characteristics according to the dimension parameters of the data characteristics to obtain fusion characteristics, and identifying a target object according to the fusion characteristics. Because the target object can be identified according to the fusion characteristics fused with the point cloud characteristics and the data characteristics, namely, when the target object is identified, the data collected by a plurality of devices is fused, the problem that the identified object is inaccurate in the related technology can be solved, and the effect of improving the accuracy of the identified object is achieved.
Drawings
Fig. 1 is a block diagram of a hardware configuration of a mobile terminal of an object recognition method according to an embodiment of the present invention;
FIG. 2 is a flow chart of a method of identifying an object according to an embodiment of the invention;
FIG. 3 is a schematic diagram of a secondary network module according to an exemplary embodiment of the present invention;
FIG. 4 is a flow chart of a method for identifying objects in accordance with a specific embodiment of the present invention;
fig. 5 is a block diagram of a structure of an apparatus for recognizing an object according to an embodiment of the present invention.
Detailed Description
Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings in conjunction with the embodiments.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order.
The method embodiments provided in the embodiments of the present application may be executed in a mobile terminal, a computer terminal, or a similar computing device. Taking an example of the method running on a mobile terminal, fig. 1 is a hardware structure block diagram of the mobile terminal of an object identification method according to an embodiment of the present invention. As shown in fig. 1, the mobile terminal may include one or more (only one shown in fig. 1) processors 102 (the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), and a memory 104 for storing data, wherein the mobile terminal may further include a transmission device 106 for communication functions and an input-output device 108. It will be understood by those skilled in the art that the structure shown in fig. 1 is only an illustration, and does not limit the structure of the mobile terminal. For example, the mobile terminal may also include more or fewer components than shown in FIG. 1, or have a different configuration than shown in FIG. 1.
The memory 104 may be used for storing computer programs, for example, software programs and modules of application software, such as computer programs corresponding to the object identification method in the embodiment of the present invention, and the processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, so as to implement the above-mentioned method. The memory 104 may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory located remotely from the processor 102, which may be connected to the mobile terminal over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission device 106 is used to receive or transmit data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a Network adapter (NIC) that can be connected to other Network devices through a base station to communicate with the internet. In one example, the transmission device 106 may be a Radio Frequency (RF) module, which is used to communicate with the internet in a wireless manner.
In the present embodiment, a method for identifying an object is provided, and fig. 2 is a flowchart of the method for identifying an object according to the embodiment of the present invention, as shown in fig. 2, the flowchart includes the following steps:
step S202, determining a target point cloud obtained by shooting a target area by first equipment at a target moment;
step S204, determining target subdata included in target data acquired by second equipment at the target moment, and determining data characteristics of the target subdata, wherein the target data is data obtained by shooting the target area by the second equipment, the shooting angle of the target area by the second equipment is the same as the shooting angle of the target area by the first equipment, and the target subdata is data of a target object included in the target data;
step S206, mapping each pixel point included in the target subdata to a point cloud coordinate system where the target point cloud is located to obtain a plurality of target pixel points;
step S208, determining point cloud characteristics based on the target pixel points and the target point cloud;
step S210, fusing the data features and the point cloud features based on the dimension parameters of the data features to obtain fusion features;
step S212, identifying the target object based on the fusion characteristics.
In the above embodiments, the first device may be a radar, such as a lidar, a microwave radar, a millimeter wave radar, or the like. The second device may be an image pickup device such as a camera (monocular camera ), video recorder, or the like. The target data can be images, videos and the like acquired by the camera equipment. The first device and the second device can be installed at the same height, the same orientation and adjacent positions, so that the angles of the target area shot by the first device and the second device are the same. Of course, the first device and the second device may be arranged to have the same orientation and different positions, and the shooting angles of the first device and the second device when shooting the target area are adjusted to be the same.
In the above embodiment, the target sub-data may be data of a target object, and when the target sub-data is determined, 2D frame detection may be performed on the target data acquired by the second device by using an object detection algorithm, a rectangular frame is determined to select the target object in the target data, and data included in the rectangular frame is determined as the target sub-data. The target sub data may include position information of a rectangular frame, pixel size information of a rectangular frame, and the like. That is, the target sub-data can be expressed as
Wherein, in the step (A),
) The pixel coordinate position of the target point of the rectangular frame which is the ith object,
) The width and height of the pixel of the rectangular frame of the ith vehicle. Wherein the target point may be a center point, a vertex, etc. of the rectangular frame.
In the above embodiment, the first device and the second device may be devices that complete joint calibration in advance, and may calculate in advance camera external parameters that the laser radar coordinate system maps to the monocular camera coordinate system, so that the target point cloud obtained by scanning the laser radar may be correctly projected onto the camera image picture and correct mapping is completed. The target point cloud acquired by the first device may include a plurality of point clouds of objects, i.e., the target point cloud may include a plurality of sub-point clouds. The target data may include data of a plurality of objects, that is, the target data may include a plurality of target sub-data. Each pixel point included in the target sub-data can be mapped to a point cloud coordinate system where the target point cloud is located, and a plurality of target pixel points are obtained.
In the above embodiment, the point cloud characteristics may be determined according to the target pixel points and the target point cloud, and the data characteristics and the point cloud characteristics are fused according to the dimension parameters of the data characteristics of the target sub-data to obtain the fusion characteristics. And inputting the fusion characteristics into a recognition network, and recognizing the target object. Wherein identifying the target object includes identifying attribute information of the target object. The target object may include a vehicle, a person, and the like. When the target object is a vehicle, the attribute information of the target object may include a vehicle type, a license plate, and the like. When the target object is a person, the attribute information of the target object may include information such as face information and an identification number.
In the above embodiment, the fusion features may be determined by the target network model, and the fusion features may be transmitted to the secondary network model and the classifier included in the target network model to identify the target object. Namely Fusion _ Fea in the Fusion featureiThe secondary network module and the classifier can be connected, and the sgd stochastic gradient descent optimizer is used for training and optimizing the whole network. The secondary network module may comprise 3 consecutive 3x3 convolutional layers followed by 1x1 convolutional layers and 1 fc fully-connected layers, and the schematic diagram of the secondary network module can be seen in fig. 3. The classifier may be a cross entropy loss function loss
。
Optionally, the main body of the above steps may be a background processor or other devices with similar processing capabilities, and may also be a machine integrated with at least a data processing device, where the data processing device may include a terminal such as a computer, a mobile phone, and the like, but is not limited thereto.
According to the method, the target point cloud obtained by shooting the target area by the first equipment at the target moment is determined, the target subdata contained in the target data acquired by the target equipment by the second equipment is determined, and the target characteristics of the target subdata are determined, wherein the target data are the data obtained by shooting the target area by the second equipment, the shooting angle of the target area by the first equipment to the target area is the same as the shooting angle of the target area by the second equipment, the target subdata is the data of the target object contained in the target data, and each pixel point contained in the target data is mapped to the point cloud coordinate system where the target point cloud is located to obtain a plurality of target pixel points; determining point cloud characteristics according to the target pixel points and the target point cloud, fusing the data characteristics and the point cloud characteristics according to the dimension parameters of the data characteristics to obtain fusion characteristics, and identifying a target object according to the fusion characteristics. Because the target object can be identified according to the fusion characteristics fused with the point cloud characteristics and the data characteristics, namely, when the target object is identified, the data collected by a plurality of devices is fused, the problem that the identified object is inaccurate in the related technology can be solved, and the effect of improving the accuracy of the identified object is achieved.
In one exemplary embodiment, determining a point cloud feature based on a plurality of the target pixel points and the target point cloud comprises: and executing the following operations for each target pixel point to obtain a characteristic value corresponding to each target pixel point: determining whether a first point with the same coordinate as the target pixel point is included in the target point cloud; under the condition that a first point exists, determining the vertical coordinate of the first point and the response intensity corresponding to the first point as the characteristic value of the target pixel point; under the condition that the first point does not exist, determining a second point which is closest to the target pixel point in the target point cloud, and determining the vertical coordinate of the second point and the response intensity corresponding to the second point as the characteristic value of the target pixel point; and determining a matrix formed by a plurality of characteristic values as the point cloud characteristic. In this embodiment, the target point cloud may be a three-dimensional point cloud, and each point included in the target point cloud may be represented as
Wherein x, y and z are j point clouds
And (3) three-dimensional coordinates under a cloud coordinate system, wherein a is the laser response intensity value of the jth point cloud. For each target object, one can be constructed
A three-dimensional matrix of dimensions, and storing the z value and the a value of the corresponding point cloud point according to the mapped pixel coordinate position,filling the pixel points which are not mapped with the z value and the a value of the nearest neighbor to obtain the point cloud characteristic matrix of each object
. And determining the point cloud characteristic matrix as the point cloud characteristics.
In an exemplary embodiment, in the case that a first point exists, determining a vertical coordinate of the first point and a response strength corresponding to the first point as a feature value of the target pixel point includes: determining the vertical coordinate of the first point and the response intensity corresponding to the first point as the characteristic value of the target pixel point under the condition that the number of the first points is one; and under the condition that the number of the first points is multiple, determining a third point with the minimum vertical coordinate in the multiple first points, and determining the vertical coordinate of the third point and the response intensity corresponding to the third point as the characteristic value of the target pixel point. In this embodiment, when each pixel point is mapped to the point cloud coordinate system, each target subdata is mapped to a plurality of discrete point cloud points. If a plurality of point clouds are mapped to the same image pixel, the point cloud with the minimum z value is selected, and other points are discarded.
In one exemplary embodiment, determining the data characteristics of the target sub-data comprises: inputting the target subdata into a target network model, and determining the characteristics extracted by the target network model; and determining the features extracted by any network layer as the data features, wherein the network layer is a network layer included in the backbone network of the target network model. In this embodiment, the target sub-data may be input into a target network model, such as a convolutional neural network model CNN, VGG, ResNet, mobilene, and the like, as a backbone network of the target network model. Deep network characteristic Camera _ Fea of each target subdata extracted by using backbone networki。
In the above embodiment, when the backbone network requires the size of the input data, if the input size of the backbone network is set to 224 × 3, the target sub-data can be unified resize into 224 × 3.
In the above embodiment, any layer of extracted deep-layer network features Camera _ Fea of the target network model may be selectediAs the data characteristics of the target sub-data. For example, the output of the network block layer that is 16 times down-sampled by the network may be used as the Camera _ FeaiI.e. data features, have a dimension of 7 × d, where d is the number of channels of the layer feature and 7 is 224 ⁄ 16.
In an exemplary embodiment, fusing the data feature and the point cloud feature based on the dimensional parameter of the data feature to obtain a fused feature includes: adjusting the point cloud characteristics according to the dimension parameters to obtain target point cloud characteristics; and fusing the data features and the target point cloud features to obtain the fused features. In this embodiment, when determining the fusion feature, the point cloud feature and the data feature may be unified into the same size, for example, the point cloud feature may be adjusted according to the dimension parameter of the data feature to obtain the target point cloud feature. And fusing the target point cloud characteristics and the data characteristics to obtain fused characteristics.
In the above embodiment, the data feature may also be adjusted according to the dimension parameter of the point cloud feature to obtain a target data feature, and the target data feature and the point cloud feature are fused to obtain a fusion feature.
It should be noted that the size relationship between the dimension parameter of the point cloud feature and the dimension parameter of the data feature may be determined, and the feature with a large dimension parameter may be adjusted according to the dimension parameter with a small dimension parameter.
In an exemplary embodiment, fusing the data feature and the target point cloud feature to obtain the fused feature includes: and connecting the data features and the target point cloud features according to the channel dimensions included in the dimension parameters to obtain the fusion features. In this embodiment, when fusing the data feature and the target point cloud feature, the features with the same dimensionality may be directly connected to obtain a fused feature. For example, the target point cloud feature Lidar _ Fea may be combinediSame data feature Camera _ FeaiConcat together according to channel dimension to obtain a blendBlend feature Fusion _ FeaiThe dimension is 7 × 2.
In an exemplary embodiment, adjusting the point cloud feature according to the dimension parameter to obtain the target point cloud feature includes: adjusting the point cloud characteristics by using a linear interpolation algorithm, and enabling the dimensionality of the adjusted point cloud characteristics to be the dimensionality parameter; and determining the adjusted point cloud characteristics as the target point cloud characteristics. In this embodiment, the point cloud feature Lidar _ Fea can be usediObtaining the data feature Camera _ Fea by nearest neighbor interpolation resize to the dimension of the data feature, e.g., 7 × 2 dimensioniThe first two dimensions are consistent.
The following describes an object recognition method with reference to specific embodiments:
fig. 4 is a flowchart of an object identification method according to an embodiment of the present invention, and as shown in fig. 4, the method includes:
step S402, a monocular camera collects a color image, and a laser radar collects a laser point cloud. The monocular camera and the laser radar are calibrated in a combined mode.
And S404, detecting the vehicle to obtain a vehicle subgraph.
And step S406, mapping to obtain the radar features of the vehicle.
Step S408, inputting the vehicle sub-graph (corresponding to the number target sub-data) into the main network module to obtain the image feature (corresponding to the data feature).
And step S410, performing linear interpolation on the radar features (corresponding to the point cloud features) of the vehicle to obtain radar features (corresponding to the target point cloud features).
And step S412, obtaining a concat image characteristic and a radar characteristic to obtain a fusion characteristic.
And step S414, inputting the fusion characteristics into the auxiliary network model and the classifier to obtain an identification result.
In the embodiment, the data acquired by the laser radar and the monocular camera in two different modes and under the same timestamp are respectively calculated, the three-dimensional features of the laser radar and the monocular camera are fused with the middle feature layer of the deep learning main network module of the monocular camera, and then the final vehicle type recognition result is obtained by training the deep learning main network module and the deep learning main network module through the auxiliary network module and the classifier. Carrying out 2D vehicle frame detection on the camera color image by adopting a vehicle detection model; registering the laser radar and the monocular camera, correctly mapping the 3D laser point cloud to a 2D camera plane, and acquiring three-dimensional point cloud characteristics in each vehicle frame; digging each vehicle color sub-image, extracting the image characteristic of the main network module, and performing concat connection with the point cloud characteristic of the vehicle; then, the auxiliary network module and the classifier are connected for classification training. Because of the technical limitation of monocular vision, some problems in vehicle type identification cannot be solved well, but along with the supplement of the scanning three-dimensional point cloud characteristics of the laser radar, the laser radar has congenital advantages in distance measurement, so that the laser radar can be used as a powerful supplement of a monocular camera, and the two are subjected to radar vision fusion to obtain a more accurate identification effect. The defects of a pure vision technology can be overcome, and the accuracy of vehicle type identification is greatly improved.
Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.
In this embodiment, an object recognition apparatus is further provided, and the apparatus is used to implement the foregoing embodiments and preferred embodiments, and the description of the apparatus is omitted for brevity. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. Although the means described in the embodiments below are preferably implemented in software, an implementation in hardware or a combination of software and hardware is also possible and contemplated.
Fig. 5 is a block diagram of a structure of an apparatus for recognizing an object according to an embodiment of the present invention, as shown in fig. 5, the apparatus including:
a first determining module 502, configured to determine a target point cloud obtained by shooting a target area by a first device at a target time;
a second determining module 504, configured to determine target sub-data included in target data acquired by a second device at the target time, and determine a data feature of the target sub-data, where the target data is data obtained by shooting the target area by the second device, an angle of shooting the target area by the second device is the same as an angle of shooting the target area by the first device, and the target sub-data is data of a target object included in the target data;
a mapping module 506, configured to map each pixel point included in the target sub-data to a point cloud coordinate system where the target point cloud is located, so as to obtain a plurality of target pixel points;
a third determining module 508, configured to determine a point cloud feature based on a plurality of the target pixel points and the target point cloud;
a fusion module 510, configured to fuse the data feature and the point cloud feature based on a dimension parameter of the data feature to obtain a fusion feature;
an identifying module 512 configured to identify the target object based on the fused feature.
In an exemplary embodiment, the third determining module 508 can determine point cloud features based on a plurality of the target pixel points and the target point cloud by: and executing the following operations for each target pixel point to obtain a characteristic value corresponding to each target pixel point: determining whether a first point with the same coordinate as the target pixel point is included in the target point cloud; under the condition that a first point exists, determining the vertical coordinate of the first point and the response intensity corresponding to the first point as the characteristic value of the target pixel point; under the condition that the first point does not exist, determining a second point which is closest to the target pixel point in the target point cloud, and determining the vertical coordinate of the second point and the response intensity corresponding to the second point as the characteristic value of the target pixel point; and determining a matrix formed by a plurality of characteristic values as the point cloud characteristic.
In an exemplary embodiment, the third determining module 508 may determine, in the presence of a first point, a vertical coordinate of the first point and a response strength corresponding to the first point as the feature value of the target pixel point by: determining the vertical coordinate of the first point and the response intensity corresponding to the first point as the characteristic value of the target pixel point under the condition that the number of the first points is one; and under the condition that the number of the first points is multiple, determining a third point with the minimum vertical coordinate in the multiple first points, and determining the vertical coordinate of the third point and the response intensity corresponding to the third point as the characteristic value of the target pixel point.
In an exemplary embodiment, the second determining module 504 may determine the data characteristics of the target sub-data by: inputting the target subdata into a target network model, and determining the characteristics extracted by the target network model; and determining the features extracted by any network layer as the data features, wherein the network layer is a network layer included in the backbone network of the target network model.
In an exemplary embodiment, the fusion module 510 may fuse the data feature and the point cloud feature based on the dimension parameter of the data feature to obtain a fused feature by: adjusting the point cloud characteristics according to the dimension parameters to obtain target point cloud characteristics; and fusing the data features and the target point cloud features to obtain the fused features.
In an exemplary embodiment, the fusion module 510 may fuse the data feature and the target point cloud feature to obtain the fusion feature by: and connecting the data features and the target point cloud features according to the channel dimensions included in the dimension parameters to obtain the fusion features.
In an exemplary embodiment, the fusion module 510 may adjust the point cloud feature according to the dimension parameter to obtain the target point cloud feature by: adjusting the point cloud characteristics by using a linear interpolation algorithm, and enabling the dimensionality of the adjusted point cloud characteristics to be the dimensionality parameter; and determining the adjusted point cloud characteristics as the target point cloud characteristics.
It should be noted that, the above modules may be implemented by software or hardware, and for the latter, the following may be implemented, but not limited to: the modules are all positioned in the same processor; alternatively, the modules are respectively located in different processors in any combination.
Embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method as set forth in any of the above.
In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to: various media capable of storing computer programs, such as a usb disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic disk, or an optical disk.
Embodiments of the present invention also provide an electronic device comprising a memory having a computer program stored therein and a processor arranged to run the computer program to perform the steps of any of the above method embodiments.
In an exemplary embodiment, the electronic apparatus may further include a transmission device and an input/output device, wherein the transmission device is connected to the processor, and the input/output device is connected to the processor.
For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary embodiments, and details of this embodiment are not repeated herein.
It will be apparent to those skilled in the art that the various modules or steps of the invention described above may be implemented using a general purpose computing device, they may be centralized on a single computing device or distributed across a network of computing devices, and they may be implemented using program code executable by the computing devices, such that they may be stored in a memory device and executed by the computing device, and in some cases, the steps shown or described may be performed in an order different than that described herein, or they may be separately fabricated into various integrated circuit modules, or multiple ones of them may be fabricated into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A method of identifying an object, comprising:
determining a target point cloud obtained by shooting a target area by first equipment at a target moment;
determining target subdata included in target data acquired by second equipment at the target moment, and determining data characteristics of the target subdata, wherein the target data are data obtained by shooting the target area by the second equipment, the shooting angle of the target area by the second equipment is the same as the shooting angle of the target area by the first equipment, and the target subdata is data of a target object included in the target data;
mapping each pixel point included in the target subdata to a point cloud coordinate system where the target point cloud is located to obtain a plurality of target pixel points;
determining a point cloud feature based on a plurality of the target pixel points and the target point cloud;
fusing the data features and the point cloud features based on the dimensional parameters of the data features to obtain fused features;
identifying the target object based on the fused features.
2. The method of claim 1, wherein determining point cloud characteristics based on a plurality of the target pixel points and the target point cloud comprises:
and executing the following operations for each target pixel point to obtain a characteristic value corresponding to each target pixel point: determining whether a first point with the same coordinate as the target pixel point is included in the target point cloud; under the condition that a first point exists, determining the vertical coordinate of the first point and the response intensity corresponding to the first point as the characteristic value of the target pixel point; under the condition that the first point does not exist, determining a second point which is closest to the target pixel point in the target point cloud, and determining the vertical coordinate of the second point and the response intensity corresponding to the second point as the characteristic value of the target pixel point;
and determining a matrix formed by a plurality of characteristic values as the point cloud characteristic.
3. The method of claim 2, wherein, in the case that a first point exists, determining a vertical coordinate of the first point and a response strength corresponding to the first point as the feature value of the target pixel point comprises:
determining the vertical coordinate of the first point and the response intensity corresponding to the first point as the characteristic value of the target pixel point under the condition that the number of the first points is one;
and under the condition that the number of the first points is multiple, determining a third point with the minimum vertical coordinate in the multiple first points, and determining the vertical coordinate of the third point and the response intensity corresponding to the third point as the characteristic value of the target pixel point.
4. The method of claim 1, wherein determining data characteristics of the target child data comprises:
inputting the target subdata into a target network model, and determining the characteristics extracted by the target network model;
and determining the features extracted by any network layer as the data features, wherein the network layer is a network layer included in the backbone network of the target network model.
5. The method of claim 1, wherein fusing the data feature and the point cloud feature based on the dimensional parameters of the data feature to obtain a fused feature comprises:
adjusting the point cloud characteristics according to the dimension parameters to obtain target point cloud characteristics;
and fusing the data characteristics and the target point cloud characteristics to obtain the fusion characteristics.
6. The method of claim 5, wherein fusing the data feature and the target point cloud feature to obtain the fused feature comprises:
and connecting the data features and the target point cloud features according to the channel dimensions included in the dimension parameters to obtain the fusion features.
7. The method of claim 5, wherein adjusting the point cloud feature according to the dimensional parameter to obtain a target point cloud feature comprises:
adjusting the point cloud characteristics by using a linear interpolation algorithm, and enabling the dimensionality of the adjusted point cloud characteristics to be the dimensionality parameter;
and determining the adjusted point cloud characteristics as the target point cloud characteristics.
8. An apparatus for identifying an object, comprising:
the first determining module is used for determining a target point cloud obtained by shooting a target area by first equipment at a target moment;
a second determining module, configured to determine target sub-data included in target data acquired by a second device at the target time, and determine data characteristics of the target sub-data, where the target data is obtained by shooting the target area by the second device, an angle of shooting the target area by the second device is the same as an angle of shooting the target area by the first device, and the target sub-data is data of a target object included in the target data;
the mapping module is used for mapping each pixel point included in the target subdata to a point cloud coordinate system where the target point cloud is located to obtain a plurality of target pixel points;
a third determining module for determining point cloud characteristics based on the plurality of target pixel points and the target point cloud;
the fusion module is used for fusing the data features and the point cloud features based on the dimension parameters of the data features to obtain fusion features;
an identification module to identify the target object based on the fused feature.
9. A computer-readable storage medium, in which a computer program is stored, which computer program, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 7.
10. An electronic device comprising a memory and a processor, wherein the memory has stored therein a computer program, and wherein the processor is arranged to execute the computer program to perform the method of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210506905.0A CN114611635B (en) | 2022-05-11 | 2022-05-11 | Object identification method and device, storage medium and electronic device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210506905.0A CN114611635B (en) | 2022-05-11 | 2022-05-11 | Object identification method and device, storage medium and electronic device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114611635A true CN114611635A (en) | 2022-06-10 |
| CN114611635B CN114611635B (en) | 2022-08-30 |
Family
ID=81870651
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210506905.0A Active CN114611635B (en) | 2022-05-11 | 2022-05-11 | Object identification method and device, storage medium and electronic device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114611635B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114782496A (en) * | 2022-06-20 | 2022-07-22 | 杭州闪马智擎科技有限公司 | Object tracking method and device, storage medium and electronic device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190122067A1 (en) * | 2017-01-24 | 2019-04-25 | Ford Global Technologies, Llc. | Object Detection Using Recurrent Neural Network And Concatenated Feature Map |
| CN111027401A (en) * | 2019-11-15 | 2020-04-17 | 电子科技大学 | End-to-end target detection method with integration of camera and laser radar |
| CN112927233A (en) * | 2021-01-27 | 2021-06-08 | 湖州市港航管理中心 | Marine laser radar and video combined target capturing method |
| CN114092850A (en) * | 2020-08-05 | 2022-02-25 | 北京万集科技股份有限公司 | Re-recognition method and device, computer equipment and storage medium |
| CN114155497A (en) * | 2021-09-24 | 2022-03-08 | 智道网联科技(北京)有限公司 | Object identification method and device and storage medium |
-
2022
- 2022-05-11 CN CN202210506905.0A patent/CN114611635B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190122067A1 (en) * | 2017-01-24 | 2019-04-25 | Ford Global Technologies, Llc. | Object Detection Using Recurrent Neural Network And Concatenated Feature Map |
| CN111027401A (en) * | 2019-11-15 | 2020-04-17 | 电子科技大学 | End-to-end target detection method with integration of camera and laser radar |
| CN114092850A (en) * | 2020-08-05 | 2022-02-25 | 北京万集科技股份有限公司 | Re-recognition method and device, computer equipment and storage medium |
| CN112927233A (en) * | 2021-01-27 | 2021-06-08 | 湖州市港航管理中心 | Marine laser radar and video combined target capturing method |
| CN114155497A (en) * | 2021-09-24 | 2022-03-08 | 智道网联科技(北京)有限公司 | Object identification method and device and storage medium |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114782496A (en) * | 2022-06-20 | 2022-07-22 | 杭州闪马智擎科技有限公司 | Object tracking method and device, storage medium and electronic device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114611635B (en) | 2022-08-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2022083402A1 (en) | Obstacle detection method and apparatus, computer device, and storage medium | |
| US10891500B2 (en) | Method and apparatus for acquiring traffic sign information | |
| KR102052114B1 (en) | Object change detection system for high definition electronic map upgrade and method thereof | |
| CN109918977B (en) | Method, device and equipment for determining idle parking space | |
| KR102103834B1 (en) | Object change detection system for high definition electronic map upgrade and method thereof | |
| CN115376109B (en) | Obstacle detection method, obstacle detection device, and storage medium | |
| CN111950543A (en) | Target detection method and device | |
| CN114863380B (en) | Lane line identification method and device and electronic equipment | |
| CN114219855A (en) | Point cloud normal vector estimation method and device, computer equipment and storage medium | |
| CN111553302B (en) | Key frame selection method, device, equipment and computer readable storage medium | |
| CN114611635B (en) | Object identification method and device, storage medium and electronic device | |
| CN111950428A (en) | Target obstacle identification method and device and carrier | |
| CN114663598A (en) | Three-dimensional modeling method, device and storage medium | |
| CN115147333A (en) | Target detection method and device | |
| CN115471574B (en) | External parameter determination method and device, storage medium and electronic device | |
| CN115830342A (en) | Method and device for determining detection frame, storage medium and electronic device | |
| CN114782496A (en) | Object tracking method and device, storage medium and electronic device | |
| CN115656991A (en) | Vehicle external parameter calibration method, device, equipment and storage medium | |
| CN112364693B (en) | Binocular vision-based obstacle recognition method, device, equipment and storage medium | |
| CN113469130A (en) | Shielded target detection method and device, storage medium and electronic device | |
| CN115063594B (en) | Feature extraction method and device based on automatic driving | |
| CN115830588B (en) | Target detection method, system, storage medium and device based on point cloud | |
| Berrio et al. | Semantic sensor fusion: From camera to sparse LiDAR information | |
| CN113870365B (en) | Camera calibration method, device, equipment and storage medium | |
| CN117576653A (en) | Target tracking method, device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information |
Inventor after: Ni Huajian Inventor after: Peng Yao Inventor after: Lin Yining Inventor after: Zhao Zhijian Inventor before: Peng Yao Inventor before: Ni Huajian Inventor before: Lin Yining Inventor before: Zhao Zhijian |
|
| CB03 | Change of inventor or designer information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |