CN115239803A - Data processing method and device - Google Patents

Data processing method and device Download PDF

Info

Publication number
CN115239803A
CN115239803A CN202110440989.8A CN202110440989A CN115239803A CN 115239803 A CN115239803 A CN 115239803A CN 202110440989 A CN202110440989 A CN 202110440989A CN 115239803 A CN115239803 A CN 115239803A
Authority
CN
China
Prior art keywords
plane
data
image data
depth camera
obtaining
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110440989.8A
Other languages
Chinese (zh)
Inventor
曲富平
王康
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alibaba Innovation Co
Original Assignee
Alibaba Singapore Holdings Pte Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba Singapore Holdings Pte Ltd filed Critical Alibaba Singapore Holdings Pte Ltd
Priority to CN202110440989.8A priority Critical patent/CN115239803A/en
Publication of CN115239803A publication Critical patent/CN115239803A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/70Determining position or orientation of objects or cameras
    • G06T7/73Determining position or orientation of objects or cameras using feature-based methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/70Denoising; Smoothing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/11Region-based segmentation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/80Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20112Image segmentation details
    • G06T2207/20132Image cropping
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30244Camera pose

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

The application discloses a data processing method, which comprises the following steps: obtaining coordinate information of a measurement target in a depth camera coordinate system; cutting image data acquired by the depth camera according to the region information to obtain an image data set for plane fitting; fitting the image data set according to a plane to obtain normal vector data of the plane; and obtaining correction data of the height of the measuring target from the ground according to the normal vector data of the plane and the coordinate information. By adopting the method, the problem that extra equipment needs to be added or the installation difficulty needs to be increased when the error caused by the camera posture is reduced in the prior art is solved.

Description

Data processing method and device
Technical Field
The present application relates to the field of computer technologies, and in particular, to a data processing method and a data processing apparatus. The application also relates to another data processing method.
Background
Depth cameras based on the ToF (Time-of-flight) principle have been applied to more and more scenes, such as a new retail line scene, an automatic container scene, and the like, due to the cost reduction and the gradual technical maturity in recent years. In these application scenarios, the height data of the measurement target (e.g. person, shelf separation) from the ground is the main input value of the algorithm. And this value can be measured without measuring or calibrating the installation attitude of the ToF camera. In the engineering installation process, because the field construction environment is harsh, the gesture collimation or consistency of the ToF camera is difficult to ensure.
In the prior art, in order to reduce the error caused by the camera pose, the following two methods are generally adopted: one is to accurately mark an angle by using a total station or an angle measuring instrument during the installation process, but the method introduces additional installation time, and the installation precision is difficult to ensure because the installation is usually performed aloft; another is to mount an additional gravitational acceleration sensor on the ToF camera, but this approach introduces additional cost.
Therefore, the prior art has the problem that additional equipment needs to be added or the installation difficulty needs to be increased when the error caused by the camera posture is reduced.
Disclosure of Invention
The application provides a data processing method and a data processing device, which aim to solve the problem that extra equipment needs to be added or the installation difficulty needs to be increased when the error caused by the camera attitude is reduced in the prior art.
The application provides a data processing method, which comprises the following steps:
obtaining coordinate information of a measurement target in a depth camera coordinate system;
cutting image data acquired by the depth camera according to the region information to obtain an image data set for plane fitting;
fitting the image data set according to a plane to obtain normal vector data of the plane;
and obtaining correction data of the height of the measuring target from the ground according to the normal vector data of the plane and the coordinate information.
Optionally, the cropping the image data acquired by the depth camera according to the region information to obtain an image data set for performing plane fitting, includes:
and cutting image data acquired by the depth camera according to the UV area coordinate information or the distance data to obtain an image data set for plane fitting.
Optionally, the cropping the image data acquired by the depth camera according to the region information to obtain an image data set for performing plane fitting, includes:
cutting image data acquired by a depth camera according to the region information to obtain an original image data set for plane fitting;
noise processing is carried out on the original image data set by adopting a neighbor algorithm to obtain a processed image data set;
the fitting the image data set according to a plane to obtain normal vector data of the plane includes:
and fitting the processed image data set according to a plane to obtain normal vector data of the plane.
Optionally, the obtaining coordinate information of the measurement target in the depth camera coordinate system includes:
obtaining a distance between an optical center of a depth camera and the measurement target;
and obtaining the coordinate information of the measuring target in a depth camera coordinate system according to the distance.
Optionally, the fitting the image data set according to a plane to obtain normal vector data of the plane includes:
obtaining point cloud data of a designated plane in the image dataset;
fitting the point cloud data according to a least square method to obtain a plane equation of the designated plane;
and obtaining normal vector data of the specified plane according to the plane equation.
Optionally, the obtaining, according to the normal vector data of the plane and the coordinate information, correction data of the height of the measurement target from the ground includes:
obtaining original data of the height of the measuring target from the ground according to the coordinate information;
obtaining trigonometric function data of an offset angle between a data coordinate axis of the depth camera and a coordinate axis of a ground coordinate system according to the normal vector data;
and obtaining correction data of the height of the measuring target from the ground according to the original data and the trigonometric function data.
Optionally, the offset angle between the data coordinate axis of the depth camera and the coordinate axis of the ground coordinate system includes at least one of the following angles:
an offset angle of the depth camera about an X-axis of a ground coordinate system;
offset angle of the depth camera about the Y-axis of the ground coordinate system.
Optionally, the obtaining, according to the original data and the trigonometric function data, correction data of the height of the measurement target from the ground includes:
obtaining a conversion relation between a corrected value of the height of the measuring target from the ground and the original data according to the original data and the trigonometric function data;
and obtaining correction data of the height of the measuring target from the ground according to the conversion relation.
The present application also provides a data processing apparatus, including:
a coordinate information obtaining unit for obtaining coordinate information of the measurement target in a depth camera coordinate system;
the image data set obtaining unit is used for cutting the image data obtained by the depth camera according to the region information to obtain an image data set used for carrying out plane fitting;
a normal vector data obtaining unit, configured to fit the image data set according to a plane to obtain normal vector data of the plane;
and the correction data obtaining unit is used for obtaining correction data of the height of the measuring target from the ground according to the normal vector data of the plane and the coordinate information.
Optionally, the image dataset obtaining unit is specifically configured to:
and cutting the image data acquired by the depth camera according to the UV area coordinate information or the distance data to obtain an image data set for plane fitting.
Optionally, the image dataset obtaining unit is specifically configured to:
cropping the image data acquired by the depth camera based on the region information, obtaining an original image dataset for performing plane fitting;
noise processing is carried out on the original image data set by adopting a neighbor algorithm to obtain a processed image data set;
the fitting the image data set according to a plane to obtain normal vector data of the plane includes:
and fitting the processed image data set according to a plane to obtain normal vector data of the plane.
Optionally, the coordinate information obtaining unit is specifically configured to:
obtaining a distance between an optical center of a depth camera and the measurement target;
and obtaining the coordinate information of the measuring target in a depth camera coordinate system according to the distance.
Optionally, the normal vector data obtaining unit is specifically configured to:
acquiring point cloud data of a designated plane in the image dataset;
fitting the point cloud data according to a least square method to obtain a plane equation of the designated plane;
and obtaining normal vector data of the specified plane according to the plane equation.
Optionally, the correction data obtaining unit is specifically configured to:
obtaining original data of the height of the measuring target from the ground according to the coordinate information;
obtaining trigonometric function data of an offset angle between a data coordinate axis of the depth camera and a coordinate axis of a ground coordinate system according to the normal vector data;
and obtaining correction data of the height of the measuring target from the ground according to the original data and the trigonometric function data.
Optionally, the offset angle between the data coordinate axis of the depth camera and the coordinate axis of the ground coordinate system includes at least one of the following angles:
an offset angle of the depth camera about an X-axis of a ground coordinate system;
offset angle of the depth camera about the Y-axis of the ground coordinate system.
Optionally, the correction data obtaining unit is specifically configured to:
obtaining a conversion relation between a corrected value of the height of the measuring target from the ground and the original data according to the original data and the trigonometric function data;
and obtaining correction data of the height of the measuring target from the ground according to the conversion relation.
The present application further provides a data processing method, including:
obtaining coordinate information of a measurement target in a depth camera coordinate system, and displaying the coordinate information in a graphical user interface;
cutting image data acquired by a depth camera according to the region information to obtain an image data set for plane fitting, and displaying the image data set in a graphical user interface;
fitting the image data set according to a plane to obtain normal vector data of the plane, and displaying the normal vector data in a graphical user interface;
and acquiring correction data of the height of the measuring target from the ground according to the normal vector data of the plane and the coordinate information, and displaying the correction data in a graphical user interface.
Compared with the prior art, the method has the following advantages:
the application provides a data processing method, which comprises the following steps: obtaining coordinate information of a measuring target in a depth camera coordinate system; cutting image data acquired by a depth camera to obtain an image data set for plane fitting; fitting the image data set according to a plane to obtain normal vector data of the plane; and obtaining correction data of the height of the measuring target from the ground according to the normal vector data of the plane and the coordinate information. According to the data processing method, the data of the height of the original measuring target from the ground output by the ToF camera is corrected by means of the measured value of the ToF camera to the ground, the real height value of the measuring target from the ground is obtained, the attitude error of the camera is reduced on the premise that no additional sensor (such as an accelerometer) is introduced and the construction difficulty is not increased, the coordinate value of the measuring target is converted into the height value to the ground, the input requirement of a target screening algorithm is met, and the identification precision of the new retail goods taking behavior is improved.
Drawings
Fig. 1 is a flowchart of a data processing method according to a first embodiment of the present application.
Fig. 2 is a schematic diagram of data coordinate axes and a ground coordinate system of a depth camera in an ideal installation state according to a first embodiment of the present disclosure.
Fig. 3 is a schematic imaging diagram of a depth camera according to a first embodiment of the present disclosure.
Fig. 4 is a top view of a floor area and a shelf area.
Fig. 5 is a schematic diagram of fitting the image data set obtained by cropping to a plane to obtain normal vector data of the plane.
Fig. 6 is a schematic diagram illustrating installation deviation of a depth camera according to a first embodiment of the present application.
Fig. 7 is a schematic diagram of a data processing apparatus according to a second embodiment of the present application.
Detailed Description
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. This invention may be embodied in many different forms than those herein set forth and should be readily appreciated by those skilled in the art without departing from the spirit and scope of the invention as defined by the appended claims.
In order to make the technical solutions of the present application better understood, a detailed description is first given of a specific application scenario embodiment of the present application.
The method can be applied to an automatic container scene, in the application scene, a depth camera based on a Time-of-flight (TOF) principle is installed on the ceiling of an aisle, the depth camera shoots from top to bottom, a laser radar in the depth camera continuously sends light pulses, and then a sensor in the camera receives light returned by an object to generate an image. When the height data of a certain shelf separated from the ground needs to be measured, firstly, obtaining the coordinate information of the shelf separated in a depth camera coordinate system; then, generating image data containing a shelf area and a ground area; then, cutting image data acquired by a depth camera to obtain an image data set for plane fitting; then, fitting the image data set according to a plane to obtain normal vector data of the plane; and finally, obtaining correction data of the height of the shelf from the ground according to the normal vector data of the plane and the coordinate information. According to the embodiment of the application, the data of the height of the original measuring target from the ground output by the ToF camera is corrected by depending on the measured value of the ToF camera from the ground, so that the real height value of the measuring target from the ground is obtained, the attitude error of the camera is reduced on the premise of not introducing an additional sensor (such as an accelerometer) and not increasing the construction difficulty, and then the coordinate value of the measuring target is converted into the height value from the ground, so that the input requirement of a target screening algorithm is met.
A first embodiment of the present application provides a data processing method, which is described below with reference to fig. 1 to 6.
As shown in fig. 1, in step S101, coordinate information of the measurement target in the depth camera coordinate system is obtained.
The depth camera may refer to a depth camera based on the ToF principle. ToF (Time-of-flight), one of the schemes for 3D imaging, obtains the object distance by detecting the Time of flight (round trip) of a light pulse by continuously transmitting the light pulse to an object and then receiving the light returned from the object with a sensor.
The measurement target may refer to an object for which data of height from the ground is to be measured by the depth camera. For example, in an automated container scenario, human and cargo separation may be targeted for measurement.
The coordinate information may refer to coordinates expressed in x, y, z.
The coordinate information of the measurement target in the depth camera coordinate system may refer to coordinate information of a target point corresponding to the measurement target in the depth camera coordinate system. The target point may refer to a central point of the measurement target, or may refer to at least one interest point of the measurement target. For example, if the measurement target is a bottle of water, the coordinate information of the measurement target in the depth camera coordinate system may refer to the coordinate information of the center point of the bottle of water in the depth camera coordinate system; if the measuring target is a person, the coordinate information of the measuring target in the depth camera coordinate system can refer to the coordinate information of the highest point of the person and also refer to the coordinate information of a contact point with a goods when the goods are taken by hands.
The obtaining of the coordinate information of the measurement target in the depth camera coordinate system includes:
obtaining a distance between an optical center of a depth camera and the measurement target;
and obtaining the coordinate information of the measuring target in a depth camera coordinate system according to the distance.
The process of obtaining coordinate information of the measurement target in the depth camera coordinate system is described in detail below.
In an ideal construction and installation process, the data coordinate axis of the ToF camera is parallel to each axis of a ground coordinate system without rotation. As shown in fig. 2.
As shown in fig. 3, according to the imaging principle of the ToF camera, the coordinates of the target point in the depth camera coordinate system can be obtained from the measured distance between the optical center and the target point as follows:
Figure BDA0003035060150000061
Figure BDA0003035060150000071
Figure BDA0003035060150000072
wherein f is x Is the focal length in the X direction, d is the distance from the optical center to the target point; u and v are coordinates of the target point in a camera UV coordinate system; u. of 0 ,v 0 Is the coordinate of the optical center in the depth camera UV coordinate system.
Since the coordinates are the original coordinates obtained in the case where the pose of the depth camera is not ideal, the data can be corrected using a subsequent step in order to reduce the error caused by the pose of the camera.
As shown in fig. 1, in step S102, the image data acquired by the depth camera is clipped according to the region information, and an image data set for performing plane fitting is obtained.
The image data acquired by the depth camera may refer to an image including a ground area and a shelf area. For example, FIG. 4 is a top view of a floor area and a shelf area. The image data acquired by the depth camera may be RGB image data or depth image data.
The clipping the image data acquired by the depth camera according to the region information to obtain an image data set for performing plane fitting includes:
and cutting image data acquired by the depth camera according to the UV area coordinate information or the distance data to obtain an image data set for plane fitting.
The clipping is performed on the image data acquired by the depth camera according to the UV area coordinate information to obtain an image data set for performing plane fitting, and the clipping is performed on the depth image data acquired by the depth camera according to the depth value (d in (u, v, d)) of the UV area coordinate information to obtain the image data set for performing plane fitting. The camera depth image data adopts UV area coordinates (u, v, d) as coordinates of pixel points, and can be cut according to a depth value d of a ground area distance depth camera. If the maximum value of the depth value of the ground area distance depth camera is d1, an area surrounded by pixel points whose distance from d1 is within a first distance threshold may be used as the ground area, and the first distance threshold may be set according to actual conditions, for example, 10cm.
For example, if the maximum value of the depth value of the ground area from the depth camera is 3.5m, an area surrounded by pixel points with d of 3.4-3.6 in the coordinates in the camera depth image may be used as the ground area, and may be cropped out as the image data set for plane fitting.
The image data obtained by the depth camera is cut according to the distance data to obtain an image data set for plane fitting, and the image data set for plane fitting can be obtained by cutting the depth image data obtained by the depth camera according to the distance information from the ground area to the depth camera.
For example, if the ground area is between 3 meters and 3.5m from the depth camera, an area with distance information between 3 meters and 3.5 meters may be cropped from the depth image data as an image data set for plane fitting.
In addition to cropping the depth image data, the RGB image data obtained by the depth camera may be cropped, and the RGB image data corresponding to the ground area is cropped from the RGB image data as an image data set for performing plane fitting.
In order to remove the influence of noise, the clipped original image data set may be subjected to noise processing to generate a noise-removed image data set.
The clipping the image data acquired by the depth camera according to the region information to obtain an image data set for performing plane fitting includes:
cutting image data acquired by the depth camera according to the region information to obtain an original image data set for plane fitting;
and carrying out noise processing on the original image data set by adopting a neighbor algorithm to obtain a processed image data set.
As shown in fig. 1, in step S103, the image data set is fitted to a plane to obtain normal vector data of the plane.
Fig. 5 is a schematic diagram of fitting the image data set obtained by cropping to a plane to obtain normal vector data of the plane. And fitting the image data set obtained by cutting according to a plane to obtain normal vector data (x 0, y0, z 0) of the plane.
It should be noted that, when the original image data set is subjected to noise processing by using a neighbor algorithm to obtain a processed image data set, the image data set is fitted according to a plane to obtain normal vector data of the plane, which means that the processed image data set is fitted according to the plane to obtain normal vector data of the plane.
The fitting the image data set according to a plane to obtain normal vector data of the plane includes:
obtaining point cloud data of a designated plane in the image dataset;
fitting the point cloud data according to a least square method to obtain a plane equation of the designated plane;
and obtaining normal vector data of the specified plane according to the plane equation.
The designated plane may refer to the ground, or to another plane parallel to the ground.
The Point Cloud data (Point Cloud) refers to data obtained in a 3D imaging sensor, typically in (X, Y, Z) coordinates.
As shown in fig. 1, in step S104, correction data of the height of the measurement target from the ground is obtained according to the normal vector data of the plane and the coordinate information.
The obtaining of the correction data of the height of the measurement target from the ground according to the normal vector data of the plane and the coordinate information includes:
obtaining original data of the height of the measuring target from the ground according to the coordinate information;
obtaining trigonometric function data of an offset angle between a data coordinate axis of the depth camera and a coordinate axis of a ground coordinate system according to the normal vector data;
and obtaining correction data of the height of the measuring target from the ground according to the original data and the trigonometric function data.
The offset angle between the data coordinate axis of the depth camera and the coordinate axis of the ground coordinate system comprises at least one of the following angles:
an offset angle of the depth camera about an X-axis of a ground coordinate system;
offset angle of the depth camera about the Y-axis of the ground coordinate system.
As shown in fig. 6, the offset angle of the depth camera about the X-axis of the ground coordinate system is β, and the offset angle of the depth camera about the Y-axis of the ground coordinate system is γ.
The obtaining of the correction data of the height of the measurement target from the ground according to the original data and the trigonometric function data includes:
obtaining a conversion relation between a corrected value of the height of the measuring target from the ground and the original data according to the original data and the trigonometric function data;
and obtaining correction data of the height of the measuring target from the ground according to the conversion relation.
The specific process of obtaining the correction data of the height of the measurement target from the ground according to the normal vector data of the plane and the coordinate information of the measurement target in the depth camera coordinate system is described below.
Firstly, obtaining the original data of the height of a measuring target from the ground according to the coordinate information of the measuring target in a depth camera coordinate system:
Z′=-sinβ·X+cosβsinγ+cosβcosγ·Z
substituting the values of the X and Y coordinates to obtain an equation only related to the original Z value:
Figure BDA0003035060150000091
then, from the normal vector data (x 0, y0, z 0), trigonometric function data of the offset angle between the coordinate axis of the data of the depth camera and the coordinate axis of the ground coordinate system are obtained:
Figure BDA0003035060150000101
Figure BDA0003035060150000102
Figure BDA0003035060150000103
Figure BDA0003035060150000104
finally, according to the original data and the trigonometric function data, obtaining a conversion relation between a correction value of the height of the measuring target from the ground and the original data; and obtaining correction data of the height of the measuring target from the ground according to the conversion relation:
Figure BDA0003035060150000105
therefore, introduction of the first embodiment of the present application is completed, and the data processing method provided by the first embodiment of the present application corrects the original data of the measurement target, which is output by the ToF camera, of the height from the ground by means of the measured value of the ToF camera to the ground, so as to obtain the real height value of the measurement target from the ground, and on the premise that no additional sensor (such as an accelerometer) is introduced and the construction difficulty is not increased, the attitude error of the camera is reduced, and then the coordinate value of the measurement target is converted into the height value from the ground, so that the input requirement of the target screening algorithm is met, and the accuracy of identifying new retail goods-taking behaviors is improved.
Corresponding to the data processing method provided in the first embodiment of the present application, a second embodiment of the present application provides a data processing apparatus.
As shown in fig. 7, the data processing apparatus includes:
a coordinate information obtaining unit 701 for obtaining coordinate information of the measurement target in a depth camera coordinate system;
an image data set obtaining unit 702, configured to crop image data obtained by the depth camera according to the region information, and obtain an image data set for performing plane fitting;
a normal vector data obtaining unit 703, configured to fit the image data set according to a plane, so as to obtain normal vector data of the plane;
a corrected data obtaining unit 704, configured to obtain corrected data of the height of the measurement target from the ground according to the normal vector data of the plane and the coordinate information.
As an embodiment, the image dataset obtaining unit is specifically configured to:
and cutting image data acquired by the depth camera according to the UV area coordinate information or the distance data to obtain an image data set for plane fitting.
As an embodiment, the image dataset obtaining unit is specifically configured to:
cutting image data acquired by a depth camera according to the region information to obtain an original image data set for plane fitting;
noise processing is carried out on the original image data set by adopting a neighbor algorithm to obtain a processed image data set;
the fitting the image data set according to a plane to obtain normal vector data of the plane includes:
and fitting the processed image data set according to a plane to obtain normal vector data of the plane.
As an embodiment, the coordinate information obtaining unit is specifically configured to:
obtaining a distance between an optical center of a depth camera and the measurement target;
and obtaining the coordinate information of the measuring target in a depth camera coordinate system according to the distance.
As an embodiment, the normal vector data obtaining unit is specifically configured to:
obtaining point cloud data of a designated plane in the image dataset;
fitting the point cloud data according to a least square method to obtain a plane equation of the designated plane;
and obtaining normal vector data of the specified plane according to the plane equation.
As an embodiment, the correction data obtaining unit is specifically configured to:
obtaining original data of the height of the measuring target from the ground according to the coordinate information;
obtaining trigonometric function data of an offset angle between a data coordinate axis of the depth camera and a coordinate axis of a ground coordinate system according to the normal vector data;
and obtaining correction data of the height of the measuring target from the ground according to the original data and the trigonometric function data.
As an embodiment, the offset angle between the data coordinate axis of the depth camera and the coordinate axis of the ground coordinate system includes at least one of the following angles:
an offset angle of the depth camera about an X-axis of a ground coordinate system;
offset angle of the depth camera about the Y-axis of the ground coordinate system.
As an embodiment, the correction data obtaining unit is specifically configured to:
obtaining a conversion relation between a corrected value of the height of the measuring target from the ground and the original data according to the original data and the trigonometric function data;
and obtaining correction data of the height of the measuring target from the ground according to the conversion relation.
It should be noted that, for the detailed description of the data processing apparatus provided in the second embodiment of the present application, reference may be made to the related description of the first embodiment of the present application, and details are not repeated here.
A third embodiment of the present application provides an electronic device corresponding to the data processing method provided in the first embodiment of the present application.
The electronic device includes:
a processor; and
a memory for storing a program of a data processing method, the apparatus performing the following steps after being powered on and running the program of the data processing method by the processor:
obtaining coordinate information of a measurement target in a depth camera coordinate system;
cutting image data acquired by the depth camera according to the region information to obtain an image data set for plane fitting;
fitting the image data set according to a plane to obtain normal vector data of the plane;
and acquiring correction data of the height of the measuring target from the ground according to the normal vector data of the plane and the coordinate information.
As an embodiment, the cropping the image data acquired by the depth camera according to the region information to obtain an image data set for plane fitting includes:
and cutting the image data acquired by the depth camera according to the UV area coordinate information or the distance data to obtain an image data set for plane fitting.
As an embodiment, the cropping the image data acquired by the depth camera according to the region information to obtain an image data set for plane fitting includes:
cutting image data acquired by the depth camera according to the region information to obtain an original image data set for plane fitting;
noise processing is carried out on the original image data set by adopting a neighbor algorithm to obtain a processed image data set;
the fitting the image data set according to a plane to obtain normal vector data of the plane includes:
and fitting the processed image data set according to a plane to obtain normal vector data of the plane.
As an embodiment, the obtaining coordinate information of the measurement target in the depth camera coordinate system includes:
obtaining a distance between an optical center of a depth camera and the measurement target;
and obtaining the coordinate information of the measuring target in a depth camera coordinate system according to the distance.
As an embodiment, the fitting the image data set to a plane to obtain normal vector data of the plane includes:
acquiring point cloud data of a designated plane in the image dataset;
fitting the point cloud data according to a least square method to obtain a plane equation of the designated plane;
and obtaining normal vector data of the specified plane according to the plane equation.
As an embodiment, the obtaining of the correction data of the height of the measurement target from the ground according to the normal vector data of the plane and the coordinate information includes:
obtaining original data of the height of the measuring target from the ground according to the coordinate information;
obtaining trigonometric function data of an offset angle between a data coordinate axis of the depth camera and a coordinate axis of a ground coordinate system according to the normal vector data;
and acquiring correction data of the height of the measuring target from the ground according to the original data and the trigonometric function data.
As an embodiment, the offset angle between the data coordinate axis of the depth camera and the coordinate axis of the ground coordinate system comprises at least one of:
an offset angle of the depth camera about an X-axis of a ground coordinate system;
offset angle of the depth camera about the Y-axis of the ground coordinate system.
As an embodiment, the obtaining the corrected data of the height of the measurement target from the ground according to the raw data and the trigonometric function data includes:
obtaining a conversion relation between a corrected value of the height of the measuring target from the ground and the original data according to the original data and the trigonometric function data;
and obtaining correction data of the height of the measuring target from the ground according to the conversion relation.
It should be noted that, for the detailed description of the electronic device provided in the third embodiment of the present application, reference may be made to the related description of the first embodiment of the present application, and details are not repeated here.
In correspondence with the data processing method provided in the first embodiment of the present application, a fourth embodiment of the present application provides a storage device storing a program of the data processing method, the program being executed by a processor to perform the steps of:
obtaining coordinate information of a measuring target in a depth camera coordinate system;
cutting image data acquired by the depth camera according to the region information to obtain an image data set for plane fitting;
fitting the image data set according to a plane to obtain normal vector data of the plane;
and acquiring correction data of the height of the measuring target from the ground according to the normal vector data of the plane and the coordinate information.
It should be noted that, for the detailed description of the storage device provided in the fourth embodiment of the present application, reference may be made to the related description of the first embodiment of the present application, and details are not described here again.
A fifth embodiment of the present application provides a data processing method, including:
obtaining coordinate information of a target commodity in a goods shelf in a depth camera coordinate system;
cutting image data acquired by the depth camera according to the region information to obtain an image data set for plane fitting;
fitting the image data set according to a plane to obtain normal vector data of the plane;
and acquiring correction data of the height of the target commodity from the ground according to the normal vector data of the plane and the coordinate information.
The data processing method provided in the fifth embodiment of the present application is a scenario embodiment of the first embodiment of the present application, and may be used in a scenario of a new retail line, an automatic container scenario, and the like. The data processing method provided by the fifth embodiment of the application includes the steps that coordinate information of a target commodity in a goods shelf in a depth camera coordinate system is obtained firstly; then, image data acquired by the depth camera is cut according to the region information to obtain an image data set for plane fitting; fitting the image data set according to a plane to obtain normal vector data of the plane; and finally, according to the normal vector data and the coordinate information of the plane, obtaining correction data of the height of the target commodity from the ground, obtaining the real height value of the target commodity from the ground, reducing the attitude error of the camera on the premise of not introducing an additional sensor (such as an accelerometer) and not increasing the construction difficulty, further converting the coordinate value of the target commodity into a ground height value, and meeting the input requirement of a target screening algorithm.
A sixth embodiment of the present application provides an item inventory method, comprising:
acquiring coordinate information of a target commodity on a goods shelf in a depth camera coordinate system;
cutting image data acquired by the depth camera according to the region information to obtain an image data set for plane fitting;
fitting the image data set according to a plane to obtain normal vector data of the plane;
obtaining correction data of the height of the target commodity from the ground according to the normal vector data of the plane and the coordinate information;
counting the types and the quantity of the commodities on the goods shelf according to the correction data of the height of the target commodity from the ground to obtain the actual inventory information of the commodities on the goods shelf;
and comparing the actual inventory information with recorded inventory information in a database to obtain inventory information of the commodities.
A seventh embodiment of the present application provides a method for adjusting goods placement, including:
acquiring coordinate information of a target commodity taken by a client in a depth camera coordinate system;
cutting image data acquired by the depth camera according to the region information to obtain an image data set for plane fitting;
fitting the image data set according to a plane to obtain normal vector data of the plane;
according to the normal vector data of the plane and the coordinate information, correction data of the height of the target commodity from the ground are obtained;
counting the commodity taking and placing frequency of a customer according to the correction data of the height of the target commodity from the ground;
and adjusting the placement of the commodities according to the taking and placing frequency.
An eighth embodiment of the present application provides a data processing method, which is similar to the first embodiment, and therefore only briefly described herein, please refer to the related contents of the first embodiment for detailed information. Firstly, an application program or an applet obtains coordinate information of a measurement target in a depth camera coordinate system according to instruction information input by a user, and displays the coordinate information in a graphical user interface. The graphical user interface in this embodiment may be a graphical user interface of an application program, or a graphical user interface in an applet.
And then, cutting the image data acquired by the depth camera according to the region information to obtain an image data set for plane fitting, and displaying the image data set in a graphical user interface. It should be noted that, in this step, the user may also perform interaction, the user may input the cropping instruction information on the graphical user interface, and the application program or the applet crops the image data acquired by the depth camera according to the region information according to the cropping instruction information input by the user.
And then, the application program or the small program fits the image data set according to the plane to obtain normal vector data of the plane, and the normal vector data is displayed in a graphical user interface. It should be noted that, in this step, the user may also perform interaction, the user may input fitting indication information on the graphical user interface, and the application program or the applet obtains normal vector data of the plane according to the fitting indication information input by the user.
And finally, the application program or the small program obtains correction data of the height of the measuring target from the ground according to the normal vector data of the plane and the coordinate information, and displays the correction data in a graphical user interface.
Although the present application has been described with reference to the preferred embodiments, it is not intended to limit the present application, and those skilled in the art can make variations and modifications without departing from the spirit and scope of the present application, therefore, the scope of the present application should be determined by the claims that follow.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both permanent and non-permanent, removable and non-removable media, may implement the information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Disks (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include non-transitory computer readable media (transient media), such as modulated data signals and carrier waves.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

Claims (10)

1. A data processing method, comprising:
obtaining coordinate information of a measuring target in a depth camera coordinate system;
cutting image data acquired by the depth camera according to the region information to obtain an image data set for plane fitting;
fitting the image data set according to a plane to obtain normal vector data of the plane;
and obtaining correction data of the height of the measuring target from the ground according to the normal vector data of the plane and the coordinate information.
2. The method of claim 1, wherein the cropping image data acquired by the depth camera according to the region information to obtain an image data set for plane fitting comprises:
and cutting image data acquired by the depth camera according to the UV area coordinate information or the distance data to obtain an image data set for plane fitting.
3. The method of claim 1, wherein the cropping the image data acquired by the depth camera according to the region information to obtain an image data set for plane fitting comprises:
cutting image data acquired by the depth camera according to the region information to obtain an original image data set for plane fitting;
carrying out noise processing on the original image data set by adopting a neighbor algorithm to obtain a processed image data set;
the fitting the image data set according to a plane to obtain normal vector data of the plane includes:
and fitting the processed image data set according to a plane to obtain normal vector data of the plane.
4. The method of claim 1, wherein obtaining coordinate information of the measurement target in a depth camera coordinate system comprises:
obtaining a distance between an optical center of a depth camera and the measurement target;
and obtaining the coordinate information of the measuring target in a depth camera coordinate system according to the distance.
5. The method of claim 1, wherein fitting the image dataset to a plane to obtain normal vector data for the plane comprises:
acquiring point cloud data of a designated plane in the image dataset;
fitting the point cloud data according to a least square method to obtain a plane equation of the designated plane;
and obtaining normal vector data of the designated plane according to the plane equation.
6. The method of claim 1, wherein obtaining the corrected height of the measurement target from the ground based on the normal vector data of the plane and the coordinate information comprises:
obtaining original data of the height of the measuring target from the ground according to the coordinate information;
obtaining trigonometric function data of an offset angle between a data coordinate axis of the depth camera and a coordinate axis of a ground coordinate system according to the normal vector data;
and acquiring correction data of the height of the measuring target from the ground according to the original data and the trigonometric function data.
7. The method of claim 6, wherein the offset angle between the data coordinate axis of the depth camera and the coordinate axis of the ground coordinate system comprises at least one of:
an offset angle of the depth camera about an X-axis of a ground coordinate system;
offset angle of the depth camera about the Y-axis of the ground coordinate system.
8. The method of claim 6, wherein obtaining the corrected height of the measurement target from the ground based on the raw data and the trigonometric function data comprises:
obtaining a conversion relation between a corrected value of the height of the measuring target from the ground and the original data according to the original data and the trigonometric function data;
and obtaining correction data of the height of the measuring target from the ground according to the conversion relation.
9. A data processing apparatus, characterized by comprising:
a coordinate information obtaining unit for obtaining coordinate information of the measurement target in a depth camera coordinate system;
an image data set obtaining unit, configured to crop image data obtained by the depth camera according to the region information, and obtain an image data set for performing plane fitting;
a normal vector data obtaining unit, configured to fit the image data set according to a plane to obtain normal vector data of the plane;
and the correction data obtaining unit is used for obtaining correction data of the height of the measuring target from the ground according to the normal vector data of the plane and the coordinate information.
10. A data processing method, comprising:
obtaining coordinate information of a measurement target in a depth camera coordinate system, and displaying the coordinate information in a graphical user interface;
cutting image data acquired by a depth camera according to the region information to obtain an image data set for plane fitting, and displaying the image data set in a graphical user interface;
fitting the image data set according to a plane to obtain normal vector data of the plane, and displaying the normal vector data in a graphical user interface;
and acquiring correction data of the height of the measuring target from the ground according to the normal vector data of the plane and the coordinate information, and displaying the correction data in a graphical user interface.
CN202110440989.8A 2021-04-23 2021-04-23 Data processing method and device Pending CN115239803A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110440989.8A CN115239803A (en) 2021-04-23 2021-04-23 Data processing method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110440989.8A CN115239803A (en) 2021-04-23 2021-04-23 Data processing method and device

Publications (1)

Publication Number Publication Date
CN115239803A true CN115239803A (en) 2022-10-25

Family

ID=83666199

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110440989.8A Pending CN115239803A (en) 2021-04-23 2021-04-23 Data processing method and device

Country Status (1)

Country Link
CN (1) CN115239803A (en)

Similar Documents

Publication Publication Date Title
US10769806B2 (en) Dimensional acquisition of packages
EP3491332B1 (en) Reflectivity map estimate from dot based structured light systems
CN109752003B (en) Robot vision inertia point-line characteristic positioning method and device
US9787960B2 (en) Image processing apparatus, image processing system, image processing method, and computer program
EP3620823A1 (en) Method and device for detecting precision of internal parameter of laser radar
CN112384891B (en) Method and system for point cloud coloring
US8300048B2 (en) Three-dimensional shape data recording/display method and device, and three-dimensional shape measuring method and device
CN108897836B (en) Method and device for robot to map based on semantics
US20170039727A1 (en) Methods and Systems for Detecting Moving Objects in a Sequence of Image Frames Produced by Sensors with Inconsistent Gain, Offset, and Dead Pixels
KR101918168B1 (en) Method for performing 3D measurement and Apparatus thereof
US20220309761A1 (en) Target detection method, device, terminal device, and medium
CN109118533B (en) Depth information processing method, device and equipment
US20150149105A1 (en) Accuracy compensation system, method, and device
US20210004978A1 (en) Method for acquiring depth information of target object and movable platform
US10097811B2 (en) Multi-part corresponder for multiple cameras
CN115685249A (en) Obstacle detection method and device, electronic equipment and storage medium
CN113610702B (en) Picture construction method and device, electronic equipment and storage medium
CN112666550B (en) Moving object detection method and device, fusion processing unit and medium
US20210027529A1 (en) Three-dimensional model processing method and three-dimensional model processing apparatus
US10223803B2 (en) Method for characterising a scene by computing 3D orientation
CN115239803A (en) Data processing method and device
US20110280473A1 (en) Rotation estimation device, rotation estimation method, and record medium
CN113805157A (en) Height measuring method, device and equipment based on target
CN115439561B (en) Robot sensor calibration method, robot and storage medium
CN111985266B (en) Scale map determining method, device, equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
TA01 Transfer of patent application right
TA01 Transfer of patent application right

Effective date of registration: 20240302

Address after: # 03-06, Lai Zan Da Building 1, 51 Belarusian Road, Singapore

Applicant after: Alibaba Innovation Co.

Country or region after: Singapore

Address before: Room 01, 45th Floor, AXA Building, 8 Shanton Road, Singapore

Applicant before: Alibaba Singapore Holdings Ltd.

Country or region before: Singapore