CN111325795A

CN111325795A - Image processing method and device, storage medium and robot

Info

Publication number: CN111325795A
Application number: CN202010117760.6A
Authority: CN
Inventors: 周韬; 王旭新; 成慧
Original assignee: Shenzhen Sensetime Technology Co Ltd
Current assignee: Shenzhen Sensetime Technology Co Ltd
Priority date: 2020-02-25
Filing date: 2020-02-25
Publication date: 2020-06-23
Anticipated expiration: 2040-02-25
Also published as: CN111325795B

Abstract

The embodiment of the disclosure discloses an image processing method, an image processing device, a storage medium and a robot, wherein the image processing method comprises the steps of determining a plurality of grabbing surfaces and a plurality of grabbing points corresponding to the plurality of grabbing surfaces from a multi-dimensional image to be processed according to image data information of the multi-dimensional image to be processed, wherein the plurality of grabbing surfaces correspond to the plurality of grabbing points one to one; determining a plurality of grabbing parameters corresponding to a plurality of grabbing surfaces; evaluating the plurality of grabbing surfaces by using the plurality of grabbing parameters, and determining a target grabbing surface from the plurality of grabbing surfaces according to an evaluation result; taking a grabbing point corresponding to the target grabbing surface as a target grabbing point; and determining the position posture of the grabbing point corresponding to the target grabbing point according to the target grabbing point, and grabbing the target object corresponding to the target grabbing point from the multi-dimensional image to be processed according to the position posture of the grabbing point.

Description

Image processing method and device, storage medium and robot

Technical Field

The present disclosure relates to the field of computer vision, and in particular, to an image processing method, an image processing apparatus, a storage medium, and a robot.

Background

In recent years, the pose calculation of an object has very important application in the fields of robots, automation and machine vision, particularly in the field of computer vision.

In the prior art, the image processing device determines the pose of the target object according to the height information of the target capture surface of the target object, so that the accuracy of determining the pose of the target object by the image processing device is reduced.

Disclosure of Invention

The embodiment of the disclosure provides an image processing method and device, a storage medium and a robot.

The technical scheme of the disclosure is realized as follows:

the embodiment provides an image processing method, which comprises the following steps:

determining a plurality of grabbing surfaces and a plurality of grabbing points corresponding to the grabbing surfaces from the multi-dimensional image to be processed according to image data information of the multi-dimensional image to be processed, wherein the grabbing surfaces correspond to the grabbing points one by one;

determining a plurality of grabbing parameters corresponding to the plurality of grabbing surfaces;

evaluating the plurality of grabbing surfaces by using the plurality of grabbing parameters, and determining a target grabbing surface from the plurality of grabbing surfaces according to an evaluation result;

taking the grabbing point corresponding to the target grabbing surface as a target grabbing point;

and determining a grabbing point position corresponding to the target grabbing point according to the target grabbing point, and grabbing a target object corresponding to the target grabbing point from the multi-dimensional image to be processed according to the grabbing point position.

The image processing device evaluates the plurality of grabbing surfaces according to the plurality of grabbing parameter values, so that the target grabbing surface is determined, the target grabbing surface is not determined according to a single height parameter value, the accuracy of determining the target grabbing surface by the image processing device is improved, the grabbing point position of the target object is determined according to the target grabbing surface with high accuracy, and the accuracy of determining the position and the attitude of the target object by the image processing device is improved.

In the above scheme, the evaluating the plurality of grasping surfaces by using the plurality of grasping parameters, and determining a target grasping surface from the plurality of grasping surfaces according to an evaluation result includes:

evaluating each of the plurality of grasping surfaces by using the plurality of grasping parameters to obtain a plurality of grasping surface evaluation values corresponding to the plurality of grasping surfaces;

and determining a first grabbing face evaluation value with the highest evaluation value from the plurality of grabbing face evaluation values, and taking the grabbing face corresponding to the first grabbing face evaluation value as the target grabbing face.

The image processing device evaluates each grabbing surface according to the plurality of parameter values respectively, and determines the target grabbing surface from the plurality of grabbing surfaces according to the evaluation value, so that the accuracy of determining the target grabbing surface by the image processing device is improved.

In the above solution, the plurality of grabbing parameters includes at least one of:

the area parameter of the grabbing surface, the height parameter of the grabbing surface, the flatness parameter of the grabbing surface and the gradient parameter of the grabbing surface.

Specifically, the plurality of grabbing parameters are stated as area parameters of the grabbing surfaces, height parameters of the grabbing surfaces, flatness parameters of the grabbing surfaces or gradient parameters of the grabbing surfaces, and the image processing device can evaluate the plurality of grabbing surfaces by using the grabbing parameters to improve the accuracy of determining the target grabbing surface.

In the foregoing solution, the determining, according to image data information of a multidimensional image to be processed, a plurality of capturing surfaces and a plurality of capturing points corresponding to the plurality of capturing surfaces from the multidimensional image to be processed includes:

inputting image data information of the multidimensional image to be processed into a deep learning network model to obtain a plurality of pixel points and a plurality of central points corresponding to the pixel points, wherein the deep learning network model is obtained by training an initial deep learning network model by utilizing sample image data information of a sample multidimensional image, and the pixel points correspond to the central points one by one;

dividing the plurality of center points into a plurality of groups of center points;

determining a grabbing point corresponding to any group of central points according to any group of central points in the multiple groups of central points until the multiple grabbing points are determined from the multiple groups of central points;

and determining a grabbing surface corresponding to any group of central points according to any group of pixel points corresponding to any group of central points in the multiple groups of central points until the multiple grabbing surfaces are determined from the multiple groups of pixel points corresponding to the multiple groups of central points, wherein any group of central points corresponds to any group of pixel points one to one.

The image processing device determines a plurality of pixel points and a plurality of central points corresponding to the pixel points from the image data information of the multi-dimensional image to be processed by utilizing the deep learning network model, and determines a plurality of grabbing surfaces and a plurality of grabbing points according to the pixel points and the central points, so that the image processing device can deduce the grabbing points corresponding to the grabbing surfaces without marking and retraining collected data.

In the above scheme, the determining, according to any one group of center points in the plurality of groups of center points, one grabbing point corresponding to the any one group of center points until the plurality of grabbing points are determined from the plurality of groups of center points includes:

averaging a group of position data of any group of central points to obtain an average position data until a plurality of groups of average position data are obtained from a plurality of groups of position data information of the plurality of groups of central points;

and taking a plurality of points corresponding to the plurality of average position data as a plurality of grabbing points, wherein the plurality of average position data correspond to the plurality of grabbing points one by one.

The image processing device determines the positions of the plurality of grabbing points by averaging a group of position data of any group of central points, so that the accuracy of the position information of the plurality of grabbing points is improved.

In the foregoing solution, before determining, according to image data information of a multidimensional image to be processed, a plurality of capturing surfaces and a plurality of capturing points corresponding to the plurality of capturing surfaces from the multidimensional image to be processed, the method further includes:

acquiring original image data information of the multidimensional image to be processed;

and preprocessing the original image data information to obtain the image data information.

Data points of the original image data are unified, so that accuracy of matching of the sample image data and the image data is improved.

In the above scheme, the preprocessing the original image data information to obtain the image data information includes:

under the condition that the quantity of original data information in the original image data information does not meet a preset quantity value, regulating the quantity of the original data information into a preset quantity;

and dividing the data of the original data information adjusted to the preset number by a preset numerical value respectively to obtain the image data information.

The original data information is divided by the preset value so as to improve the convergence degree of the original data information during calculation and improve the accuracy of the calculation result.

In the above scheme, the image data information includes color channel data information and depth data information.

The method determines the grabbing point position of the target object according to the RGB information and the depth information of the multi-dimensional image to be processed so as to improve the accuracy of calculating the grabbing point position of the target object.

An embodiment of the present disclosure provides an image processing apparatus, including:

the device comprises a determining unit, a processing unit and a processing unit, wherein the determining unit is used for determining a plurality of grabbing surfaces and a plurality of grabbing points corresponding to the grabbing surfaces from a multi-dimensional image to be processed according to image data information of the multi-dimensional image to be processed, and the grabbing surfaces correspond to the grabbing points one by one; determining a plurality of grabbing parameters corresponding to the plurality of grabbing surfaces; taking a grabbing point corresponding to the target grabbing surface as a target grabbing point; according to the target grabbing point, a grabbing point position corresponding to the target grabbing point is determined, and a target object corresponding to the target grabbing point is grabbed from the RGBD image according to the grabbing point position;

and the evaluation unit is used for evaluating the plurality of grabbing surfaces by using the plurality of grabbing parameters and determining the target grabbing surface from the plurality of grabbing surfaces according to an evaluation result.

In the foregoing scheme, the evaluation unit is specifically configured to evaluate each of the multiple grasping surfaces by using multiple parameters of the multiple grasping parameters, respectively, to obtain multiple grasping surface evaluation values corresponding to the multiple grasping surfaces;

the determining unit is specifically configured to determine a first grasping surface evaluation value with a highest evaluation value from the plurality of grasping surface evaluation values, and use a grasping surface corresponding to the first grasping surface evaluation value as the target grasping surface.

In the above scheme, the determining unit is specifically configured to input image data information of the multidimensional image to be processed into a deep learning network model, to obtain a plurality of pixel points and a plurality of central points corresponding to the plurality of pixel points, where the deep learning network model is a model obtained by training an initial deep learning network model using sample image data information of a sample multidimensional image, and the plurality of pixel points and the plurality of central points are in one-to-one correspondence; dividing the plurality of center points into a plurality of groups of center points; determining a grabbing point corresponding to any group of central points according to any group of central points in the multiple groups of central points until the multiple grabbing points are determined from the multiple groups of central points; and determining a grabbing surface corresponding to any group of central points according to any group of pixel points corresponding to any group of central points in the multiple groups of central points until the multiple grabbing surfaces are determined from the multiple groups of pixel points corresponding to the multiple groups of central points, wherein any group of central points corresponds to any group of pixel points one to one.

In the foregoing solution, the determining unit is specifically configured to average a group of position data of any one group of central points to obtain an average position data, until a plurality of average position data are obtained from a plurality of groups of position data information of the plurality of groups of central points; and taking a plurality of points corresponding to the plurality of average position data as a plurality of grabbing points, wherein the plurality of average position data correspond to the plurality of grabbing points one by one.

In the above scheme, the apparatus further comprises an obtaining unit and a preprocessing unit;

the acquisition unit is used for acquiring original image data information of the multi-dimensional image to be processed;

the preprocessing unit is used for preprocessing the original image data information to obtain the image data information.

In the foregoing solution, the preprocessing unit is specifically configured to adjust the number of the original data information to a preset number when the number of the original data information in the original image data information does not satisfy a preset number value; and dividing the data of the original data information adjusted to the preset number by a preset numerical value respectively to obtain the image data information.

a memory storing an image processing program executable by the graphics processor, and a graphics processor through which the method described above is performed when the image processing program is executed.

The embodiment of the present disclosure provides a storage medium, on which a computer program is stored, and is applied to an image processing apparatus, wherein the computer program is used for implementing the method when being executed by a graphics processor.

The embodiment of the disclosure provides a robot, which comprises a mechanical arm and an image processing device, wherein the image processing device is used for executing the method, and the mechanical arm is used for grabbing a target object at a grabbing point position under the condition that the image processing device determines the grabbing point position of the target object.

The embodiment of the disclosure provides an image processing method, an image processing device, a storage medium and a robot, wherein the image processing method comprises the following steps: determining a plurality of grabbing surfaces and a plurality of grabbing points corresponding to the plurality of grabbing surfaces from the multi-dimensional image to be processed according to image data information of the multi-dimensional image to be processed, wherein the plurality of grabbing surfaces correspond to the plurality of grabbing points one to one; determining a plurality of grabbing parameters corresponding to a plurality of grabbing surfaces; evaluating the plurality of grabbing surfaces by using the plurality of grabbing parameters, and determining a target grabbing surface from the plurality of grabbing surfaces according to an evaluation result; taking a grabbing point corresponding to the target grabbing surface as a target grabbing point; and determining the position posture of the grabbing point corresponding to the target grabbing point according to the target grabbing point, and grabbing the target object corresponding to the target grabbing point from the multi-dimensional image to be processed according to the position posture of the grabbing point. By adopting the method, the image processing device firstly determines a plurality of grabbing parameter values corresponding to a plurality of grabbing surfaces, and then evaluates the plurality of grabbing surfaces according to the plurality of grabbing parameter values, so that the target grabbing surface is determined from the plurality of grabbing surfaces, the target grabbing surface is not determined according to a single height parameter value, the accuracy of determining the target grabbing surface by the image processing device is improved, the image processing device determines the grabbing point position of the target object according to the target grabbing surface with high accuracy, and the accuracy of determining the position and the attitude of the target object by the image processing device is improved.

Drawings

Fig. 1 is a flowchart of a first image processing method according to the present embodiment;

fig. 2 is a flowchart of an image processing method according to the present embodiment;

fig. 3 is a first schematic diagram illustrating a composition structure of an image processing apparatus according to the present embodiment;

fig. 4 is a schematic diagram of a second composition structure of the image processing apparatus according to the present embodiment.

Detailed Description

The technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings in the embodiments of the present disclosure.

An embodiment of the present disclosure provides an image processing method, and fig. 1 is a flowchart of the image processing method provided in the embodiment of the present disclosure, and as shown in fig. 1, the image processing method may include:

s101, according to image data information of the multi-dimensional image to be processed, a plurality of grabbing surfaces and a plurality of grabbing points corresponding to the grabbing surfaces are determined from the multi-dimensional image to be processed, wherein the grabbing surfaces correspond to the grabbing points one by one.

The image processing method provided by the embodiment of the disclosure is suitable for processing the image data information of the multi-dimensional image to be processed, and determining the scene of the capturing point position of the target object.

In the embodiment of the present disclosure, the image processing method is applied to an image processing apparatus, which may be integrated in a robot, so that the robot grasps a target object according to the position and posture of the grasping point of the target object. In some possible implementations, the image processing method may be implemented by a processor calling computer readable instructions stored in a memory.

It should be noted that the to-be-processed multidimensional image may be a to-be-processed three-dimensional image, a to-be-processed four-dimensional image, or a to-be-processed two-dimensional image, which may be determined specifically according to an actual situation, and this is not limited in the embodiment of the present disclosure.

In the embodiment of the present disclosure, when the image processing apparatus acquires the image data information of the multidimensional image to be processed, the image processing apparatus may determine, according to the image data information, the plurality of capturing surfaces and the plurality of capturing points corresponding to the plurality of capturing surfaces from the image to be processed.

It should be noted that, the image processing device may obtain the image data information of the to-be-processed multidimensional image through an image acquisition device such as a camera, and the image data information is read from the to-be-processed multidimensional image, the image processing device may also directly obtain the image data information of the to-be-processed multidimensional image at another device, and a specific manner for obtaining the image data information of the to-be-processed multidimensional image by the image processing device may be determined according to an actual situation, which is not limited in this embodiment of the disclosure.

It should be noted that, if the image Processing apparatus acquires the multidimensional image to be processed by using an image acquisition apparatus such as a camera and acquires the image Processing data in a manner of reading image data information from the multidimensional image to be processed, the image Processing apparatus may control the camera to acquire the multidimensional image to be processed by using a Central Processing Unit (CPU), and acquire the image data information according to the multidimensional image to be processed. When the image processing device acquires the image data information, the image processing device transmits the image data information to a Graphics Processing Unit (GPU), and the GPU processes the image data information to determine the position of the capture point of the target object.

In the embodiment of the present disclosure, the image processing apparatus performs parallel processing on the image data information of the to-be-processed multidimensional image by using the GPU, that is, the GPU simultaneously processes the image data information at all pixel points in the to-be-processed multidimensional image, which increases the speed of the image processing apparatus when processing the image data information, that is, increases the speed of the image processing apparatus when determining the grab point pose of the target object by using the image data information in the to-be-processed multidimensional image.

It should be noted that the plurality of grasping surfaces correspond to the plurality of grasping points one to one.

The plurality of grabbing surfaces are a plurality of surfaces of a target object to be grabbed in the multi-dimensional image to be processed, the plurality of grabbing points are central points of the plurality of grabbing surfaces, and one grabbing surface corresponds to one grabbing point.

For example, the target object in the multi-dimensional image to be processed may be two hexahedrons, the plurality of grabbing surfaces may be two front surfaces of the two hexahedrons, that is, the number of the plurality of graspable surfaces is 2, the first graspable surface is the front surface of the first hexahedron, the second graspable surface is the front surface of the second hexahedron, and the plurality of graspable points are respectively the center points of the two front surfaces, that is, the number of the plurality of graspable points is 2, the first graspable point is the center point of the front surface of the first hexahedron, and the second graspable point is the center point of the front surface of the second hexahedron.

Illustratively, the target object in the multi-dimensional image to be processed may be a tetrahedron, the plurality of grabbing surfaces may be two sides of the tetrahedron, that is, the number of the plurality of grabbing surfaces is 2, the first grabbing surface is a first side of the tetrahedron, the second grabbing surface is a side adjacent to the first side, that is, a second side, the plurality of grabbing points are respectively center points of the two sides, that is, the number of the plurality of grabbing points is 2, the first grabbing point is a center point of the first side, and the second grabbing point is a center point of the second side.

In an embodiment of the disclosure, the image data information comprises color channel data information and depth data information.

Illustratively, the image data information may be RGBD data information of the multi-dimensional image to be processed, wherein the color channel data information may be RGB data information of the multi-dimensional image to be processed, and the depth data information may be depth information of the multi-dimensional image to be processed. When the image data information is RGBD data of the multidimensional image to be processed, the image processing device can determine the grabbing point position of the target object according to RGB data of the multidimensional image to be processed and depth image data of the multidimensional image to be processed, and accuracy of the image processing device in calculating the grabbing point position of the target object is improved.

In the embodiment of the present disclosure, the mode of determining, by the image processing apparatus, the plurality of capturing surfaces and the plurality of capturing points corresponding to the plurality of capturing surfaces from the image to be processed according to the image data information of the multi-dimensional image to be processed may be: the image processing device inputs image data information of the multi-dimensional image to be processed into the deep learning network model to obtain a plurality of pixel points and a plurality of central points corresponding to the pixel points.

In the embodiment of the disclosure, the image processing device includes a deep learning network model, and when the image processing device acquires image data information of a multidimensional image to be processed, the image processing device inputs the image data information into the deep learning network model, and the deep learning network model determines a plurality of pixel points and a plurality of center points from the image data information.

In the embodiment of the disclosure, the image processing apparatus trains the initial deep learning network model by using the sample image data information, and adjusts the initial parameters of the initial deep learning network model, so that the initial deep learning network model after adjusting the initial parameters can classify and detect the sample image data information, learn the surface information of the sample object in the sample multidimensional image, determine the sample image point and the sample center point, and thereby obtain the deep learning network model. When the deep learning network model obtains the image data information of the multi-dimensional image to be processed, the deep learning network model can classify and detect the image data information, and a plurality of pixel points and a plurality of central points are determined from the image data information.

It should be noted that a plurality of pixel points correspond to a plurality of center points one to one, wherein one pixel point corresponds to one center point.

The sample image data information is data information for training the initial deep learning network model. The sample image data information may be information of multiple faces of a polyhedron, information of a ball, information of multiple faces of a toy, and information of multiple faces of other objects, and the specific sample image data information may be determined according to actual conditions, which is not limited in the embodiment of the present disclosure.

In the embodiment of the present disclosure, the information of the plurality of planes may be position information of the plurality of planes, may also be size information of the plurality of planes, and may also be information of a plurality of sample image points in the plurality of planes, which may be determined according to actual situations, and the embodiment of the present disclosure does not limit this.

In the embodiment of the present disclosure, before the image processing apparatus trains the initial deep learning network model, the image processing apparatus needs to acquire many sample image data information, such as: information on a plurality of faces of the polyhedron, information on a plurality of faces of the toy, information on a face of the ball, information on a face of the cup, and the like. When the image processing device acquires the sample image data information, the image processing device trains the initial deep learning network model by using the sample image data information and adjusts initial parameters of the initial deep learning network model, so that the initial deep learning network model after the initial parameters are adjusted can classify and detect the sample image data information, learn the surface information of a sample object in a sample multi-dimensional image, and determine a sample image point and a sample central point, thereby obtaining the deep learning network model. Therefore, when the image processing device inputs the image data information of the multidimensional image to be processed into the deep learning network model, the deep learning network model can determine pixel points on a plurality of capturing surfaces from the image data information of the multidimensional image to be processed.

It should be noted that the sample image point is a point on the sample image, and the sample center point is a center point of the sample image.

It can be understood that the model parameters in the deep learning network model are parameters obtained by learning the surface information of the object in the multi-dimensional image of the sample, when the deep learning network model obtains the image data information including the unknown object in the multi-dimensional image to be processed, the deep learning network model can directly determine the surface information of the unknown object from the image data information according to the model parameters, so as to determine a plurality of pixel points and a plurality of central points corresponding to the surface information, without labeling and retraining the image data information of the unknown object, thereby improving the generalization and practicability of the image processing device when processing the image data information.

In the embodiment of the present disclosure, after the image processing apparatus determines a plurality of pixel points and a plurality of center points corresponding to the plurality of pixel points from the image data information of the multidimensional image to be processed by using the deep learning network model, the image processing apparatus may divide the plurality of center points into a plurality of groups of center points.

In the embodiment of the present disclosure, the image processing device divides the plurality of central points to obtain a plurality of groups of central points, and may cluster the plurality of central points for the image processing device, thereby determining a plurality of groups of central points corresponding to the plurality of capturing surfaces. The image processing device may also divide the plurality of central points by using the deep learning network model to determine the plurality of groups of capture points, and a specific manner in which the image processing device divides the plurality of central points into the plurality of groups of central points may be determined according to an actual situation, which is not limited in the embodiment of the present disclosure.

It should be noted that the mode of clustering the plurality of central points by the image processing device may be a mean shift clustering algorithm, a hierarchical clustering algorithm, or a density clustering algorithm, and a specific clustering algorithm may be determined according to an actual situation, which is not limited in the embodiment of the present disclosure.

It should be noted that the plurality of grabbing surfaces correspond to the plurality of groups of center points one to one, wherein one grabbing surface corresponds to one group of grabbing points.

In this disclosure, after the image processing apparatus divides the plurality of central points into the plurality of groups of central points, the image processing apparatus may determine one capturing point corresponding to any one group of central points according to any one group of central points in the plurality of groups of central points until the plurality of capturing points are determined from the plurality of groups of central points.

In this disclosure, the image processing apparatus determines a manner of determining a capture point corresponding to any one group of center points according to any one group of center points in the plurality of groups of center points, may randomly determine a point from any one group of center points for the image processing apparatus, and use the point as a capture point, or randomly select a part of center points from any one group of center points for the image processing apparatus, and determine a capture point according to position information of the part of center points, the image processing apparatus may also determine a capture point according to position information of all points of any one group of center points, and the specific image processing apparatus determines a manner of determining a capture point according to one group of center points in the plurality of groups of center points, which is not limited in this disclosure.

In the embodiment of the present disclosure, when the image processing apparatus determines one grabbing point corresponding to any one group of central points from any one group of central points in the plurality of groups of central points according to a certain mode, the image processing apparatus may also determine the remaining grabbing points from other groups of central points in the plurality of groups of central points according to the certain mode, so as to obtain a plurality of grabbing points.

In this embodiment of the present disclosure, the process that the image processing apparatus determines one grabbing point corresponding to any one group of central points according to any one group of central points in the plurality of groups of central points until determining the plurality of grabbing points from the plurality of groups of central points may be: the image processing device averages a group of position data of any group of central points to obtain an average position data until the image processing device obtains a plurality of average position data from a plurality of groups of position data information of a plurality of groups of central points.

It should be noted that one average position data corresponds to one set of position data.

It should be noted that the group of position data may be a group of three-dimensional coordinate data of a central point, a group of three-dimensional pose data of a central point, or a group of other position data of a central point, which may be determined specifically according to an actual situation, and this is not limited in the embodiment of the present disclosure.

In the embodiment of the present disclosure, after the image processing apparatus obtains the plurality of average position data, the image processing apparatus may take a plurality of points corresponding to the plurality of average position data as the plurality of capture points.

It should be noted that a plurality of average position data correspond to a plurality of grab points one to one, where one average position data corresponds to one grab point.

In the embodiment of the present disclosure, after the image processing device determines the plurality of grabbing points from the plurality of sets of center points, the image processing device may determine one grabbing surface corresponding to any set of center points according to any set of pixel points corresponding to any set of center points in the plurality of sets of center points until determining the plurality of grabbing surfaces from the plurality of sets of pixel points corresponding to the plurality of sets of center points.

It should be noted that any group of center points corresponds to any group of pixel points, wherein a group of center points corresponds to a group of pixel points.

It should be noted that a plurality of groups of pixel points correspond to a plurality of capturing surfaces one to one, wherein one group of pixel points corresponds to one capturing surface.

In the embodiment of the present disclosure, before the image processing apparatus determines the plurality of capturing surfaces and the capturing points corresponding to the plurality of capturing surfaces from the multi-dimensional image to be processed according to the image data information of the multi-dimensional image to be processed, the image processing apparatus needs to first acquire original image data information of the multi-dimensional image to be processed.

It should be noted that, the manner in which the image processing apparatus acquires the original image data information of the multidimensional image to be processed may be that the image processing apparatus acquires the multidimensional image to be processed through an image acquisition apparatus such as a camera and reads the original image data information from the multidimensional image to be processed, or that the image processing apparatus directly acquires the original image data information of the multidimensional image to be processed from another apparatus, and the manner in which the image processing apparatus specifically acquires the original image data information of the multidimensional image to be processed may be determined according to actual circumstances, which is not limited in this embodiment of the disclosure.

It should be noted that the original image data information may be original RGBD data information.

In the embodiment of the present disclosure, when the image processing apparatus acquires original image data information of a multidimensional image to be processed, the image processing apparatus performs preprocessing on the original image data information to obtain image data information.

It can be understood that the image processing device pre-processes the original image data information, so that the accuracy of the pre-processed original image data information matched with the sample image data information is improved, and the accuracy of the image processing device in processing the image data information of the multi-dimensional image to be processed is improved.

The mode of preprocessing the original image data by the image processing apparatus may be to remove noise in the original image data information, to amplify the original image data information, to change the number of pieces of original image data information, or to process other original image data information, which may be determined according to actual circumstances, and is not limited in the embodiment of the present disclosure.

In the embodiment of the present disclosure, the image processing apparatus performs preprocessing on the original image data information, and a process of obtaining the image data information may be: when the number of pieces of original data information in the pieces of original image data information does not satisfy a preset number value, the image processing apparatus adjusts the number of pieces of original data information to the preset number.

It can be understood that the preset number is a preset data information amount when the image processing apparatus processes the image data information, and the image processing apparatus adjusts the data amount of the original data information to the preset number, so that the accuracy when the adjusted original data information is matched with the sample image data is improved, and the accuracy when the image processing apparatus processes the image data information of the multi-dimensional image to be processed is improved.

In the embodiment of the disclosure, when the image processing apparatus obtains the original image data information of the multidimensional image to be processed, the image processing apparatus compares the number of the original image data information with a preset number value, and when the number of the original image data information in the original image data information does not satisfy the preset number value, the image processing apparatus adjusts the number of the original image data information to the preset number.

It should be noted that the number of the original data information does not satisfy the preset number value, and may be that the number of the original data information is greater than or less than the preset number value, and the number of the original data information satisfies the preset number value, and may be that the number of the original data information is equal to the preset number value.

It should be noted that the preset quantity value is a quantity value of original image data information preset in the image processing apparatus, for example, the preset quantity value may be 65536, the number of points of the abscissa of the corresponding to-be-processed multi-dimensional image may be 256, and the number of points of the ordinate of the corresponding to-be-processed multi-dimensional image may be 256.

Illustratively, when the abscissa of the multidimensional image to be processed acquired by the image processing device has 1024 points, and the ordinate of the multidimensional image to be processed has 1024 points, the image processing device compares the data amount of the original image data information of the multidimensional image to be processed with a preset quantity value, that is, 1024 points are multiplied by 1024 points, 1048576 points in total are compared with 65536 points, and since 1048576 points of the data amount of the original image data information is greater than 65536 points, the quantity of the original data information does not satisfy the preset quantity value.

In the embodiment of the present disclosure, the image processing apparatus may adjust the number of the raw data information to the preset number by increasing the raw data information when the image processing apparatus determines that the number of the raw data information is smaller than the preset number, and decreasing the raw data information when the image processing apparatus determines that the number of the raw data information is larger than the preset number.

It should be noted that, the manner in which the image processing apparatus adds the original data information may be to add one or more pieces of data information between two adjacent pieces of original data information of the original data information, a value of the added one or more pieces of data information may be determined according to the two adjacent pieces of original data information, the manner in which the image processing apparatus adds the original data information may be to add one or more pieces of data information between two adjacent pieces of original data information of the original data information, a value of the one or more pieces of data information may be determined according to all pieces of original data information, the image processing apparatus may also add the original data information in other manners, a specific manner in which the image processing apparatus adds the original data information may be determined according to actual situations, which is not limited in this embodiment of the disclosure.

It should be noted that, the way of reducing the original data information by the image processing apparatus may be to delete data at the position of an odd-numbered point of the original data information for the image processing apparatus, to delete data at the position of an even-numbered point of the original data information for the image processing apparatus, or to reduce the original data information for another way, which may be determined according to actual situations, and this is not limited in the embodiment of the present disclosure.

In the embodiment of the present disclosure, after the image processing apparatus adjusts the number of the original data information to the preset number, the image processing apparatus divides the data of the original data information adjusted to the preset number by the preset value, respectively, to obtain the image data information.

It can be understood that the image processing apparatus divides the original data information by the preset value, so that the original data information can be converged quickly, the convergence degree of the original data information during calculation is improved, and the accuracy of the image processing apparatus in processing the image data information of the multi-dimensional image to be processed is improved.

It should be noted that the preset value is a value preset in the image processing apparatus.

In the embodiment of the present disclosure, the number of the preset values in the image processing apparatus may be multiple, and when the value ranges of the original image data information are different, the preset values may correspond to different values, and the value of the specific preset value may be determined according to an actual situation, which is not limited in the embodiment of the present disclosure.

S102, determining a plurality of grabbing parameters corresponding to the plurality of grabbing surfaces.

In the embodiment of the present disclosure, after the image processing apparatus determines the plurality of capturing surfaces from the multi-dimensional image to be processed, the image processing apparatus may determine the plurality of capturing parameters corresponding to each of the plurality of capturing surfaces, respectively.

It should be noted that the plurality of grabbing parameters include at least one of the following: the area parameter of the grabbing surface, the height parameter of the grabbing surface, the flatness parameter of the grabbing surface and the gradient parameter of the grabbing surface.

It should be noted that the plurality of capture parameters may be determined by the image processing apparatus based on the plurality of capture planes.

In the embodiment of the present disclosure, the image processing apparatus assigns values to a plurality of capturing surfaces corresponding to a single capturing parameter, and determines a plurality of capturing parameter values corresponding to the single capturing parameter for the plurality of capturing surfaces until the image processing apparatus assigns values to the plurality of capturing surfaces corresponding to the plurality of capturing parameters, and determines a plurality of capturing parameters corresponding to the plurality of capturing surfaces.

In the embodiment of the disclosure, a first group of grabbing parameter values corresponding to different area parameters of a grabbing surface is set in an image processing device, a second group of grabbing parameter values corresponding to different height parameters of the grabbing surface, a third group of grabbing parameter values corresponding to different flatness parameters of the grabbing surface and a fourth group of grabbing parameter values corresponding to different inclination parameters of the grabbing surface, when the image processing device obtains one grabbing surface of a plurality of grabbing surfaces, the image processing device determines a first grabbing parameter value corresponding to the grabbing area parameter from a first group of preset parameter values according to the grabbing area parameter of the grabbing surface; according to the height parameter of the grabbing surface, determining a second grabbing parameter value corresponding to the height parameter of the grabbing surface from a second group of grabbing parameter values; determining a third grabbing parameter value corresponding to the flatness parameter of the grabbing surface from a third group of grabbing parameter values according to the flatness parameter of the grabbing surface; and determining a fourth grabbing parameter value corresponding to the inclination parameter of the grabbing surface from the fourth group of grabbing parameter values according to the inclination parameter of the grabbing surface until the image processing device determines that each grabbing surface of the plurality of grabbing surfaces corresponds to four grabbing parameter values.

S103, evaluating the plurality of grabbing surfaces by using the plurality of grabbing parameters, and determining a target grabbing surface from the plurality of grabbing surfaces according to an evaluation result.

In the embodiment of the present disclosure, after the image processing apparatus determines a plurality of capturing parameters corresponding to the plurality of capturing surfaces, the image processing apparatus may evaluate the plurality of capturing surfaces by using the plurality of capturing parameters, and determine the target capturing surface from the plurality of capturing surfaces.

In the embodiment of the present disclosure, the image processing apparatus evaluates the plurality of capturing surfaces by using the plurality of capturing parameters, and the process of determining the target capturing surface from the plurality of capturing surfaces may be: the image processing device evaluates each of the plurality of grasping surfaces by using the plurality of grasping parameters to obtain a plurality of grasping surface evaluation values corresponding to the plurality of grasping surfaces.

In this embodiment of the disclosure, the image processing apparatus may obtain each parameter value corresponding to each grasping surface according to an area parameter of the grasping surface corresponding to each grasping surface, a height parameter of the grasping surface, a flatness parameter of the grasping surface, and an inclination parameter of the grasping surface until obtaining a plurality of first parameter values corresponding to a plurality of grasping surfaces, and obtain a plurality of grasping surface evaluation values according to the plurality of first parameter values and a plurality of grasping probability values of the plurality of grasping surfaces.

It should be further noted that the plurality of grabbing probability values are probability values of a plurality of grabbing planes as the target grabbing plane.

For example, the image processing apparatus may determine a plurality of grasping face evaluation values by formula (1).

If a group of grabbing parameters corresponding to one grabbing surface are the area parameter of the grabbing surface and the height parameter of the grabbing surface, P (c)_i| x) is the evaluation value of each of the plurality of gripping surfaces under the condition that the gripping parameters are the area parameter of the gripping surface and the height parameter of the gripping surface, wherein c_iFor one of the plurality of gripping surfaces, x is an area parameter of the one of the plurality of gripping surfaces and a height parameter of the one of the plurality of gripping surfaces, P (x | c)_i) For each grabbing plane corresponding total grabbing parameter value, P (c)_i) The probability that each grasping face is grasped without considering the grasping parameter, p (x) is the probability that the grasping parameter is the area parameter of the grasping face and the height parameter of the grasping face, that is, the value of p (x) is 1.

In this embodiment of the present disclosure, the total grabbing parameter value corresponding to each grabbing surface may be a product of a group of grabbing parameters corresponding to each grabbing surface, and the total grabbing parameter value corresponding to each grabbing surface may also be a sum of a group of grabbing parameters corresponding to each grabbing surface, which may be specifically determined according to an actual situation, which is not limited in this embodiment of the present disclosure.

And if the total grabbing parameter value corresponding to each grabbing surface is the product of a group of grabbing parameters corresponding to each grabbing surface, and the grabbing parameters are the area parameters of the grabbing surfaces and the height parameters of the grabbing surfaces, the total grabbing parameter value corresponding to each grabbing surface is the product of the area parameters of the grabbing surfaces corresponding to the grabbing surface and the height parameters of the grabbing surfaces.

Exemplarily, the image processing device determines that only two grabbing faces exist, and the grabbing parameters are an area parameter of the grabbing face and a height parameter of the grabbing face, where an area parameter value of the grabbing face corresponding to a first grabbing face is 0.5, an area parameter value of the grabbing face corresponding to a second grabbing face is 0.2, a height parameter value of the grabbing face corresponding to the first grabbing face is 0.4, an area parameter value of the grabbing face corresponding to the second grabbing face is 0.6, and then a total grabbing parameter value corresponding to the first grabbing face is 0.2, which is the product of the area parameter value of the first grabbing face being 0.5 and the height parameter value of the first grabbing face being 0.4; the total grabbing parameter value corresponding to the second grabbing surface is the product of the area parameter value of the second grabbing surface being 0.2 and the height parameter value of the second grabbing surface being 0.6, and is 0.12.

In the embodiment of the present disclosure, the image processing apparatus may determine the probability of each of the plurality of grasping faces being grasped according to the number of the plurality of grasping faces.

Illustratively, if the image processing apparatus determines that there are 5 grasping faces, the probability that each grasping face is grasped, i.e., P (c)_i) The value of (A) is 0.2; if the image processing apparatus determines that there are 2 grasping faces, the probability that each grasping face is grasped, i.e., P (c)_i) The value of (A) is 0.5; if the image processing apparatus determines that there are 3 grasping faces, the probability that each grasping face is grasped, i.e., P (c)_i) Has a value of 1/3.

In the embodiment of the present disclosure, if a group of grabbing parameters corresponding to one grabbing surface is an area parameter of the grabbing surface, a height parameter of the grabbing surface, and a flatness parameter of the grabbing surface, x in formula (1) is an area parameter of one grabbing surface of the multiple grabbing surfaces, a height parameter of one grabbing surface of the multiple grabbing surfaces, and a flatness parameter of one grabbing surface of the multiple grabbing surfaces, P (c)_i| x) is a pluralityThe evaluation value of each of the grasping surfaces under the condition that the grasping parameter is the area parameter of the grasping surface, the height parameter of the grasping surface and the flatness parameter of the grasping surface, P (x) is the probability that the grasping parameter is the area parameter of the grasping surface, the height parameter of the grasping surface and the flatness parameter of the grasping surface, that is, the value of P (x) is 1, and P (x | c)_i) And the total grabbing parameter value corresponding to each grabbing surface.

If the parameter of the grabbing surface is the product of the total grabbing parameter value corresponding to each grabbing surface and a group of grabbing parameters corresponding to each grabbing surface, and the grabbing parameters are the area parameter of the grabbing surface, the height parameter of the grabbing surface and the flatness parameter of the grabbing surface, the total grabbing parameter value corresponding to each grabbing surface is the product of the area parameter of the grabbing surface, the height parameter of the grabbing surface and the flatness parameter of the grabbing surface corresponding to the grabbing surface.

In the embodiment of the present disclosure, if a group of grabbing parameters corresponding to one grabbing surface is an area parameter of the grabbing surface, a height parameter of the grabbing surface, a flatness parameter of the grabbing surface, and an inclination parameter of the grabbing surface, x in formula (1) is an area parameter of one grabbing surface of the multiple grabbing surfaces, a height parameter of one grabbing surface of the multiple grabbing surfaces, a flatness parameter of one grabbing surface of the multiple grabbing surfaces, and an inclination parameter of one grabbing surface of the multiple grabbing surfaces, and P (c)_i| x) is the evaluation value of each of the plurality of gripping surfaces under the condition that the gripping parameters are the area parameter of the gripping surface, the height parameter of the gripping surface, the flatness parameter of the gripping surface and the inclination parameter of the gripping surface, and P (x) is the probability that the gripping parameters are the area parameter of the gripping surface, the height parameter of the gripping surface, the flatness parameter of the gripping surface and the flatness parameter of the gripping surface, that is, the value of P (x) is 1, and P (x | c)_i) And the total grabbing parameter value corresponding to each grabbing surface.

If the parameter of the grabbing surface is the product of the total grabbing parameter value corresponding to each grabbing surface and a group of grabbing parameters corresponding to each grabbing surface, and the grabbing parameters are the area parameter of the grabbing surface, the height parameter of the grabbing surface, the flatness parameter of the grabbing surface and the gradient parameter of the grabbing surface, the total grabbing parameter value corresponding to each grabbing surface is the product of the area parameter of the grabbing surface corresponding to the grabbing surface, the height parameter of the grabbing surface, the flatness parameter of the grabbing surface and the gradient parameter of the grabbing surface.

In the embodiment of the present disclosure, after the image processing apparatus obtains the plurality of grasping face evaluation values corresponding to the plurality of grasping faces, the image processing apparatus may determine a first grasping face evaluation value having a highest evaluation value from the plurality of grasping face evaluation values, and may use the grasping face corresponding to the first grasping face evaluation value as the target grasping face.

In the embodiment of the present disclosure, the manner in which the image processing apparatus determines the first grasping face evaluation value with the highest evaluation value from the plurality of grasping face evaluation values may be that the image processing apparatus first randomly determines one grasping face evaluation value from the plurality of grasping face evaluation values, and compares the grasping face evaluation value with other grasping face evaluation values, thereby determining the first grasping face evaluation value with the highest grasping face evaluation value, and the image processing apparatus may use the grasping face corresponding to the first grasping face evaluation value as the target grasping face; the image processing device can also sort the plurality of grabbing face evaluation values in a descending sorting mode, and the grabbing face corresponding to the grabbing face evaluation value sorted at the first position is used as a target grabbing face; the image processing device may further sort the plurality of grasping face evaluation values in a descending order, and use the grasping face corresponding to the grasping face evaluation value sorted at the last position as the target grasping face, where a specific manner of determining the target grasping face may be determined according to an actual situation, which is not limited in the embodiment of the present disclosure.

And S104, taking the corresponding grabbing point of the target grabbing surface as a target grabbing point.

In the embodiment of the present disclosure, after the image processing apparatus evaluates the plurality of capturing surfaces by using the plurality of capturing parameter values and determines the target capturing surface from the plurality of capturing surfaces, the image processing apparatus uses the capturing point corresponding to the target capturing surface as the target capturing point.

The target grabbing point may be a central point of the target grabbing surface. Of course, in other embodiments, the target capture point may not be the center point of the target capture plane, and may be a neighborhood point of the center point of the target capture plane, for example.

And S105, according to the target grabbing points, grabbing point positions corresponding to the target grabbing points are determined, and the target objects corresponding to the target grabbing points are grabbed from the multi-dimensional image to be processed according to the grabbing point positions.

In the embodiment of the present disclosure, after the image processing device determines the target grabbing point, the image processing device may determine the grabbing point pose corresponding to the target grabbing point according to the target grabbing point, so as to grab the target object corresponding to the target grabbing point from the multi-dimensional image to be processed according to the grabbing point pose.

It should be noted that the pose of the grab point corresponding to the target grab point may be a six-dimensional pose of the target grab point, a five-dimensional pose of the target grab point, or a pose of another dimension of the target grab point, which may be specifically determined according to an actual situation, which is not limited in the embodiment of the present disclosure.

It should be noted that the information of the target capture point may be three-dimensional coordinate point information.

When the grabbing point pose corresponding to the target grabbing point is the six-dimensional pose of the target grabbing point, the image processing device can determine the grabbing point pose corresponding to the target grabbing point according to the three-dimensional coordinate point information of the target grabbing point and the rotation degree information of the target grabbing point.

In this embodiment of the disclosure, the manner in which the image processing device determines the rotation information of the target capture point may be that the image processing device performs plane fitting according to a plurality of target pixel points on the target capture surface to obtain a fitted target capture surface, the image processing device determines a tangent line of the target capture point on the fitted plane, and the image processing device uses a vertical vector of the tangent line as the rotation information, thereby obtaining the rotation information of the target capture point.

The disclosed embodiment provides an exemplary image processing method flowchart, as shown in fig. 2, when a CPU in an intelligent robot controls a camera to acquire raw image data information of a multi-dimensional image to be processed, that is, when an image processing apparatus acquires raw RGBD image data information of the multi-dimensional image to be processed, the CPU controls to transmit the raw RGBD image data to a GPU, the GPU pre-processes the raw RGBD image data to obtain RGBD image data information, that is, image data information, the GPU determines a plurality of capturing surfaces and a plurality of capturing points corresponding to the plurality of capturing surfaces from the RGBD image data information by using a depth learning network model, the GPU evaluates each of the plurality of capturing surfaces by using a plurality of capturing parameters, determines a target capturing surface from the plurality of capturing surfaces according to an evaluation result of each capturing surface, and takes the capturing point corresponding to the target capturing surface as the target capturing point, and determining the grabbing point position of the target object according to the target grabbing point, and after the image processing device determines the grabbing point position of the target object, the intelligent robot can grab the target object at the grabbing point position.

The image processing device determines a plurality of grabbing parameter values corresponding to a plurality of grabbing surfaces, and evaluates the plurality of grabbing surfaces according to the plurality of grabbing parameter values, so that a target grabbing surface is determined from the plurality of grabbing surfaces, the target grabbing surface is not determined according to a single height parameter value, the accuracy of determining the target grabbing surface by the image processing device is improved, the grabbing point position of a target object is determined by the image processing device according to the target grabbing surface with high accuracy, and the accuracy of determining the position and the attitude of the target object by the image processing device is improved.

When the image processing device is applied to the logistics transmission process, the image processing device can determine the grabbing point pose of the target express according to the implementation mode, so that sorting and stacking of the express are achieved.

When the image processing device is applied to the medicine and beauty makeup processes, the image processing device can determine the grabbing point pose of the target medicine according to the implementation mode, or the image processing device can determine the grabbing point pose of the target beauty makeup product according to the implementation mode, so that classified packaging of the medicine and the beauty makeup product is achieved.

When the image processing device is applied to heavy industry, the image processing device can determine the grabbing point pose of the target industrial product according to the implementation mode, so that the target industrial product is carried.

When the image processing device is applied to the garbage processing process, the image processing device can determine the position and the posture of the grabbing point of the target garbage according to the implementation mode, so that the garbage classification processing is realized.

Based on the same inventive concept, the disclosed embodiments provide an image processing apparatus 1, corresponding to an image processing method; fig. 3 is a schematic diagram illustrating a first composition structure of an image processing apparatus according to an embodiment of the present disclosure, where the image processing apparatus 1 may include:

the determining unit 11 is configured to determine, according to image data information of a to-be-processed multidimensional image, a plurality of capturing surfaces and a plurality of capturing points corresponding to the plurality of capturing surfaces from the to-be-processed multidimensional image, where the plurality of capturing surfaces correspond to the plurality of capturing points one to one; determining a plurality of grabbing parameters corresponding to the plurality of grabbing surfaces; taking a grabbing point corresponding to the target grabbing surface as a target grabbing point; according to the target grabbing point, a grabbing point position corresponding to the target grabbing point is determined, and a target object corresponding to the target grabbing point is grabbed from the RGBD image according to the grabbing point position;

the evaluation unit 12 is configured to evaluate the plurality of grabbing surfaces by using the plurality of grabbing parameters, and determine the target grabbing surface from the plurality of grabbing surfaces according to an evaluation result.

In some embodiments of the present disclosure, the evaluating unit 12 is specifically configured to evaluate each of the plurality of capturing surfaces by using a plurality of parameters of the plurality of capturing parameters, respectively, to obtain a plurality of capturing surface evaluation values corresponding to the plurality of capturing surfaces;

the determining unit 11 is specifically configured to determine a first capturing surface evaluation value with a highest evaluation value from the plurality of capturing surface evaluation values, and use a capturing surface corresponding to the first capturing surface evaluation value as the target capturing surface.

In some embodiments of the present disclosure, the plurality of grabbing parameters comprises at least one of:

In some embodiments of the present disclosure, the determining unit 11 is specifically configured to input image data information of the multidimensional image to be processed into a deep learning network model, so as to obtain a plurality of pixel points and a plurality of central points corresponding to the plurality of pixel points, where the deep learning network model is a model obtained by training an initial deep learning network model by using sample image data information of a sample multidimensional image, and the plurality of pixel points and the plurality of central points are in one-to-one correspondence; dividing the plurality of center points into a plurality of groups of center points; determining a grabbing point corresponding to any group of central points according to any group of central points in the multiple groups of central points until the multiple grabbing points are determined from the multiple groups of central points; and determining a grabbing surface corresponding to any group of central points according to any group of pixel points corresponding to any group of central points in the multiple groups of central points until the multiple grabbing surfaces are determined from the multiple groups of pixel points corresponding to the multiple groups of central points, wherein any group of central points corresponds to any group of pixel points one to one.

In some embodiments of the present disclosure, the determining unit 11 is specifically configured to average a group of position data of any group of central points to obtain an average position data, until a plurality of average position data are obtained from a plurality of groups of position data information of the plurality of groups of central points; and taking a plurality of points corresponding to the plurality of average position data as a plurality of grabbing points, wherein the plurality of average position data correspond to the plurality of grabbing points one by one.

In some embodiments of the present disclosure, the apparatus further comprises an acquisition unit 13 and a pre-processing unit 14;

the acquiring unit 13 is configured to acquire original image data information of the multidimensional image to be processed;

the preprocessing unit 14 is configured to preprocess the original image data information to obtain the image data information.

In some embodiments of the present disclosure, the preprocessing unit 14 is specifically configured to adjust the number of the original data information to a preset number if the number of the original data information in the original image data information does not satisfy a preset number value; and dividing the data of the original data information adjusted to the preset number by a preset numerical value respectively to obtain the image data information.

In some embodiments of the present disclosure, the image data information comprises color channel data information and depth data information.

In practical applications, the determining Unit 11, the evaluating Unit 12, the obtaining Unit 13, and the preprocessing Unit 14 may be implemented by a processor 15 on the motion quality evaluating apparatus 1, specifically implemented by a GPU (Graphics processing Unit), an MPU (Microprocessor Unit), a DSP (Digital signal processing) or a Field Programmable Gate Array (FPGA), and the like; the above data storage may be realized by the memory 16 on the motion quality estimation apparatus 1.

The embodiment of the present disclosure also provides an image processing apparatus 1, as shown in fig. 4, the image processing apparatus 1 including: a processor 15 and a memory 16, said memory 16 storing an image processing program executable by said processor 14, said program, when executed, performing by said processor 15 an image processing method as described above.

In practical applications, the Memory 16 may be a volatile Memory (volatile Memory), such as a Random-Access Memory (RAM); or a non-volatile Memory (non-volatile Memory), such as a Read-Only Memory (ROM), a flash Memory (flash Memory), a Hard disk (Hard disk Drive, HDD) or a Solid-State Drive (SSD); or a combination of the above types of memories and provides instructions and data to the processor 15.

The disclosed embodiments provide a computer readable storage medium having thereon a computer program which, when executed by a processor 15, implements an image processing method as described above.

The embodiment provides a robot, which comprises a mechanical arm and an image processing device, wherein the image processing device is used for executing the method, and the mechanical arm is used for grabbing a target object at a grabbing point position under the condition that the image processing device determines the grabbing point position of the target object.

Specifically, after the position and posture of the target object grabbing point determined by the image processing device are obtained, the mechanical arm can calculate the grabbing position and posture of the mechanical arm grabbing the target object at the grabbing point position according to the grabbing point position and posture, so that a motion path of the mechanical arm is planned for object grabbing.

As will be appreciated by one skilled in the art, embodiments of the present disclosure may be provided as a method, system, or computer program product. Accordingly, the present disclosure may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present disclosure may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.

The present disclosure is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the disclosure. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

The above description is only for the preferred embodiment of the present disclosure, and is not intended to limit the scope of the present disclosure.

Claims

1. An image processing method, characterized in that the method comprises:

2. The method of claim 1, wherein evaluating the plurality of gripping surfaces using the plurality of gripping parameters and determining a target gripping surface from the plurality of gripping surfaces based on the evaluation comprises:

3. The method of claim 1 or 2, wherein the plurality of grabbing parameters comprises at least one of:

4. The method according to any one of claims 1 to 3, wherein determining a plurality of capturing surfaces and a plurality of capturing points corresponding to the plurality of capturing surfaces from the multi-dimensional image to be processed according to the image data information of the multi-dimensional image to be processed comprises:

5. The method of claim 4, wherein determining a grabbing point corresponding to any one of the sets of center points according to any one of the sets of center points until the plurality of grabbing points are determined from the sets of center points comprises:

6. The method according to any one of claims 1 to 5, wherein before determining a plurality of grasping faces and a plurality of grasping points corresponding to the plurality of grasping faces from the multidimensional image to be processed according to the image data information of the multidimensional image to be processed, the method further comprises:

7. The method of claim 6, wherein said pre-processing the raw image data information to obtain the image data information comprises:

8. The method according to any of claims 1-7, wherein the image data information comprises color channel data information and depth data information.

9. An image processing apparatus, characterized in that the apparatus comprises:

10. An image processing apparatus, characterized in that the apparatus comprises:

a memory storing an image processing program executable by the graphics processor and a graphics processor, the image processing program when executed to perform the method of any of claims 1 to 8 by the graphics processor.

11. A storage medium on which a computer program is stored for application to an image processing apparatus, characterized in that the computer program, when executed by a graphics processor, implements the method of any one of claims 1 to 8.

12. A robot, comprising: a robot arm for gripping the target object at a gripping point position in a case where the image processing apparatus determines the gripping point position of the target object, and an image processing apparatus for performing the method according to any one of claims 1 to 8.