WO2024087331A1 - 一种基于三元组对比网络的机器人抓取预测方法 - Google Patents

一种基于三元组对比网络的机器人抓取预测方法 Download PDF

Info

Publication number
WO2024087331A1
WO2024087331A1 PCT/CN2022/137001 CN2022137001W WO2024087331A1 WO 2024087331 A1 WO2024087331 A1 WO 2024087331A1 CN 2022137001 W CN2022137001 W CN 2022137001W WO 2024087331 A1 WO2024087331 A1 WO 2024087331A1
Authority
WO
WIPO (PCT)
Prior art keywords
sample
data
grasping
dimensional features
training
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2022/137001
Other languages
English (en)
French (fr)
Inventor
易正琨
柳程亮
吴新宇
崔允端
谢祥
田琼
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Institute of Advanced Technology of CAS
Original Assignee
Shenzhen Institute of Advanced Technology of CAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Institute of Advanced Technology of CAS filed Critical Shenzhen Institute of Advanced Technology of CAS
Publication of WO2024087331A1 publication Critical patent/WO2024087331A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J15/00—Gripping heads and other end effectors
    • B25J15/08—Gripping heads and other end effectors having finger members
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J13/00—Controls for manipulators
    • B25J13/08—Controls for manipulators by means of sensing devices, e.g. viewing or touching devices
    • B25J13/087—Controls for manipulators by means of sensing devices, e.g. viewing or touching devices for sensing other physical parameters, e.g. electrical or chemical properties
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls
    • B25J9/1656—Program controls characterised by programming, planning systems for manipulators
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks

Definitions

  • the present invention relates to the field of robotics technology, and more specifically, to a robot grasping prediction method based on a triplet contrast network.
  • robots are used in industries ranging from automobile assembly, metal processing to 3C manufacturing, home appliance production, food packaging, and catering services.
  • robot grasping is a common area that needs to be further studied and optimized. Before executing the grasping, accurately predicting the success rate of this grasping based on the existing information is of great benefit to further generate high-quality and stable grasping.
  • patent application CN114700947A proposes a method for predicting the probability of slipping in grasping by visual-tactile fusion, in which a visual camera and a tactile sensor are installed on the robot arm at the same time to obtain information from the two modes.
  • the visual and tactile perception fusion algorithm uses the collected visual information to obtain the external contour information of the target object, and the tactile information to obtain the surface morphology and texture of the target object.
  • the convolutional neural network is used to extract the features of the visual information and the features of the tactile information.
  • the new features obtained after the fusion of the two modal feature information are used to predict the probability of slipping in the object grasping process.
  • Patent application CN114083535A provides a physical measurement method and device for the quality of a robot's grasping posture.
  • the method includes: determining the flatness score of the contact point between the robot's candidate grasping posture and the object; determining the center of gravity score of the candidate grasping posture holding the object; and evaluating the quality of the candidate grasping posture based on the flatness score and the center of gravity score.
  • the physical measurement method for the quality of a robot's grasping posture provided by this solution is based on the characteristics that humans tend to touch the flatter parts of an object when picking up an object in life and are accustomed to touching the center of an object. It evaluates the quality of the robot's grasping posture from the flatness of the object itself and gravity analysis through two physical measurement scores.
  • the purpose of the present invention is to overcome the defects of the prior art and provide a robot grasping prediction method based on a triplet comparison network.
  • the method comprises the following steps:
  • each sample data reflects the correspondence between the tactile data at multiple moments in the process of the robot arm grasping an object and a classification label, where the classification label is used to indicate whether the grasping is successful;
  • the encoder is trained based on the set loss function to obtain optimized parameters, wherein during the training process, two samples with the same label are randomly selected from the training set as anchor samples and positive samples, respectively, and a sample with the opposite label is randomly selected as a negative sample; the sample data are respectively input into the corresponding encoder for encoding to obtain high-dimensional features of the anchor sample, high-dimensional features of the positive sample, and high-dimensional features of the negative sample;
  • the trained encoder and the trained classifier are used to predict grasping results based on the tactile data collected in real time.
  • the advantages of the present invention are that the proposed robot grasping prediction method based on triple comparison network uses a deep neural network to train the encoder by comparing samples with each other, thereby improving sample utilization and obtaining better network parameters from fewer samples.
  • the present invention adds a self-attention mechanism to the network, which enables the network to more effectively capture the internal correlation of features, thereby further improving the network prediction accuracy.
  • FIG1 is a flow chart of a robot grasping prediction method based on a triplet contrast network according to an embodiment of the present invention
  • FIG2 is a schematic diagram of an encoder network according to an embodiment of the present invention.
  • FIG3 is a schematic diagram of encoder contrastive learning training according to an embodiment of the present invention.
  • FIG4 is a schematic diagram of classifier training according to an embodiment of the present invention.
  • FIG5 is a schematic diagram of a process of a robot grasping prediction method based on a triplet contrast network according to an embodiment of the present invention.
  • the present invention mainly relates to a robot grasping prediction method based on a triplet comparison network, which belongs to the field of robot technology.
  • the technical solution is as follows:
  • the provided robot grasping prediction method based on triple contrast network includes the following steps:
  • Step S110 constructing a training set, wherein each piece of sample data reflects the corresponding relationship between the tactile data at multiple moments in the process of the robot arm grasping the object and whether the grasping is successful.
  • robot grasping datasets based on tactile or containing tactile information can be obtained by autonomous collection or online downloading of public datasets.
  • the Calandra public dataset is used.
  • a Weiss WSG-50 robotic arm with GELSIGHT tactile sensors installed on two fingers is used to collect tactile data.
  • a Microsoft Kinect 2 depth camera is placed in front of the grasping workbench to collect visual data.
  • 106 different objects are grasped 9296 times.
  • time Ta when the robotic arm is placed in the initial position, it is marked as time Ta ; when the robotic arm grasps the target object but has not yet lifted it, it is marked as time Tb ; the time after the grasped object hovers in the air for 2 seconds is Tc .
  • the corresponding visual and tactile data are obtained for these three moments.
  • the dataset is preprocessed to enhance the data, including random horizontal flipping, random cropping, normalization, etc., and divided into training and testing sets.
  • the tactile data image is randomly flipped (the tactile data from two fingers are flipped at the same time or not), and the 256 ⁇ 256 tactile data is randomly cropped to 224 ⁇ 224.
  • the training set and the test set are divided based on the small sample scenario. For example, from the perspective of the small sample number of the categories of the grasped objects, 10 kinds of grasping data of grasped objects are randomly selected from the data set to form the training set. From the perspective of the small sample of grasping data, 20 data are randomly selected from the grasping data of each object as the training set samples. Thus, a training set with only 200 samples can be obtained. In addition, 1000 grasping data are randomly selected from the grasping data of the remaining untrained objects as the test set.
  • the constructed training set has a small number of samples.
  • Each sample data reflects the correspondence between the tactile data (or tactile and visual data) at multiple moments in the process of the robot arm grasping an object and the classification label.
  • the classification label is used to indicate whether the grasping is successful.
  • Step S120 constructing an encoder, and generating a large number of sample triplets based on a small number of samples in the training set to train the encoder.
  • step S120 includes the following sub-steps:
  • Step S121 constructing an encoder
  • the encoder training framework is adaptable to scenarios with different numbers of tactile sensors.
  • the tactile data collected by the sensor at time T a and T b are represented as Ga and G b respectively, and Ga is subtracted from G b to highlight the change in the tactile information between the two moments, and Ga and G b -Ga are superimposed in the channel direction as a six-channel image input into the pre-trained Resnet-50 backbone network without the last layer to extract high-dimensional features.
  • the high-dimensional features extracted from the data of different tactile sensors are spliced to obtain high-dimensional features. Then, it is input into the self-attention module to obtain fused high-dimensional features.
  • the schematic diagram of the encoder in the dual tactile sensor scenario is shown in Figure 2.
  • the encoder includes two backbone networks, each backbone network corresponding to the tactile data of one sensor. It should be understood that when there are multiple sensors and each sensor has the same model and the contact situation with the target object is similar, each backbone network can share parameters to save the size of the model space, or not share parameters to fit a more complex mapping network to obtain better performance.
  • the backbone network used can be replaced by a network other than Resnet-50, such as VGG, AlexNet, etc.
  • the input vision-based tactile modality information can theoretically be replaced by other modality data, such as visual photography information, etc.
  • Step S122 training the encoder
  • two samples with the same label are randomly selected from the training set as anchor samples and positive samples, respectively, and a sample with the opposite label is randomly selected as a negative sample.
  • the sample data are respectively input into the encoder to encode the high-dimensional features of the anchor sample, positive sample, and negative sample, which are represented as fa , fp , and fn , respectively.
  • the triplet contrastive loss function is used as the evaluation index, which is formulated as follows:
  • LT represents the triple contrast loss.
  • the encoder will make the Euclidean distance between the high-dimensional features encoded by samples of the same label as small as possible, and the Euclidean distance between the high-dimensional features encoded by samples of different labels as large as possible.
  • this triple form a large number of sample triplets can be generated to train the encoder when there are fewer samples, thereby alleviating the overfitting problem and training an effective encoder.
  • Step S130 using the trained encoder to encode the input sample data into high-dimensional features, and using the high-dimensional features to train a classifier, wherein the parameters of the encoder are frozen during the training process.
  • step S130 includes the following sub-steps:
  • Step S131 constructing a classifier
  • the encoder parameters are frozen, the input samples are encoded into high-dimensional features, and then the high-dimensional features are input into the subsequent classifier network to obtain the prediction results.
  • the classifier network is shown in Figure 4, which as a whole includes a self-attention module and a multi-layer perceptron.
  • the multi-layer perceptron can be constructed using multiple fully connected layers.
  • the encoded high-dimensional features are input into the self-attention module, and then the processed features are input into a multilayer perceptron consisting of two fully connected layers.
  • the number of input neurons in the first fully connected layer is 2048*N, where N represents the number of sensor sources of input tactile data, and the number of output neurons is 1024, followed by a Relu nonlinear transformation activation layer; the number of input neurons in the second fully connected layer is 1024, the number of output neurons is 2, followed by a Sigmoid nonlinear transformation activation layer, and the final output is a score that predicts whether the grasping is successful or not.
  • Step S132 training the classifier
  • the training set is used to train the above classifier neural network using the binary cross-entropy loss function.
  • the binary cross-entropy loss function LB is formally expressed as follows:
  • x represents the input sample data
  • test set can be used to verify the effectiveness of the model, and the evaluation indicator is the correct prediction accuracy.
  • Step S140 combining the trained encoder and classifier into a prediction model to predict the robot's grasping result based on the real-time collected tactile data.
  • the encoder and classifier After the encoder and classifier are trained, they can be combined into an overall prediction model, which can be used to predict whether the robot arm's grasping is successful based on the real-time collected tactile data.
  • the present invention has at least the following advantages:
  • the present invention provides a deep neural network (including encoders and classifiers, etc.) that can adapt to any number of tactile sensors to predict grasping stability through tactile information before grasping. It can effectively predict grasping stability in scenarios combining single or multiple tactile sensor information while keeping the framework basically unchanged. Moreover, its framework and core ideas are independent of the types of tactile sensors and robot arms, and have good generalization.
  • the present invention trains encoders based on triple contrast learning.
  • the feature encoder is trained with the goal of reducing the Euclidean distance of deep high-dimensional features after encoding similar samples and expanding the Euclidean distance of deep high-dimensional features after encoding heterogeneous samples, so as to explore the relationship between samples, and generate a large number of new training units by constructing triplets, thereby improving the network prediction accuracy performance in small sample scenarios, thereby alleviating the overfitting problem that is easy to occur during the training process when there are fewer samples, and obtaining better encoder parameters.
  • the relationship between samples is excavated, and the samples are not considered separately for end-to-end training. This approach makes fuller use of the potential information of sample data in the training process and obtains better training results.
  • the present invention introduces a self-attention mechanism into the network.
  • the self-attention mechanism helps the network to explore the internal correlation of features during the training process, highlight the important parts of high-dimensional deep features that help improve classification accuracy, and further improve the prediction accuracy of the model.
  • the present invention may be a system, a method and/or a computer program product.
  • the computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.
  • Computer readable storage medium can be a tangible device that can keep and store the instructions used by the instruction execution device.
  • Computer readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof.
  • Non-exhaustive list of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination thereof.
  • RAM random access memory
  • ROM read-only memory
  • EPROM or flash memory erasable programmable read-only memory
  • SRAM static random access memory
  • CD-ROM compact disk read-only memory
  • DVD digital versatile disk
  • memory stick a floppy disk
  • mechanical encoding device for example, a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination thereof.
  • Computer readable storage medium used here is not interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic waves, an electromagnetic wave propagated by a waveguide or other transmission medium (for example, a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.
  • the computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing/processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and/or a wireless network.
  • the network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and/or edge servers.
  • the network adapter card or network interface in each computing/processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing/processing device.
  • the computer program instructions for performing the operation of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, Python, etc., and conventional procedural programming languages, such as "C" language or similar programming languages.
  • Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server.
  • the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet).
  • an electronic circuit such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present invention.
  • These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions/actions specified in one or more boxes in the flowchart and/or block diagram is generated.
  • These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and/or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions/actions specified in one or more boxes in the flowchart and/or block diagram.
  • Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions/actions specified in one or more boxes in the flowchart and/or block diagram.
  • each box in the flowchart or block diagram can represent a part of a module, program segment or instruction, and the part of the module, program segment or instruction contains one or more executable instructions for realizing the specified logical function.
  • the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved.
  • each box in the block diagram and/or flowchart, and the combination of the boxes in the block diagram and/or flowchart can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that it is equivalent to implement it by hardware, implement it by software, and implement it by combining software and hardware.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Mechanical Engineering (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Artificial Intelligence (AREA)
  • Robotics (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • Manipulator (AREA)
  • Image Analysis (AREA)

Abstract

本发明公开一种基于三元组对比网络的机器人抓取预测方法。该方法包括：构建训练集，其中每条样本反映机器臂抓取物体过程中多个时刻的触觉数据与分类标签之间的对应关系；基于设定的损失函数训练编码器，在训练过程中，在训练集随机抽取两个标签一致的样本分别作为锚样本和正样本，并随机抽取一个标签相反的样本作为负样本，将样本数据分别输入对应编码器中编码得到锚样本的高维特征、正样本的高维特征以及负样本的高维特征；冻结编码器的优化参数，并利用经训练的编码器将输入样本数据编码为高维特征，进而将该高维特征输入分类器进行训练；利用经训练的编码器和分类器针对实时的触觉数据预测抓取结果。本发明提升了机器人抓取预测的准确率。

Description

一种基于三元组对比网络的机器人抓取预测方法 技术领域
本发明涉及机器人技术领域，更具体地，涉及一种基于三元组对比网络的机器人抓取预测方法。
背景技术
随着时代的发展，智能机器人技术已经在全球众多行业内广泛应用，机器人的应用场景越来越丰富。例如，从汽车装配、金属加工到3C制造、家电生产、食品包装以及餐饮服务等行业均存在机器人应用。在这些应用场景中，机器人抓取是一个常见并且有待深入研究优化的领域。在执行抓取之前，根据现有信息准确预测本次抓取的成功率对于进一步生成高质量稳定的抓取大有裨益。
在现有技术中，专利申请CN114700947A提出了一种视触融合的抓取滑动概率预测方法，该方法在机器手上同时安装有视觉相机和触觉传感器，用于获得两个模态的信息。视觉与触觉感知融合算法利用采集的视觉信息获取目标物体的外部轮廓信息，触觉信息获取目标物体的表面形貌与质地软硬等信息，将采集的信息预处理后利用卷积神经网络提取视觉信息的特征和触觉信息的特征，两个模态特征信息融合之后获得的新的特征用于预测物体抓取过程中的滑动概率。
专利申请CN114083535A提供了一种机器手抓取姿势质量的物理度量方法及装置。该方法包括：确定机器手的候选抓取姿势与物体之间接触点的平坦度分数；确定所述候选抓取姿势夹持所述物体的重心分数；基于所述平坦度分数和所述重心分数评估候选抓取姿势的质量。该方案提供的机器手抓取姿势质量的物理度量方法，基于人类在生活中拿起物体时倾向于接触物体上更平坦的部位以及习惯于接触物体的中心部位的特性，通过两种物理度量分数分别从物体自身平坦度和重力分析上评价机器手抓取姿势的质量。
上述现有技术虽然从不同的角度提出了能够预测抓取是否成功的方案，但就训练集的数据量而言，一些现有方案基于数据驱动，需要大量抓取数据 作为训练集支撑，训练代价高昂。从特征提取方法上来看，一些现有方案训练过程未考虑样本与样本之间的关系，没能充分发掘和利用样本信息，或者自创构建手工特征，局限性强，泛化性弱，可信度不高。
综上，人们在抓握物体时可以感受预测当前的力度是否足以稳定的抓取物品，而近期研究表明，通过机器人抓取物品提升阶段之前的触觉信息，也可以有效预测机器人抓取是否成功。然而，当前的机器人抓取预测算法在训练过程中忽略了样本与样本之间的关系，仅仅通过单纯的端到端的方式来训练。此外，这些方法通常基于数据驱动，在可供训练的抓取样本数据较少的情况下难以有效训练模型。
发明内容
本发明的目的是克服上述现有技术的缺陷，提供一种基于三元组对比网络的机器人抓取预测方法。该方法包括以下步骤：
构建训练集，其中每条样本数据反映机器臂抓取物体过程中多个时刻的触觉数据与分类标签之间的对应关系，该分类标签用于指示抓取是否成功；
基于设定的损失函数训练编码器，获得优化参数，其中在训练过程中，在所述训练集随机抽取两个标签一致的样本分别作为锚样本和正样本，并随机抽取一个标签相反的样本作为负样本；将样本数据分别输入对应编码器中编码得到锚样本的高维特征、正样本的高维特征以及负样本的高维特征；
冻结所述编码器的优化参数，并利用经训练的编码器将输入样本数据编码为高维特征，进而将该高维特征输入分类器进行训练；
利用经训练的编码器和经训练的分类器针对实时采集的触觉数据预测抓取结果。
与现有技术相比，本发明的优点在于，所提出的基于三元组对比网络的机器人抓取预测方法，利用深度神经网络，通过将样本相互对比来训练编码器，提高了样本利用率，能够从更少的样本中获得更佳的网络参数。并且，本发明在网络中加入了自注意力机制，该机制使网络能更有效的捕捉特征的内部相关性，从而进一步提升了网络预测准确率。
通过以下参照附图对本发明的示例性实施例的详细描述，本发明的其 它特征及其优点将会变得清楚。
附图说明
被结合在说明书中并构成说明书的一部分的附图示出了本发明的实施例，并且连同其说明一起用于解释本发明的原理。
图1是根据本发明一个实施例的基于三元组对比网络的机器人抓取预测方法的流程图；
图2是根据本发明一个实施例的编码器网络的示意图；
图3是根据本发明一个实施例的编码器对比学习训练的示意图；
图4是根据本发明一个实施例的分类器训练示意图；
图5是根据本发明一个实施例的基于三元组对比网络的机器人抓取预测方法的过程示意图。
具体实施方式
现在将参照附图来详细描述本发明的各种示例性实施例。应注意到：除非另外具体说明，否则在这些实施例中阐述的部件和步骤的相对布置、数字表达式和数值不限制本发明的范围。
以下对至少一个示例性实施例的描述实际上仅仅是说明性的，决不作为对本发明及其应用或使用的任何限制。
对于相关领域普通技术人员已知的技术、方法和设备可能不作详细讨论，但在适当情况下，所述技术、方法和设备应当被视为说明书的一部分。
在这里示出和讨论的所有例子中，任何具体值应被解释为仅仅是示例性的，而不是作为限制。因此，示例性实施例的其它例子可以具有不同的值。
应注意到：相似的标号和字母在下面的附图中表示类似项，因此，一旦某一项在一个附图中被定义，则在随后的附图中不需要对其进行进一步讨论。
本发明主要涉及一种基于三元组对比网络的机器人抓取预测方法，属于机器人技术领域。其技术方案内容如下：
参见图1和图5所示，所提供的基于三元组对比网络的机器人抓取预 测方法包括以下步骤：
步骤S110，构建训练集，其中每条样本数据反映机器臂抓取物体过程中多个时刻的触觉数据与抓取是否成功之间的对应关系。
在实际应用中，可以通过自主采集或在线下载公开数据集，以获取基于触觉或包含触觉信息的机器人抓取数据集。
例如，采用Calandra公开数据集，数据集中使用一个在两指上分别安装有GELSIGHT触觉传感器的Weiss WSG-50机械臂采集触觉数据，在抓取工作台前置放一个Microsoft Kinect 2深度相机采集视觉数据，抓取106种不同物品共9296次，数据集采集过程中，机器臂置于初始位置时，标记为时刻T a，机器臂抓住目标物品但尚未提升时，标记为时刻T b，抓住物品在空中悬停2秒后的时刻为T c，针对这三个时刻获取对应的视触觉数据。
进一步地，对数据集进行预处理来增强数据，包括随机水平翻转，随机裁剪，归一化处理等，并将其划分为训练集和测试集。
例如，仅使用T a、T b时刻的触觉数据。作为数据增强，将触觉数据图像随机翻转(来自两指的触觉数据同时翻转或不反转)，并将256×256的触觉数据随机裁剪为224×224。
在一个实施例中，为了提高训练的效率并降低对样本量的依赖，基于小样本场景划分训练集和测试集。例如，从抓取物品类别数目的小样本角度，从数据集中随机选取10种抓取物品的抓取数据构成训练集。从抓取数据的小样本角度，从每个物品的抓取数据中随机选取20个数据作为训练集样本。于是可以得到一个仅有200个样本的训练集，此外，从剩余未被训练的物品的抓取数据中随机选取1000次抓取数据作为测试集。
综上，所构建的训练集样本量较少，每条样本数据反映机器臂抓取物体过程中多个时刻的触觉数据(或触觉以及视觉数据)与分类标签之间的对应关系，分类标签用于指示抓取是否成功。
步骤S120，构建编码器，并基于训练集中的少量样本产生大量的样本三元组对编码器进行训练。
具体地，步骤S120包括以下子步骤：
步骤S121，构建编码器
编码器训练框架适应不同数目的触觉传感器情景。对于只有单个触觉传感器的情况，将传感器在T a、T b时刻采集到的触觉数据分别表示为G a、G b，将G a与G b作差以突出两时刻间触觉信息的变化部分，并将G a与G b-G a在通道方向叠加为一个六通道图像输入预训练的去掉最后一层的Resnet-50骨干网络以提取高维特征。对于由多个触觉传感器的情况，将不同传感器的数据以上述形式进行处理后，将不同触觉传感器数据提取的高维特征拼接得到高维特征。然后，将其输入自注意力模块得到融合高维特征，双触觉传感器情景下编码器示意图参见图2，该编码器包含两个骨干网络，每个骨干网络对应一个传感器的触觉数据。应理解的是，在多传感器且各传感器型号相同、与目标物体接触情况近似时，各骨干网络可以共享参数以节约模型空间大小，或不共享参数以拟合更复杂的映射网络获得更优的性能。
应理解的是，所采用的骨干网络，可使用除了Resnet-50以外的网络代替，如VGG、AlexNet等。此外，输入的基于视觉的触觉模态信息，理论上也可以由其他模态数据代替，如视觉拍照信息等。
步骤S122，训练编码器
参见图3所示，在训练过程中，在训练集随机抽取两个标签一致(即均抓取成功或失败)的样本分别作为锚样本和正样本，再随机抽取一个标签相反的样本作为负样本，将样本数据分别输入编码器中编码得到锚样本、正样本、负样本的高维特征分别表示为f a、f p、f n，然后，应用三元对比损失函数(Triplet Contrastive Loss Function)作为评价指标，将其公式化表示如下：
Figure PCTCN2022137001-appb-000001
其中L T表示三元对比损失。通过将L T最小化，编码器将使同类标签样本编码高维特征之间的欧氏距离尽量小，异类样本编码高维特征之间的欧氏距离尽量大。通过设计这种三元组的形式，可以在样本较少时产生大量的样本三元组对编码器进行训练，从而缓解过拟合问题，进而训练出有效的编码器。
综上，在样本量较小的情况下，通过构建三元组对比学习深度神经网络能够挖掘样本与样本之间的相互关系，提升了编码器的精确度，并结合自 注意力机制考虑不同样本对编码器的贡献程度，进一步挖掘了样本间的关联关系。
步骤S130，利用经训练的编码器将输入样本数据编码为高维特征，并利用该高维特征训练分类器，其中训练过程中冻结编码器的参数。
在一个实施例中，步骤S130包括以下子步骤：
步骤S131，构建分类器
在获取合适的编码器参数以后，将编码器参数冻结，将输入样本编码为高维特征，再将高维特征输入后续的分类器网络中以获得预测结果，分类器网络如图4所示，其整体上包括自注意力模块和多层感知机，多层感知机可采用多层全连接层构建。
具体地，将编码高维特征输入自注意力模块，然后将处理后的特征输入由两层全连接层构成的多层感知机中，其中第一层全连接层的输入神经元数量为2048*N，N表示输入触觉数据的传感器来源个数，输出神经元数量为1024，后接一层Relu非线性变换激活层；第二层全连接层的输入神经元数量为1024，输出神经元数量为2，后接一层Sigmoid非线性变换激活层，最终输出分别预测抓取成功或失败的得分。
步骤S132，训练分类器
使用训练集对上述分类器神经网络用二值交叉熵(Binary Cross-Entropy)损失函数进行训练，二值交叉熵损失函数L B形式化表示如下：
L B＝-(ylog(p(x)+(1-y)log(1-p(x))))  (2)
其中，x表示输入的样本数据，y表示该样本实际上抓取成功或失败的类别，例如用y＝1表示抓取成功，y＝0表示抓取失败，p(x)表示在当前输入样本数据情况下，分类器预测该样本抓取成功的概率。
在分类器训练完成后，可使用测试集验证该模型的有效性，评价指标为正确预测准确率。
步骤S140，将经训练的编码器和分类器组合为预测模型，针对实时采集的触觉数据预测机器人的抓取结果。
在完成编码器和分类器的训练后，可将编码器和分类器组合为整体的预测模型，利用该预测模型针对实时采集的触觉数据预测机器臂的抓取是否 成功。
综上所述，相对于现有技术，本发明至少具有以下优势：
1)本发明提供一种适应任意数量触觉传感器的深度神经网络(包括编码器和分类器等)来通过抓取前的触觉信息预测抓取稳定性，可以在框架基本不变的情况下，结合单个或多个触觉传感器信息场景下进行有效的抓取稳定性预测，且其框架、思路核心思想与触觉传感器、机器手类型无关，具有较好的泛化性。
2)本发明基于三元组对比学习训练编码器。通过对比学习，以缩小同类样本编码后深度高维特征欧氏距离，扩大异类样本编码后深度高维特征欧氏距离为目标训练特征编码器，以此发掘样本间的关系，并且通过构建三元组产生大量新的训练单元，从而提升了小样本情境下的网络预测准确率表现，从而在样本较少的情况下缓和训练过程中容易产生的过拟合问题，获得较好的编码器参数。通过对比样本，挖掘了样本之间的相互关系，而并不单独考虑样本进行端到端训练。这样的方式使训练过程对样本数据潜在信息的利用更充分，获得更好的训练效果。
3)本发明在网络中引入自注意力机制。自注意力机制有助于网络在训练过程中挖掘特征内部的相关性，突出高维深度特征中有助于提高分类准确率的重要部分，进一步提升模型的预测准确率。
4)本发明所提出网络结构已在Calandra公开数据集上得到很好的验证，相比现有模型预测性能有所提升。
本发明可以是系统、方法和/或计算机程序产品。计算机程序产品可以包括计算机可读存储介质，其上载有用于使处理器实现本发明的各个方面的计算机可读程序指令。
计算机可读存储介质可以是可以保持和存储由指令执行设备使用的指令的有形设备。计算机可读存储介质例如可以是但不限于电存储设备、磁存储设备、光存储设备、电磁存储设备、半导体存储设备或者上述的任意合适的组合。计算机可读存储介质的更具体的例子(非穷举的列表)包括：便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、静态随机存取存储器(SRAM)、 便携式压缩盘只读存储器(CD-ROM)、数字多功能盘(DVD)、记忆棒、软盘、机械编码设备、例如其上存储有指令的打孔卡或凹槽内凸起结构、以及上述的任意合适的组合。这里所使用的计算机可读存储介质不被解释为瞬时信号本身，诸如无线电波或者其他自由传播的电磁波、通过波导或其他传输媒介传播的电磁波(例如，通过光纤电缆的光脉冲)、或者通过电线传输的电信号。
这里所描述的计算机可读程序指令可以从计算机可读存储介质下载到各个计算/处理设备，或者通过网络、例如因特网、局域网、广域网和/或无线网下载到外部计算机或外部存储设备。网络可以包括铜传输电缆、光纤传输、无线传输、路由器、防火墙、交换机、网关计算机和/或边缘服务器。每个计算/处理设备中的网络适配卡或者网络接口从网络接收计算机可读程序指令，并转发该计算机可读程序指令，以供存储在各个计算/处理设备中的计算机可读存储介质中。
用于执行本发明操作的计算机程序指令可以是汇编指令、指令集架构(ISA)指令、机器指令、机器相关指令、微代码、固件指令、状态设置数据、或者以一种或多种编程语言的任意组合编写的源代码或目标代码，所述编程语言包括面向对象的编程语言—诸如Smalltalk、C++、Python等，以及常规的过程式编程语言—诸如“C”语言或类似的编程语言。计算机可读程序指令可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络—包括局域网(LAN)或广域网(WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。在一些实施例中，通过利用计算机可读程序指令的状态信息来个性化定制电子电路，例如可编程逻辑电路、现场可编程门阵列(FPGA)或可编程逻辑阵列(PLA)，该电子电路可以执行计算机可读程序指令，从而实现本发明的各个方面。
这里参照根据本发明实施例的方法、装置(系统)和计算机程序产品的流程图和/或框图描述了本发明的各个方面。应当理解，流程图和/或框图 的每个方框以及流程图和/或框图中各方框的组合，都可以由计算机可读程序指令实现。
这些计算机可读程序指令可以提供给通用计算机、专用计算机或其它可编程数据处理装置的处理器，从而生产出一种机器，使得这些指令在通过计算机或其它可编程数据处理装置的处理器执行时，产生了实现流程图和/或框图中的一个或多个方框中规定的功能/动作的装置。也可以把这些计算机可读程序指令存储在计算机可读存储介质中，这些指令使得计算机、可编程数据处理装置和/或其他设备以特定方式工作，从而，存储有指令的计算机可读介质则包括一个制造品，其包括实现流程图和/或框图中的一个或多个方框中规定的功能/动作的各个方面的指令。
也可以把计算机可读程序指令加载到计算机、其它可编程数据处理装置、或其它设备上，使得在计算机、其它可编程数据处理装置或其它设备上执行一系列操作步骤，以产生计算机实现的过程，从而使得在计算机、其它可编程数据处理装置、或其它设备上执行的指令实现流程图和/或框图中的一个或多个方框中规定的功能/动作。
附图中的流程图和框图显示了根据本发明的多个实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段或指令的一部分，所述模块、程序段或指令的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个连续的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或动作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。对于本领域技术人员来说公知的是，通过硬件方式实现、通过软件方式实现以及通过软件和硬件结合的方式实现都是等价的。
以上已经描述了本发明的各实施例，上述说明是示例性的，并非穷尽性的，并且也不限于所披露的各实施例。在不偏离所说明的各实施例的范围 和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。本文中所用术语的选择，旨在最好地解释各实施例的原理、实际应用或对市场中的技术改进，或者使本技术领域的其它普通技术人员能理解本文披露的各实施例。本发明的范围由所附权利要求来限定。

Claims (10)

  1. 一种基于三元组对比网络的机器人抓取预测方法，包括以下步骤：
    构建训练集，其中每条样本数据反映机器臂抓取物体过程中多个时刻的触觉数据与分类标签之间的对应关系，该分类标签用于指示抓取是否成功；
    基于设定的损失函数训练编码器，获得优化参数，其中在训练过程中，在所述训练集随机抽取两个标签一致的样本分别作为锚样本和正样本，并随机抽取一个标签相反的样本作为负样本；将样本数据分别输入对应编码器中编码得到锚样本的高维特征、正样本的高维特征以及负样本的高维特征；
    冻结所述编码器的优化参数，并利用经训练的编码器将输入样本数据编码为高维特征，进而将该高维特征输入分类器进行训练；
    利用经训练的编码器和经训练的分类器针对实时采集的触觉数据预测抓取结果。
  2. 根据权利要求1所述的方法，其特征在于，根据以下步骤构建所述训练集：
    采集机器臂抓取物体的触觉数据，包括三个时刻的触觉数据，分别是机器臂置于初始位置时标记为时刻T a，机器臂抓住目标物品但尚未提升时标记为时刻T b，抓住物品在空中悬停设定时间后标记为时刻T c；
    通过将T a、T b时刻的触觉数据图像进行随机翻转对数据进行增强，并将触觉数据随机裁剪为设定的尺寸，获得数据集；
    从所述数据集中随机选取设定数量的抓取物品种类，并针对每种物品随机选取设定数量的数据，构建为所述训练集。
  3. 根据权利要求1所述的方法，其特征在于，所述编码器采用三元对比损失函数作为评价指标进行训练，表示为：
    Figure PCTCN2022137001-appb-100001
    其中L T表示三元对比损失，f a、f p、f n分别表示锚样本、正样本和负样本的高维特征。
  4. 根据权利要求1所述的方法，其特征在于，利用二值交叉熵损失函数训练所述分类器，表示为：
    L B＝-(ylog(p(x)+(1-y)log(1-p(x))))
    其中，x表示输入的样本数据，y表示该样本实际上抓取成功或失败的 类别标签，p(x)表示在当前输入样本数据情况下，预测抓取成功的概率。
  5. 根据权利要求1所述的方法，其特征在于，所述编码器包括骨干网络，该骨干网络用于从G a与G b-G a在通道方向叠加的图像中提取高维特征，其中，G a、G b分别表示在T a、T b时刻采集到的触觉数据。
  6. 根据权利要求5所述的方法，其特征在于，所述骨干网络设置为多个，每个骨干网络用于从对应的触觉数据中提取高维特征，并将所提取的高维特征进行拼接后，输入到第一自注意力模块，得到融合高维特征。
  7. 根据权利要求6所述的方法，其特征在于，所述分类器包括第二自注意力模块和多层感知机，该多层感知机包含两层全连接层，其中第一层全连接层后接一层Relu非线性变换激活层；第二层全连接层输出神经元数量为2，后接一层Sigmoid非线性变换激活层，所述分类器输出预测抓取成功或失败的得分。
  8. 根据权利要求5所述的方法，其特征在于，所述骨干网络基于Resnet-50、VGG或AlexNet网络构建。
  9. 一种计算机可读存储介质，其上存储有计算机程序，其中，该计算机程序被处理器执行时实现根据权利要求1至8中任一项所述方法的步骤。
  10. 一种计算机设备，包括存储器和处理器，在所述存储器上存储有能够在处理器上运行的计算机程序，其特征在于，所述处理器执行所述计算机程序时实现权利要求1至8中任一项所述的方法的步骤。
PCT/CN2022/137001 2022-10-24 2022-12-06 一种基于三元组对比网络的机器人抓取预测方法 Ceased WO2024087331A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211303570.9A CN115519579B (zh) 2022-10-24 2022-10-24 一种基于三元组对比网络的机器人抓取预测方法
CN202211303570.9 2022-10-24

Publications (1)

Publication Number Publication Date
WO2024087331A1 true WO2024087331A1 (zh) 2024-05-02

Family

ID=84703947

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/137001 Ceased WO2024087331A1 (zh) 2022-10-24 2022-12-06 一种基于三元组对比网络的机器人抓取预测方法

Country Status (2)

Country Link
CN (1) CN115519579B (zh)
WO (1) WO2024087331A1 (zh)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118506113A (zh) * 2024-07-19 2024-08-16 武汉数聚速达网络科技有限责任公司 一种基于深度学习的图像识别模型训练方法及系统
CN119152503A (zh) * 2024-09-06 2024-12-17 浙江科聪控制技术有限公司 移动机器人的料笼识别系统及方法
CN119669539A (zh) * 2024-12-02 2025-03-21 天津大学 一种基于样本和标签联合嵌入的多维数据探索方法
CN119858159A (zh) * 2025-01-24 2025-04-22 浙江大学 基于安全约束机制的模仿学习的机器人细胞微操作方法
CN120635985A (zh) * 2025-06-10 2025-09-12 享刻智能技术(北京)有限公司 结合状态记忆机制的扩散策略模型优化方法及具身智能体
CN120663335A (zh) * 2025-08-25 2025-09-19 同济大学 用于机器人阻抗自适应的视触双通道控制硬件方法及系统
CN120735007A (zh) * 2025-06-27 2025-10-03 北京灵初智能科技有限公司 机械臂训练数据生成方法与训练方法、介质和机械臂系统

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116000929A (zh) * 2022-12-30 2023-04-25 上海交通大学 一种基于知识嵌入的服务机械臂任务为导向的抓取方法

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105005787A (zh) * 2015-06-24 2015-10-28 清华大学 一种基于灵巧手触觉信息的联合稀疏编码的材质分类方法
CN105956351A (zh) * 2016-07-05 2016-09-21 上海航天控制技术研究所 一种基于机器学习的触觉信息分类计算建模方法
US20190196436A1 (en) * 2017-12-21 2019-06-27 X Development Llc Robot interaction with objects based on semantic information associated with embedding spaces
CN110691676A (zh) * 2017-06-19 2020-01-14 谷歌有限责任公司 使用神经网络和几何感知对象表示的机器人抓取预测
CN111459278A (zh) * 2020-04-01 2020-07-28 中国科学院空天信息创新研究院 基于触觉阵列的机器人抓取状态判别方法
CN112668607A (zh) * 2020-12-04 2021-04-16 深圳先进技术研究院 一种用于目标物体触觉属性识别的多标签学习方法
CN112733965A (zh) * 2021-02-03 2021-04-30 西安理工大学 一种基于小样本学习的无标签图像分类方法
CN113168567A (zh) * 2018-10-29 2021-07-23 Hrl实验室有限责任公司 用于小样本转移学习的系统和方法
CN113327295A (zh) * 2021-06-18 2021-08-31 华南理工大学 一种基于级联全卷积神经网络的机器人快速抓取方法
CN113610151A (zh) * 2021-08-05 2021-11-05 哈尔滨理工大学 一种基于原型网络的自编码器网络机制解决小样本图像分类的系统、方法、设备及存储介质
CN115034286A (zh) * 2022-04-24 2022-09-09 国家计算机网络与信息安全管理中心 一种基于自适应损失函数的异常用户识别方法和装置

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12493792B2 (en) * 2019-10-24 2025-12-09 Nvidia Corporation Reinforcement learning of tactile grasp policies
CN114201581B (zh) * 2021-11-29 2024-06-14 中国科学院深圳先进技术研究院 一种基于对比学习的长文本检索模型

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105005787A (zh) * 2015-06-24 2015-10-28 清华大学 一种基于灵巧手触觉信息的联合稀疏编码的材质分类方法
CN105956351A (zh) * 2016-07-05 2016-09-21 上海航天控制技术研究所 一种基于机器学习的触觉信息分类计算建模方法
CN110691676A (zh) * 2017-06-19 2020-01-14 谷歌有限责任公司 使用神经网络和几何感知对象表示的机器人抓取预测
US20190196436A1 (en) * 2017-12-21 2019-06-27 X Development Llc Robot interaction with objects based on semantic information associated with embedding spaces
CN113168567A (zh) * 2018-10-29 2021-07-23 Hrl实验室有限责任公司 用于小样本转移学习的系统和方法
CN111459278A (zh) * 2020-04-01 2020-07-28 中国科学院空天信息创新研究院 基于触觉阵列的机器人抓取状态判别方法
CN112668607A (zh) * 2020-12-04 2021-04-16 深圳先进技术研究院 一种用于目标物体触觉属性识别的多标签学习方法
CN112733965A (zh) * 2021-02-03 2021-04-30 西安理工大学 一种基于小样本学习的无标签图像分类方法
CN113327295A (zh) * 2021-06-18 2021-08-31 华南理工大学 一种基于级联全卷积神经网络的机器人快速抓取方法
CN113610151A (zh) * 2021-08-05 2021-11-05 哈尔滨理工大学 一种基于原型网络的自编码器网络机制解决小样本图像分类的系统、方法、设备及存储介质
CN115034286A (zh) * 2022-04-24 2022-09-09 国家计算机网络与信息安全管理中心 一种基于自适应损失函数的异常用户识别方法和装置

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118506113A (zh) * 2024-07-19 2024-08-16 武汉数聚速达网络科技有限责任公司 一种基于深度学习的图像识别模型训练方法及系统
CN119152503A (zh) * 2024-09-06 2024-12-17 浙江科聪控制技术有限公司 移动机器人的料笼识别系统及方法
CN119669539A (zh) * 2024-12-02 2025-03-21 天津大学 一种基于样本和标签联合嵌入的多维数据探索方法
CN119858159A (zh) * 2025-01-24 2025-04-22 浙江大学 基于安全约束机制的模仿学习的机器人细胞微操作方法
CN120635985A (zh) * 2025-06-10 2025-09-12 享刻智能技术(北京)有限公司 结合状态记忆机制的扩散策略模型优化方法及具身智能体
CN120735007A (zh) * 2025-06-27 2025-10-03 北京灵初智能科技有限公司 机械臂训练数据生成方法与训练方法、介质和机械臂系统
CN120663335A (zh) * 2025-08-25 2025-09-19 同济大学 用于机器人阻抗自适应的视触双通道控制硬件方法及系统
CN120663335B (zh) * 2025-08-25 2025-11-04 同济大学 用于机器人阻抗自适应的视触双通道控制硬件方法及系统

Also Published As

Publication number Publication date
CN115519579B (zh) 2025-06-10
CN115519579A (zh) 2022-12-27

Similar Documents

Publication Publication Date Title
CN115519579B (zh) 一种基于三元组对比网络的机器人抓取预测方法
CN113378770B (zh) 手势识别方法、装置、设备、存储介质
CN115051929B (zh) 基于自监督目标感知神经网络的网络故障预测方法及装置
WO2023065859A1 (zh) 物品推荐方法、装置及存储介质
CN113971751A (zh) 训练特征提取模型、检测相似图像的方法和装置
CN115471771B (zh) 一种基于语义级时序关联建模的视频时序动作定位方法
CN113628059A (zh) 一种基于多层图注意力网络的关联用户识别方法及装置
CN114170688B (zh) 人物交互关系识别方法、装置、电子设备
CN110390305A (zh) 基于图卷积神经网络的手势识别的方法及装置
CN116523823A (zh) 用于半监督对象检测的鲁棒伪标签生成的系统和方法
CN114841338B (zh) 训练模型参数的方法、决策确定方法、装置及电子设备
EP4060553B1 (en) Systems, methods, and storage media for creating image data embeddings to be used for image recognition
CN112668607B (zh) 一种用于目标物体触觉属性识别的多标签学习方法
JP2026509806A (ja) 視覚－言語トランスフォーマーを事前訓練する方法及びこれを通じて事前訓練された視覚－言語トランスフォーマーを含む人工知能システム
CN112131886A (zh) 一种文本的方面级别情感分析方法
Chao et al. Deep learning‐based grasp‐detection method for a five‐fingered industrial robot hand
CN110111365B (zh) 基于深度学习的训练方法和装置以及目标跟踪方法和装置
CN116091513A (zh) 一种基于主动学习与适配算法的点云分割方法
JP2024028206A (ja) トポロジー認識ｃａｄ検索のための機械学習
US20260065053A1 (en) Data processing method and related apparatus
Annamalai et al. Facial matching and reconstruction techniques in identification of missing person using deep learning
CN119168835A (zh) 一种机械臂抓取预测方法、电子设备及存储介质
Shi et al. vmf-contact: Uncertainty-aware evidential learning for probabilistic contact-grasp in noisy clutter
Kim et al. COVID-19 outbreak prediction using Seq2Seq+ Attention and Word2Vec keyword time series data
Tong et al. Human intention recognition using context relationships in complex scenes

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22963309

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22963309

Country of ref document: EP

Kind code of ref document: A1