WO2023020103A1 - 目标检测模型的更新方法及装置 - Google Patents

目标检测模型的更新方法及装置 Download PDF

Info

Publication number
WO2023020103A1
WO2023020103A1 PCT/CN2022/099510 CN2022099510W WO2023020103A1 WO 2023020103 A1 WO2023020103 A1 WO 2023020103A1 CN 2022099510 W CN2022099510 W CN 2022099510W WO 2023020103 A1 WO2023020103 A1 WO 2023020103A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
target
target object
detection model
model
Prior art date
Application number
PCT/CN2022/099510
Other languages
English (en)
French (fr)
Inventor
刘伟峰
单纪昌
胡金龙
程云建
刘旭
Original Assignee
北京京东乾石科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京京东乾石科技有限公司 filed Critical 北京京东乾石科技有限公司
Priority to EP22857416.6A priority Critical patent/EP4336385A1/en
Publication of WO2023020103A1 publication Critical patent/WO2023020103A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/16—Programme controls
    • B25J9/1679—Programme controls characterised by the tasks executed
    • B25J9/1687—Assembly, peg and hole, palletising, straight line, weaving pattern movement
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/16—Programme controls
    • B25J9/1694—Programme controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
    • B25J9/1697—Vision controlled systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/40—Robotics, robotics mapping to robotics vision
    • G05B2219/40006—Placing, palletize, un palletize, paper roll placing, box stacking
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/40—Robotics, robotics mapping to robotics vision
    • G05B2219/40564—Recognize shape, contour of object, extract position and orientation
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/45—Nc applications
    • G05B2219/45063—Pick and place manipulator

Definitions

  • the embodiments of the present disclosure relate to the field of computer technology, and in particular to a method and device for updating a target detection model.
  • the picking task performed by the picking robot means that the picking robot picks the target object to the designated position on the basis of visual guidance.
  • High-precision target detection is the core technical foundation of this application scenario.
  • the target detection technology based on deep learning is a relatively practical technology at present.
  • the deep learning technology for target detection belongs to the category of supervised learning. The basic process is: (1) Collect scene data in batches in advance; (2) Manually label the data; (3) Train the target detection model; (4) Put the trained target
  • the detection model is deployed to an online application. It is understandable that object detection models generally have timeliness issues.
  • Embodiments of the present disclosure propose a method and device for updating a target detection model.
  • an embodiment of the present disclosure provides a method for updating a target detection model, including: constructing a three-dimensional model of the target according to image data of the target at multiple angles; The synthetic image of the target object; the synthetic image is used as a sample image, and the target object is used as a label to obtain training samples and generate a training sample set; the target detection model is trained through the training sample set to obtain an updated target detection model, wherein the target The detection model is used to characterize the correspondence between the input image and the detection result corresponding to the target object in the input image.
  • the above-mentioned construction of the three-dimensional model of the target according to the image data of the target at multiple angles includes: controlling the picking robot to perform During the process of picking the target object, the first image acquisition device collects the two-dimensional image data and three-dimensional image data of the target object at the preset information acquisition position at multiple angles; according to the two-dimensional image data and the three-dimensional image data, Build a 3D model of the target object.
  • the above-mentioned generation of a composite image including a target object representing a target object according to the three-dimensional model includes: determining the distance between the second image acquisition device corresponding to the picking robot, the preset information acquisition position, and the first image acquisition device Corresponding relationship of the coordinate system; according to the corresponding relationship of the coordinate system, determine the adjusted three-dimensional model of the three-dimensional model under the perspective of the second image acquisition device; according to the adjusted three-dimensional model and the preset background image representing the picking scene of the picking robot, generate a selection including A composite image of the target object corresponding to the target object.
  • the above method further includes: determining the weight of the target object; and generating a composite image including the target object corresponding to the target object according to the adjusted three-dimensional model and the preset background image representing the picking scene of the picking robot,
  • the method includes: generating a synthetic image including the target object corresponding to the target object according to the adjusted three-dimensional model, the preset background image, the weight, the preset resolution of the synthetic image, and the parameters of the second image acquisition device.
  • the above-mentioned training of the target detection model through the training sample set to obtain the updated target detection model includes: in response to determining that the detection accuracy of the target detection model is lower than a preset threshold, using a machine learning algorithm to pass the training sample set Train the target detection model to get the updated target detection model.
  • the above method further includes: using the updated target detection model to perform target detection on subsequent input images to obtain a detection result; and controlling the picking robot to perform the picking task according to the detection result.
  • an embodiment of the present disclosure provides an apparatus for updating a target detection model, including: a construction unit configured to construct a three-dimensional model of the target object according to image data of the target object at multiple angles; an image generation unit, It is configured to generate a synthetic image including a target object representing the target object according to the three-dimensional model; the sample generation unit is configured to use the synthetic image as a sample image and the target object as a label to obtain training samples and generate a training sample set; The update unit is configured to train the target detection model through the training sample set to obtain an updated target detection model, wherein the target detection model is used to represent the corresponding relationship between the input image and the detection result corresponding to the target object in the input image .
  • the construction unit is further configured to: in the process of controlling the picking robot to perform the picking task for the target object according to the detection result of the target object in the input image by the target detection model, through the first image acquisition
  • the device collects two-dimensional image data and three-dimensional image data of the target object at a preset information collection position under multiple angles; according to the two-dimensional image data and the three-dimensional image data, a three-dimensional model of the target object is constructed.
  • the image generation unit is further configured to: determine the coordinate system correspondence between the second image acquisition device corresponding to the picking robot, the preset information acquisition position, and the first image acquisition device; according to the coordinate system correspondence Determine the adjusted three-dimensional model of the three-dimensional model under the perspective of the second image acquisition device; generate a composite image including the target object corresponding to the target object according to the adjusted three-dimensional model and the preset background image representing the picking scene of the picking robot.
  • the above-mentioned apparatus further includes: a determining unit configured to determine the weight of the target object; and an image generating unit further configured to: according to the adjusted three-dimensional model, the preset background image, the weight, and the preset weight of the composite image.
  • the resolution and the parameters of the second image acquisition device are set to generate a composite image including the target object corresponding to the target object.
  • the update unit is further configured to: in response to determining that the detection accuracy of the target detection model is lower than a preset threshold, use a machine learning algorithm to train the target detection model through a training sample set to obtain an updated target detection model .
  • the above-mentioned device further includes: an obtaining unit configured to perform target detection on subsequent input images through an updated target detection model to obtain a detection result; an execution unit configured to control the picking robot to execute Pick a task.
  • an embodiment of the present disclosure provides a computer-readable medium on which a computer program is stored, wherein, when the program is executed by a processor, the method described in any implementation manner of the first aspect is implemented.
  • an embodiment of the present disclosure provides an electronic device, including: one or more processors; a storage device, on which one or more programs are stored, and when the one or more programs are executed by the one or more processors Execute, so that one or more processors implement the method described in any implementation manner of the first aspect.
  • the embodiments of the present disclosure provide a computer program product including a computer program.
  • the computer program When the computer program is executed by a processor, the method described in any implementation manner in the first aspect is implemented.
  • FIG. 1 is an exemplary system architecture diagram to which an embodiment of the present disclosure can be applied;
  • FIG. 2 is another exemplary system architecture diagram to which an embodiment of the present disclosure can be applied;
  • FIG. 3 is a flowchart of an embodiment of an update method of a target detection model according to the present disclosure
  • FIG. 4 is a schematic diagram of an application scenario of a method for updating a target detection model according to this embodiment
  • FIG. 5 is a flow chart of another embodiment of a method for updating a target detection model according to the present disclosure
  • Fig. 6 is a structural diagram of an embodiment of an updating device for a target detection model according to the present disclosure
  • FIG. 7 is a schematic structural diagram of a computer system suitable for implementing the embodiments of the present disclosure.
  • FIG. 1 shows an exemplary architecture 100 to which the object detection model updating method and device of the present disclosure can be applied.
  • the system architecture 100 may include image acquisition devices 101 , 102 , networks 103 , 104 , a control device 105 and a picking robot 106 .
  • Communication connections between the image acquisition devices 101, 102, the control device 105, and the picking robot 106 form a topological network, and the networks 103, 104 are used to provide communication links between the image acquisition devices 101, 102, the control device 105, and the picking robot 106 medium.
  • the networks 103, 104 may include various connection types, such as wire, wireless communication links, or fiber optic cables, among others.
  • the image acquisition devices 101 and 102 may be hardware devices or software having functions of 2D image acquisition, 3D image acquisition and information transmission.
  • the image acquisition devices 101, 102 are hardware, they can be various electronic devices that support network connection, image acquisition, interaction, display, processing and other functions, including but not limited to 2D cameras, 3D cameras, smart phones, tablet computers and Desktop computers and more.
  • the image acquisition devices 101 and 102 are software, they can be installed in the electronic devices listed above. It can be implemented, for example, as a plurality of software or software modules for providing distributed services, or as a single software or software module. No specific limitation is made here.
  • the control device 105 may be a server that provides various services, such as automatically generating training data and automatically updating the target detection model in the process of controlling the picking robot to perform the picking task according to the detection result of the target object in the image to be detected by the target detection model.
  • server may be a cloud server.
  • the control device 105 obtains the detection result of the target object in the image to be detected through the target detection model, and controls the picking robot to perform the picking task according to the detection result; Image data from multiple angles to construct a 3D model of the target to obtain training data, and then update the target detection model through the training data.
  • control device may be hardware or software.
  • control device When the control device is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or as a single server.
  • control device When the control device is software, it can be implemented as multiple software or software modules (such as software or software modules for providing distributed services), or as a single software or software module. No specific limitation is made here.
  • Picking robots can be various robots with picking functions, such as multi-degree-of-freedom robotic arms.
  • the method and device for updating the target detection model construct a three-dimensional model of the target based on the image data of the target at multiple angles; according to the three-dimensional model, a composite object including the target object representing the target is generated image; take the synthetic image as a sample image, and use the target object as a label to obtain training samples and generate a training sample set; train the target detection model through the training sample set to obtain an updated target detection model, thereby providing an automatic generation training
  • the data and the method of automatically updating the target detection model improve the convenience of updating the target detection model and the accuracy of the detection results.
  • FIG. 2 another exemplary architecture 200 of an update system for a target detection model is shown, including a first image acquisition device 201, a second image acquisition device 202, a third image acquisition device 203, a control device 204 and Picking robot 205 .
  • the third image acquisition device 203 is arranged at the picking station, and the turnover box containing the target object to be picked is placed at the picking station, and the third image acquisition device 203 is used to collect the image of the target object corresponding to the target object in the turnover box .
  • the control device 204 is used to use the image collected by the third image acquisition device 203 as the input image of the target detection model to obtain the detection result, and then control the picking robot to perform the picking task (transfer the target object from the picking station to the palletizing station) according to the detection result. bits).
  • the second image acquisition device 202 is correspondingly arranged on the picking robot.
  • the first image collection device 201 is set at a preset information collection position. Its preset information collection position can be any position that the picking robot passes through during the moving process of picking objects. For example, the preset position is set close to the picking station where the objects to be picked are placed.
  • the first image acquisition device 201 is used to capture objects from multiple angles in response to determining that the objects picked by the picking robot move to a preset information collection position during the process of the picking robot picking the objects from the picking station to the palletizing position.
  • the image data below; the control device 204 is further used to construct a three-dimensional model of the target object according to the image data of the target object under multiple angles, so as to generate training data and update the target detection model.
  • the method for updating the target detection model provided by the embodiments of the present disclosure can be executed by the control device.
  • each part (such as each unit) included in the target detection model update device can be all set in the control device.
  • the numbers of image acquisition devices, networks, control devices and picking robots in Fig. 1 are only schematic. According to the realization requirements, there may be any number of image acquisition devices, networks, control devices and picking robots.
  • the system architecture may only include the electronic device on which the method for updating the target detection model runs (eg, a control device).
  • a flow 300 of an embodiment of a method for updating a target detection model is shown, including the following steps:
  • Step 301 constructing a three-dimensional model of the target according to the image data of the target at multiple angles.
  • the execution subject of the method for updating the target detection model can remotely or locally acquire the image data of the target at multiple angles through a wired connection network or a wireless connection network. , and then construct a 3D model of the target based on the image data of the target at multiple angles.
  • the target detection model is used to characterize the correspondence between the input image and the corresponding detection results of the target object in the input image, which can be obtained based on neural network model training with target detection function, including but not limited to convolutional neural network, residual Differential neural network, recurrent neural network. It should be noted that the object detection model is a pre-trained model that has been applied in the picking scene.
  • Inputting the image to be detected into the target detection model can determine the detection frame indicating the target in the image to be detected, and then according to the calibration information of the image acquisition device that acquires the image to be detected (such as the third image acquisition device 203 in Figure 2), Determine the location information of objects at picking stations in real-world environments. Furthermore, the picking robot can be controlled to pick the target at the position represented by the determined position information.
  • the target object may be various physical objects.
  • the target objects are various commodities.
  • the image data includes 2D (2-dimension, two-dimensional) image data and 3D image data.
  • the 2D image data is an RGB (red, green, blue, red, green, blue) image
  • the 3D image data is point cloud image data.
  • each group of cameras includes a 2D camera and a 3D camera.
  • the execution subject can input the image data of the target object from multiple angles into a shared network model for 3D modeling (for example, Reality Capture issued by Capturing Reality Company), to obtain a 3D model of the target object.
  • a shared network model for 3D modeling for example, Reality Capture issued by Capturing Reality Company
  • the above-mentioned execution subject may perform the above-mentioned step 301 in the following manner: First, according to the detection result of the target object in the input image by the target detection model, control the picking robot to execute the target In the process of picking the object, the first image acquisition device collects the two-dimensional image data and three-dimensional image data of the target object at the preset information acquisition position under multiple angles; then, according to the two-dimensional image data and the three-dimensional image data , to build a 3D model of the target object.
  • a preset information collection position is set between the picking station and the palletizing position, and a first image collection device including multiple sets of cameras is set around the preset information collection position.
  • the target object is temporarily placed at the preset information collection position for the second
  • An image acquisition device acquires two-dimensional image data and three-dimensional image data of the target object under multiple angles.
  • Step 302 Generate a composite image including a target object representing the target object according to the three-dimensional model.
  • the execution subject may generate a composite image including the target object representing the target object according to the three-dimensional model.
  • the above execution subject inputs the 3D model of the target object into an image synthesis tool (such as Keyshot), sets a background image for the 3D model, and obtains a synthesized image.
  • an image synthesis tool such as Keyshot
  • sets a background image for the 3D model and obtains a synthesized image.
  • multiple composite images may be generated for the same three-dimensional model of the target object.
  • the above execution subject can perform the above step 302 in the following manner:
  • the second image acquisition device, the preset information acquisition position and the first image acquisition device are sequentially provided with corresponding coordinate systems.
  • the execution subject can determine the coordinate system correspondence between the second image acquisition device corresponding to the picking robot, the preset information acquisition position, and the first image acquisition device through a calibration algorithm.
  • the adjusted three-dimensional model of the three-dimensional model under the viewing angle of the second image acquisition device is determined according to the coordinate system correspondence.
  • the three-dimensional model is adjusted to the perspective of the second image acquisition device corresponding to the picking robot, so as to fully fit the real scene of the picking robot workstation.
  • a composite image including the target object corresponding to the target object is generated.
  • the objects at the picking station are generally placed in turnover boxes, and the preset background image may be a background image including turnover box objects corresponding to empty turnover boxes. In this way, the realism of the synthesized image can be further improved.
  • the execution subject may also determine the weight of the target object.
  • a weight sensor is set at a preset information collection position to obtain the weight of the target object.
  • the above-mentioned executive body may perform the above-mentioned third step in the following manner:
  • a composite image including the target object corresponding to the target object is generated.
  • the preset resolution is used to characterize the resolution of the desired synthesized image, and the parameters of the second image acquisition device represent its internal reference data.
  • the above-mentioned execution subject inputs data such as the adjusted 3D model, the preset background image, weight, the preset resolution of the synthesized image, and the parameters of the second image acquisition device into the virtual physics engine tool (such as pybullet) as input data, and obtains composite image.
  • the virtual physics engine tool such as pybullet
  • the generated composite image may include multiple targets of the same type, or may include multiple targets of different types.
  • the different kinds of objects may be a variety of objects picked so far.
  • the adjusted three-dimensional model and weight information of each target obtained by the above-mentioned executive body can obtain composite images of multiple target objects of different types through the virtual physics engine tool, so as to The composed training data is trained to obtain a target detection model with higher detection accuracy.
  • Step 303 using the synthesized image as a sample image and the target object as a label to obtain training samples and generate a training sample set.
  • the execution subject may synthesize images as sample images, use the target object as a label, obtain training samples, and generate a training sample set.
  • the synthetic image is synthesized based on the 3D model, and the target objects can be obtained naturally, thus eliminating the need for manual labeling of labels, and expanding the training sample set for updating the target detection model.
  • Step 304 training the target detection model through the training sample set to obtain an updated target detection model.
  • the execution subject may train the target detection model through the training sample set to obtain an updated target detection model.
  • untrained training samples are selected from the training sample set, the sample images in the selected training samples are input into the target detection model, and the labels corresponding to the input training samples are used as the expected output to obtain the target detection model. actual output. Then, calculate the loss between the actual output and the label. Finally, the gradient is calculated according to the loss, and the parameters of the target detection model are updated based on the gradient descent method and the stochastic gradient descent method.
  • the execution subject may execute the update process cyclically until the preset end condition is obtained, and an updated target detection model is obtained.
  • the preset end condition may be, for example, that the number of training times exceeds a preset number of times threshold, the training time exceeds a preset time threshold, and the loss tends to converge.
  • the above execution subject may perform the above step 304 in the following manner:
  • a machine learning algorithm is used to train the target detection model through the training sample set to obtain an updated target detection model.
  • the preset threshold may be specifically set according to actual conditions. As an example, when the detection accuracy required by the picking task is high, a higher preset threshold may be set; when the detection accuracy required by the picking task is not high, a lower preset threshold may be set.
  • the above execution subject can input the synthesized image into the unupdated target detection model to obtain the detection frame of the target object actually output by the target detection model; and then determine the IoU (Intersection -over-Union, intersection and ratio), average the IoU corresponding to multiple synthetic images to determine the detection accuracy of the target detection model.
  • IoU Intersection -over-Union, intersection and ratio
  • the execution subject uses the updated target detection model to perform target detection on the subsequent input image to obtain the detection result; control according to the detection result Picking robots perform picking tasks.
  • FIG. 4 is a schematic diagram 400 of an application scenario of the method for updating a target detection model according to this embodiment.
  • the application scenario in FIG. 4 it includes an image acquisition device 401 , an image acquisition device 402 , a server 403 and a picking robot 404 , and an object detection model is set in the server 403 .
  • the image acquisition device 401 is installed at the sorting station, captures the target at the picking station to obtain the image to be detected, and performs target detection through the target detection model to obtain the detection result, and then controls the picking robot to pick the target and move it to the pallet bit.
  • the image acquisition device 402 collects the image data of the objects at the preset information acquisition position under multiple angles.
  • the server first constructs a 3D model of the target object based on the image data of the target object at multiple angles; then, according to the 3D model, generates a composite image including the target object representing the target object ; Using the synthetic image as a sample image and the target object as a label to obtain a training sample and generate a training sample set; then, train the target detection model through the training sample set to obtain an updated target detection model.
  • the three-dimensional model of the target is constructed according to the image data of the target at multiple angles; according to the three-dimensional model, a composite image including the target object representing the target is generated; to synthesize the image Take the target object as a sample image, obtain training samples, and generate a training sample set; train the target detection model through the training sample set, and obtain an updated target detection model, thereby providing a method for automatically generating training data and automatically updating targets.
  • the detection model method improves the convenience of updating the target detection model and the accuracy of detection results.
  • FIG. 5 a schematic flow 500 of an embodiment of a method for updating a target detection model according to the present disclosure is shown, including the following steps:
  • Step 501 during the process of controlling the picking robot to perform the picking task for the target object according to the detection result of the target object in the input image by the target detection model, the target object at the preset information collection position is captured by the first image acquisition device 2D image data, 3D image data and weights at multiple angles.
  • Step 502 construct a 3D model of the target object according to the 2D image data and the 3D image data.
  • Step 503 determine the coordinate system correspondence between the second image acquisition device corresponding to the picking robot, the preset information acquisition position, and the first image acquisition device.
  • Step 504 according to the coordinate system correspondence, determine the adjusted three-dimensional model of the three-dimensional model under the viewing angle of the second image acquisition device.
  • Step 505 according to the adjusted 3D model, preset background image, weight, preset resolution of the composite image, and parameters of the second image acquisition device, generate a composite image including the target object corresponding to the target object.
  • Step 506 using the synthesized image as a sample image and the target object as a label to obtain training samples and generate a training sample set.
  • Step 507 train the target detection model through the training sample set to obtain an updated target detection model.
  • the target detection model is used to characterize the corresponding relationship between the input image and the detection result corresponding to the target object in the input image.
  • step 508 target detection is performed on subsequent input images through the updated target detection model to obtain a detection result.
  • Step 509 control the picking robot to perform the picking task according to the detection result.
  • the process 500 of the method for updating the target detection model in this embodiment specifically illustrates the construction process of the composite image and the application of the updated target detection model.
  • the process provides a closed-loop continuous learning method for the target detection model, and builds a training sample set based on the construction of a 3D model of the target, which is suitable for targets of all shapes and improves the universality and detection of the target detection model. precision.
  • the present disclosure provides an embodiment of an object detection model updating device, which corresponds to the method embodiment shown in FIG. 3 .
  • the device can be specifically applied to various electronic devices.
  • the device for updating the target detection model includes: a construction unit 601 configured to construct a three-dimensional model of the target according to image data of the target at multiple angles; an image generation unit 602 configured to The model generates a synthetic image including a target object representing the target object; the sample generation unit 603 is configured to use the synthetic image as a sample image and the target object as a label to obtain a training sample and generate a training sample set; the update unit 604, It is configured to train the target detection model through the training sample set to obtain an updated target detection model, wherein the target detection model is used to characterize the corresponding relationship between the input image and the detection result corresponding to the target object in the input image.
  • the construction unit 601 is further configured to: according to the detection result of the target object in the input image by the target detection model, control the picking robot to perform the picking task for the target During the process, the first image acquisition device collects two-dimensional image data and three-dimensional image data of the target object at the preset information acquisition position under multiple angles; according to the two-dimensional image data and three-dimensional image data, the three-dimensional model of the target object is constructed .
  • the image generation unit 602 is further configured to: determine the coordinate system between the second image acquisition device corresponding to the picking robot, the preset information acquisition position, and the first image acquisition device Correspondence: According to the correspondence of the coordinate system, determine the adjusted three-dimensional model of the three-dimensional model under the perspective of the second image acquisition device; according to the adjusted three-dimensional model and the preset background image representing the picking scene of the picking robot, generate a corresponding Composite image of the target object.
  • the above-mentioned device further includes: a determining unit (not shown in the figure), configured to determine the weight of the target object; and an image generating unit 602, further configured to: according to the adjusted After the three-dimensional model, the preset background image, the weight, the preset resolution of the synthesized image, and the parameters of the second image acquisition device, a synthesized image including the target object corresponding to the target object is generated.
  • a determining unit not shown in the figure
  • an image generating unit 602 further configured to: according to the adjusted After the three-dimensional model, the preset background image, the weight, the preset resolution of the synthesized image, and the parameters of the second image acquisition device, a synthesized image including the target object corresponding to the target object is generated.
  • the update unit 604 is further configured to: in response to determining that the detection accuracy of the target detection model is lower than a preset threshold, use a machine learning algorithm to train the target detection model through the training sample set , to get the updated object detection model.
  • the above-mentioned device further includes: an obtaining unit (not shown in the figure), configured to perform target detection on subsequent input images through an updated target detection model, and obtain a detection result
  • the execution unit (not shown in the figure) is configured to control the picking robot to perform the picking task according to the detection result.
  • the construction unit in the update device of the target detection model constructs a three-dimensional model of the target according to the image data of the target at multiple angles; the image generation unit generates the target object including the representative target according to the three-dimensional model synthetic image; the sample generation unit uses the synthetic image as a sample image and the target object as a label to obtain training samples and generate a training sample set; the update unit trains the target detection model through the training sample set to obtain an updated target detection model, wherein , the target detection model is used to represent the correspondence between the input image and the corresponding detection results of the target object in the input image, thus providing a device for automatically generating training data and automatically updating the target detection model, which improves the target detection model The convenience of updates and the accuracy of test results.
  • FIG. 7 it shows a schematic structural diagram of a computer system 700 suitable for implementing the devices of the embodiments of the present disclosure (such as the devices 101 , 102 , 105 , and 106 shown in FIG. 1 ).
  • the device shown in FIG. 7 is only an example, and should not limit the functions and scope of use of the embodiments of the present disclosure.
  • a computer system 700 includes a processor (such as a CPU, central processing unit) 701, which can be loaded into a random access memory (RAM) according to a program stored in a read-only memory (ROM) 702 or from a storage section 708.
  • the program in 703 performs various appropriate actions and processing.
  • various programs and data necessary for the operation of the system 700 are also stored.
  • the processor 701 , ROM 702 and RAM 703 are connected to each other via a bus 704 .
  • An input/output (I/O) interface 705 is also connected to the bus 704 .
  • the following components are connected to the I/O interface 705: an input section 706 including a keyboard, a mouse, etc.; an output section 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage section 708 including a hard disk, etc. and a communication section 709 including a network interface card such as a LAN card, a modem, or the like.
  • the communication section 709 performs communication processing via a network such as the Internet.
  • a drive 710 is also connected to the I/O interface 705 as needed.
  • a removable medium 711 such as a magnetic disk, optical disk, magneto-optical disk, semiconductor memory, etc. is mounted on the drive 710 as necessary so that a computer program read therefrom is installed into the storage section 708 as necessary.
  • embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, where the computer program includes program codes for executing the methods shown in the flowcharts.
  • the computer program may be downloaded and installed from a network via communication portion 709 and/or installed from removable media 711 .
  • the computer program is executed by the processor 701, the above-mentioned functions defined in the method of the present disclosure are performed.
  • the computer-readable medium in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two.
  • a computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable Programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.
  • a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
  • a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave carrying computer-readable program code therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.
  • a computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.
  • Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
  • Computer program code for carrying out the operations of the present disclosure can be written in one or more programming languages, or combinations thereof, including object-oriented programming languages—such as Java, Smalltalk, C++, and conventional procedural programming language—such as "C" or a similar programming language.
  • the program code may execute entirely on the client computer, partly on the client computer, as a stand-alone software package, partly on the client computer and partly on a remote computer or entirely on the remote computer or server.
  • the remote computer can be connected to the client computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (such as through an Internet service provider). Internet connection).
  • LAN local area network
  • WAN wide area network
  • Internet service provider such as AT&T, MCI, Sprint, EarthLink, MSN, GTE, etc.
  • each block in a flowchart or block diagram may represent a module, program segment, or portion of code that contains one or more logical functions for implementing specified executable instructions.
  • the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved.
  • each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations can be implemented by a dedicated hardware-based system that performs the specified functions or operations , or may be implemented by a combination of dedicated hardware and computer instructions.
  • the units involved in the embodiments described in the present disclosure may be implemented by software or by hardware.
  • the described units may also be set in a processor, for example, may be described as: a processor including a construction unit, an image generation unit, a sample generation unit, and an update unit.
  • a processor including a construction unit, an image generation unit, a sample generation unit, and an update unit.
  • the names of these units do not constitute a limitation of the unit itself in some cases, for example, the update unit can also be described as "the unit that trains the target detection model through the training sample set and obtains the updated target detection model" .
  • the present disclosure also provides a computer-readable medium, which may be included in the device described in the above embodiments, or may exist independently without being assembled into the device.
  • the above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by the device, the computer device: constructs a three-dimensional model of the target object according to image data of the target object at multiple angles; According to the three-dimensional model, generate a synthetic image including the target object representing the target object; use the synthetic image as a sample image, and use the target object as a label to obtain training samples and generate a training sample set; train the target detection model through the training sample set, and obtain The updated target detection model, wherein the target detection model is used to characterize the correspondence between the input image and the detection result corresponding to the target object in the input image.

Abstract

本公开公开了一种目标检测模型的更新方法及装置。方法的一具体实施方式包括：根据目标物在多个角度下的图像数据，构建目标物的三维模型；根据三维模型，生成包括表征目标物的目标物对象的合成图像；以合成图像为样本图像，以目标物对象为标签，得到训练样本，生成训练样本集；通过训练样本集训练目标检测模型，得到更新后的目标检测模型。

Description

目标检测模型的更新方法及装置
相关申请的交叉引用
本专利申请要求于2021年08月17日提交的、申请号为202110957524.X、发明名称为“目标检测模型的更新方法及装置”的中国专利申请的优先权，该申请的全文以引用的方式并入本申请中。
技术领域
本公开实施例涉及计算机技术领域，具体涉及一种目标检测模型的更新方法及装置。
背景技术
在智能仓储自动化领域，拣选机器人执行拣选任务的方式具有较好的应用前景。拣选机器人执行拣选任务是指拣选机器人在视觉引导的基础上，把目标物拣选到指定位置。高精度的目标检测是该应用场景的核心技术基础。基于深度学习的目标检测技术是目前比较实用的技术。针对目标检测的深度学习技术属于监督学习范畴，基本过程是：(1)事先批量采集场景数据；(2)对数据进行人工标注；(3)训练目标检测模型；(4)把训练好的目标检测模型部署到线上应用。可以理解，目标检测模型一般会产生时效性问题。例如，电商场景中，商品更新换代频繁，基于早期数据训练好的目标检测模型的精度会随着时间推移而退化。如果重新执行上述目标检测模型的得到过程，虽然可以使模型再度恢复精度，但其过程费时费力。
发明内容
本公开实施例提出了一种目标检测模型的更新方法及装置。
第一方面，本公开实施例提供了一种目标检测模型的更新方法，包括：根据目标物在多个角度下的图像数据，构建目标物的三维模型；根据三维模型，生成包括表征目标物的目标物对象的合成图像；以合成图像为样本图像，以目标物对象为标签，得到训练样本，生成训练 样本集；通过训练样本集训练目标检测模型，得到更新后的目标检测模型，其中，目标检测模型用于表征输入图像与输入图像中的目标物对象对应的检测结果之间的对应关系。
在一些实施例中，上述根据目标物在多个角度下的图像数据，构建目标物的三维模型，包括：在根据目标检测模型对输入图像中的目标物对象的检测结果，控制拣选机器人执行对于目标物的拣选任务的过程中，通过第一图像采集装置采集处于预设信息采集位置的目标物在多个角度下的二维图像数据和三维图像数据；根据二维图像数据和三维图像数据，构建目标物的三维模型。
在一些实施例中，上述根据三维模型，生成包括表征目标物的目标物对象的合成图像，包括：确定拣选机器人对应的第二图像采集装置、预设信息采集位置和第一图像采集装置之间的坐标系对应关系；根据坐标系对应关系，确定三维模型在第二图像采集装置的视角下的调整后三维模型；根据调整后三维模型和表征拣选机器人的拣选场景的预设背景图像，生成包括目标物对应的目标物对象的合成图像。
在一些实施例中，上述方法还包括：确定目标物的重量；以及上述根据调整后三维模型和表征拣选机器人的拣选场景的预设背景图像，生成包括目标物对应的目标物对象的合成图像，包括：根据调整后三维模型、预设背景图像、重量、合成图像的预设分辨率、第二图像采集装置的参数，生成包括目标物对应的目标物对象的合成图像。
在一些实施例中，上述通过训练样本集训练目标检测模型，得到更新后的目标检测模型，包括：响应于确定目标检测模型的检测精度低于预设阈值，利用机器学习算法，通过训练样本集训练目标检测模型，得到更新后的目标检测模型。
在一些实施例中，上述方法还包括：通过更新后的目标检测模型对后续的输入图像进行目标检测，得到检测结果；根据检测结果控制拣选机器人执行拣选任务。
第二方面，本公开实施例提供了一种目标检测模型的更新装置，包括：构建单元，被配置成根据目标物在多个角度下的图像数据，构建目标物的三维模型；图像生成单元，被配置成根据三维模型，生成 包括表征目标物的目标物对象的合成图像；样本生成单元，被配置成以合成图像为样本图像，以目标物对象为标签，得到训练样本，生成训练样本集；更新单元，被配置成通过训练样本集训练目标检测模型，得到更新后的目标检测模型，其中，目标检测模型用于表征输入图像与输入图像中的目标物对象对应的检测结果之间的对应关系。
在一些实施例中，构建单元，进一步被配置成：在根据目标检测模型对输入图像中的目标物对象的检测结果，控制拣选机器人执行对于目标物的拣选任务的过程中，通过第一图像采集装置采集处于预设信息采集位置的目标物在多个角度下的二维图像数据和三维图像数据；根据二维图像数据和三维图像数据，构建目标物的三维模型。
在一些实施例中，图像生成单元，进一步被配置成：确定拣选机器人对应的第二图像采集装置、预设信息采集位置和第一图像采集装置之间的坐标系对应关系；根据坐标系对应关系，确定三维模型在第二图像采集装置的视角下的调整后三维模型；根据调整后三维模型和表征拣选机器人的拣选场景的预设背景图像，生成包括目标物对应的目标物对象的合成图像。
在一些实施例中，上述装置还包括：确定单元，被配置成确定目标物的重量；以及图像生成单元，进一步被配置成：根据调整后三维模型、预设背景图像、重量、合成图像的预设分辨率、第二图像采集装置的参数，生成包括目标物对应的目标物对象的合成图像。
在一些实施例中，更新单元，进一步被配置成：响应于确定目标检测模型的检测精度低于预设阈值，利用机器学习算法，通过训练样本集训练目标检测模型，得到更新后的目标检测模型。
在一些实施例中，上述装置还包括：得到单元，被配置成通过更新后的目标检测模型对后续的输入图像进行目标检测，得到检测结果；执行单元，被配置成根据检测结果控制拣选机器人执行拣选任务。
第三方面，本公开实施例提供了一种计算机可读介质，其上存储有计算机程序，其中，程序被处理器执行时实现如第一方面任一实现方式描述的方法。
第四方面，本公开实施例提供了一种电子设备，包括：一个或多 个处理器；存储装置，其上存储有一个或多个程序，当一个或多个程序被一个或多个处理器执行，使得一个或多个处理器实现如第一方面任一实现方式描述的方法。
第五方面，本公开实施例提供了一种包括计算机程序的计算机程序产品，该计算机程序在被处理器执行时实现如第一方面中任一实现方式描述的方法。
附图说明
通过阅读参照以下附图所作的对非限制性实施例所作的详细描述，本公开的其它特征、目的和优点将会变得更明显：
图1是本公开的一个实施例可以应用于其中的示例性系统架构图；
图2是本公开的一个实施例可以应用于其中的又一示例性系统架构图；
图3是根据本公开目标检测模型的更新方法的一个实施例的流程图；
图4是根据本实施例的目标检测模型的更新方法的应用场景的示意图；
图5是根据本公开的目标检测模型的更新方法的又一个实施例的流程图；
图6是根据本公开的目标检测模型的更新装置的一个实施例的结构图；
图7是适于用来实现本公开实施例的计算机系统的结构示意图。
具体实施方式
下面结合附图和实施例对本公开作进一步的详细说明。可以理解的是，此处所描述的具体实施例仅仅用于解释相关发明，而非对该发明的限定。另外还需要说明的是，为了便于描述，附图中仅示出了与有关发明相关的部分。
需要说明的是，在不冲突的情况下，本公开中的实施例及实施例中的特征可以相互组合。下面将参考附图并结合实施例来详细说明本 公开。
图1示出了可以应用本公开的目标检测模型的更新方法及装置的示例性架构100。
如图1所示，系统架构100可以包括图像采集装置101、102，网络103、104，控制装置105和拣选机器人106。图像采集装置101、102与控制装置105、拣选机器人106之间通信连接构成拓扑网络，网络103、104用以在图像采集装置101、102与控制装置105、拣选机器人106之间提供通信链路的介质。网络103、104可以包括各种连接类型，例如有线、无线通信链路或者光纤电缆等等。
图像采集装置101、102可以是具有2D图像采集功能、3D图像采集功能和信息传输功能的硬件设备或软件。当图像采集装置101、102为硬件时，其可以是支持网络连接，图像获取、交互、显示、处理等功能的各种电子设备，包括但不限于2D相机、3D相机、智能手机、平板电脑和台式计算机等等。当图像采集装置101、102为软件时，可以安装在上述所列举的电子设备中。其可以实现成例如用来提供分布式服务的多个软件或软件模块，也可以实现成单个软件或软件模块。在此不做具体限定。
控制装置105可以是提供各种服务的服务器，例如在根据目标检测模型对待检测图像中的目标物的检测结果，控制拣选机器人执行拣选任务的过程中，自动生成训练数据、自动更新目标检测模型的服务器。作为示例，控制装置105可以是云端服务器。具体的，控制装置105通过目标检测模型得到对待检测图像中的目标物的检测结果，根据检测结果控制拣选机器人执行拣选任务；并在拣选机器人执行拣选任务的过程中，获取所拣选的目标物在多个角度下的图像数据，构建目标物的三维模型，以得到训练数据，进而通过训练数据更新目标检测模型。
需要说明的是，控制装置可以是硬件，也可以是软件。当控制装置为硬件时，可以实现成多个服务器组成的分布式服务器集群，也可以实现成单个服务器。当控制装置为软件时，可以实现成多个软件或软件模块(例如用来提供分布式服务的软件或软件模块)，也可以实现 成单个软件或软件模块。在此不做具体限定。
拣选机器人可以是具有拣选功能的各种机器人，例如多自由度机械臂。
本公开实施例提供的目标检测模型的更新方法及装置，通过根据目标物在多个角度下的图像数据，构建目标物的三维模型；根据三维模型，生成包括表征目标物的目标物对象的合成图像；以合成图像为样本图像，以目标物对象为标签，得到训练样本，生成训练样本集；通过训练样本集训练目标检测模型，得到更新后的目标检测模型，从而提供了一种自动生成训练数据、自动更新目标检测模型的方法，提高了目标检测模型更新的便捷性和检测结果的准确度。
如图2所示，示出了目标检测模型的更新系统的又一个示例性架构200，包括，第一图像采集装置201、第二图像采集装置202、第三图像采集装置203、控制装置204和拣选机器人205。
第三图像采集装置203设置于拣选工位处，拣选工位处放置装有待拣选的目标物的周转箱，第三图像采集装置203用于采集包括周转箱中的目标物对应的目标对象的图像。控制装置204用于将第三图像采集装置203采集到的图像作为目标检测模型的输入图像，得到检测结果，进而控制拣选机器人根据检测结果执行拣选任务(将目标物由拣选工位转移至码垛位)。
第二图像采集装置202对应设置于拣选机器人上。第一图像采集装置201设置于预设信息采集位置。其预设信息采集位置可以是拣选机器人拣选目标物的移动过程所经过的任意位置。例如，预设位置靠近放置待拣选的目标物的拣选工位设置。第一图像采集装置201用于在拣选机器人将目标物从拣选工位拣选至码垛位的过程中，响应于确定拣选机器人拣选目标物移动至预设信息采集位置，获取目标物在多个角度下的图像数据；控制装置204进一步用于根据目标物在多个角度下的图像数据构建目标物的三维模型，以生成训练数据，更新目标检测模型。
本公开的实施例所提供的目标检测模型的更新方法可以由控制装 置执行，相应地，目标检测模型的更新装置包括的各个部分(例如各个单元)可以全部设置于控制装置中。
应该理解，图1中的图像采集装置、网络、控制装置和拣选机器人的数目仅仅是示意性的。根据实现需要，可以具有任意数目的图像采集装置、网络、控制装置和拣选机器人。当目标检测模型的更新方法运行于其上的电子设备不需要与其他电子设备进行数据传输时，该系统架构可以仅包括目标检测模型的更新方法运行于其上的电子设备(例如控制装置)。
继续参考图3，示出了目标检测模型的更新方法的一个实施例的流程300，包括以下步骤：
步骤301，根据目标物在多个角度下的图像数据，构建目标物的三维模型。
本实施例中，目标检测模型的更新方法的执行主体(例如图1中的终端设备或服务器)可以通过有线连接网络或无线连接网络从远程或从本地获取目标物在多个角度下的图像数据，进而根据目标物在多个角度下的图像数据，构建目标物的三维模型。
目标检测模型用于表征输入图像与输入图像中的目标物对象对应的检测结果之间的对应关系，可以基于具有目标物检测功能的神经网络模型训练得到，包括但不限于卷积神经网络、残差神经网络、循环神经网络。需要说明的是，目标检测模型为预先训练的、已投入拣选场景中应用的模型。
将待检测图像输入目标检测模型，可以确定出指示待检测图像中目标物的检测框，进而根据获取待检测图像的图像采集装置(如图2中的第三图像采集装置203)的标定信息，确定现实环境中拣选工位上目标物的位置信息。进而，可以控制拣选机器人拣选处于所确定的位置信息所表征的位置处的目标物。
其中，目标物可以是各种实体物物品。作为示例，目标物是各种商品。
图像数据包括2D(2-dimension，二维)图像数据和3D图像数据。 作为示例，2D图像数据是RGB(red，green，blue，红色，绿色，蓝色)图像，3D图像数据为点云图像数据。
本实施例中，可以通过处于不同位置的多组相机，拍摄得到目标物在多个角度下的图像数据。其中，每组相机包括2D相机和3D相机。
本实施例中，上述执行主体可以将目标物在多个角度下的图像数据输入用于3D建模的享有网络模型(例如，Capturing Reality公司发布Reality Capture)，得到目标物的三维模型。
在本实施例的一些可选的实现方式中，上述执行主体可以通过如下方式执行上述步骤301：首先，在根据目标检测模型对输入图像中的目标物对象的检测结果，控制拣选机器人执行对于目标物的拣选任务的过程中，通过第一图像采集装置采集处于预设信息采集位置的目标物在多个角度下的二维图像数据和三维图像数据；然后，根据二维图像数据和三维图像数据，构建目标物的三维模型。
具体的，在拣选工位和码垛位之间设置预设信息采集位置，环绕预设信息采集位置设置包括多组相机的第一图像采集装置。在拣选机器人根据目标检测模型的检测结果在拣选工位拣选目标物向码垛位移动的过程中，响应于到达预设信息采集位置，将目标物暂时放置于预设信息采集位置，以供第一图像采集装置采集得到目标物在多个角度下的二维图像数据和三维图像数据。
步骤302，根据三维模型，生成包括表征目标物的目标物对象的合成图像。
本实施例中，上述执行主体可以根据三维模型，生成包括表征目标物的目标物对象的合成图像。
作为示例，上述执行主体将目标物的三维模型输入图像合成工具(例如Keyshot)，为三维模型设置背景图像，得到合成图像。具体的，可以基于三维模型的不同角度和/或不同的背景图像，对于同一目标物的三维模型，生成多个合成图像。
为了得到真实度更高的合成图像，以使得基于合成图像得到的训练数据更新的目标检测模型更适用于如图2所示的拣选机器人工作站的真实场景，在本实施例的一些可选的实现方式中，上述执行主体可 以通过如下方式执行上述步骤302：
第一，确定拣选机器人对应的第二图像采集装置、预设信息采集位置和第一图像采集装置之间的坐标系对应关系。
第二图像采集装置、预设信息采集位置和第一图像采集装置依次设置有对应的坐标系。上述执行主体可以通过标定算法确定拣选机器人对应的第二图像采集装置、预设信息采集位置和第一图像采集装置之间的坐标系对应关系。
第二，根据坐标系对应关系，确定三维模型在第二图像采集装置的视角下的调整后三维模型。
将三维模型调整至拣选机器人对应的第二图像采集装置的视角下，以充分贴合拣选机器人工作站的真实场景。
第三，根据调整后三维模型和表征拣选机器人的拣选场景的预设背景图像，生成包括目标物对应的目标物对象的合成图像。
作为示例，拣选工位处的目标物一般放置于周转箱中，预设背景图像可以是包括空周转箱对应的周转箱对象的背景图像。如此，可进一步提高合成图像的真实性。
在本实施例的一些可选的实现方式中，上述执行主体还可以确定目标物的重量。作为示例，在预设信息采集位置设置重量传感器，以获取目标物的重量。
在本实现方式中，上述执行主体可以通过如下方式执行上述第三步骤：
根据调整后三维模型、预设背景图像、重量、合成图像的预设分辨率、第二图像采集装置的参数，生成包括目标物对应的目标物对象的合成图像。
其中，预设分辨率用于表征所期望的合成图像的分辨率，第二图像采集装置的参数表征其内参数据。
具体的，上述执行主体将调整后三维模型、预设背景图像、重量、合成图像的预设分辨率、第二图像采集装置的参数等数据作为输入数据输入虚拟物理引擎工具(如pybullet)，得到合成图像。
需要说明的是，所生成的合成图像可以包括同种类的多个目标物 对象，也可以包括不同种类的多个目标物对象。作为示例，不同种类的目标物可以是截止到当前所拣选的多种目标物。多种目标物的拣选过程中，上述执行主体得到的每种目标物的调整后三维模型和重量信息，通过虚拟物理引擎工具可以得到不同种类的多个目标物对象的合成图像，以通过合成图像组成的训练数据训练得到检测精度更高的目标检测模型。
步骤303，以合成图像为样本图像，以目标物对象为标签，得到训练样本，生成训练样本集。
本实施例中，上述执行主体可以合成图像为样本图像，以目标物对象为标签，得到训练样本，生成训练样本集。
合成图像是根据三维模型合成的，其中的目标物对象可自然获得，从而即免去了对于标签的人工标注环节，可扩充用于更新目标检测模型的训练样本集。
步骤304，通过训练样本集训练目标检测模型，得到更新后的目标检测模型。
本实施例中，上述执行主体可以通过训练样本集训练目标检测模型，得到更新后的目标检测模型。
具体的，首先，从训练样本集中选取未经过训练的训练样本，将所选取的训练样本中的样本图像输入目标检测模型，以所输入的训练样本对应的标签为期望输出，得到目标检测模型的实际输出。然后，计算实际输出与标签之间的损失。最后，根据损失计算梯度，并基于梯度下降法、随机梯度下降法进行目标检测模型的参数更新。
上述执行主体可以循环执行上述更新过程，直至得到预设结束条件，得到更新后的目标检测模型。其中，预设结束条件例如可以是训练次数超过预设次数阈值，训练时间超过预设时间阈值，损失趋于收敛。
在本实施例的一些可选的实现方式中，上述执行主体可以通过如下方式执行上述步骤304：
响应于确定目标检测模型的检测精度低于预设阈值，利用机器学习算法，通过训练样本集训练目标检测模型，得到更新后的目标检测 模型。
其中，预设阈值可以根据实际情况具体设置。作为示例，当拣选任务要求的检测精度较高，可以设置较高的预设阈值；当拣选任务要求的检测精度并不高，可以设置较低的预设阈值。
作为示例，上述执行主体可以将合成图像输入未更新的目标检测模型，得到目标检测模型实际输出的目标对象的检测框；进而确定实际输出的检测框与标签对应的检测框之间的IoU(Intersection-over-Union，交并比)，将多个合成图像对应的IoU求取平均值，确定目标检测模型的检测精度。
在本实施例的一些可选的实现方式中，在得到更新后的目标检测模型后，上述执行主体通过更新后的目标检测模型对后续的输入图像进行目标检测，得到检测结果；根据检测结果控制拣选机器人执行拣选任务。
继续参见图4，图4是根据本实施例的目标检测模型的更新方法的应用场景的一个示意图400。在图4的应用场景中，包括图像采集装置401、图像采集装置402、服务器403和拣选机器人404，服务器403中设置有目标检测模型。图像采集装置401设置于拣选工位处，摄取拣选工位处的目标物得到待检测图像，并通过目标检测模型进行目标检测，得到检测结果，进而控制拣选机器人拣选目标物将其移动至码垛位。拣选机器人执行对于目标物的拣选任务的过程中，通过图像采集装置402采集处于预设信息采集位置的目标物在多个角度下的图像数据。在得到多个角度下的图像数据后，服务器首先根据目标物在多个角度下的图像数据，构建目标物的三维模型；然后，根据三维模型，生成包括表征目标物的目标物对象的合成图像；以合成图像为样本图像，以目标物对象为标签，得到训练样本，生成训练样本集；然后，通过训练样本集训练目标检测模型，得到更新后的目标检测模型。
本公开的上述实施例提供的方法，通过根据目标物在多个角度下的图像数据，构建目标物的三维模型；根据三维模型，生成包括表征 目标物的目标物对象的合成图像；以合成图像为样本图像，以目标物对象为标签，得到训练样本，生成训练样本集；通过训练样本集训练目标检测模型，得到更新后的目标检测模型，从而提供了一种自动生成训练数据、自动更新目标检测模型的方法，提高了目标检测模型更新的便捷性和检测结果的准确度。
继续参考图5，示出了根据本公开的目标检测模型的更新方法的一个实施例的示意性流程500，包括以下步骤：
步骤501，在根据目标检测模型对输入图像中的目标物对象的检测结果，控制拣选机器人执行对于目标物的拣选任务的过程中，通过第一图像采集装置采集处于预设信息采集位置的目标物在多个角度下的二维图像数据、三维图像数据和重量。
步骤502，根据二维图像数据和三维图像数据，构建目标物的三维模型。
步骤503，确定拣选机器人对应的第二图像采集装置、预设信息采集位置和第一图像采集装置之间的坐标系对应关系。
步骤504，根据坐标系对应关系，确定三维模型在第二图像采集装置的视角下的调整后三维模型。
步骤505，根据调整后三维模型、预设背景图像、重量、合成图像的预设分辨率、第二图像采集装置的参数，生成包括目标物对应的目标物对象的合成图像。
步骤506，以合成图像为样本图像，以目标物对象为标签，得到训练样本，生成训练样本集。
步骤507，通过训练样本集训练目标检测模型，得到更新后的目标检测模型。
其中，目标检测模型用于表征输入图像与输入图像中的目标物对象对应的检测结果之间的对应关系。
步骤508，通过更新后的目标检测模型对后续的输入图像进行目标检测，得到检测结果。
步骤509，根据检测结果控制拣选机器人执行拣选任务。
从本实施例中可以看出，与图3对应的实施例相比，本实施例中的目标检测模型的更新方法的流程500具体说明了合成图像的构建过程、更新后的目标检测模型的应用过程，提供了一种针对于目标检测模型的闭环持续学习方法，基于构建目标物的三维模型的方式构建训练样本集，适用于所有形状的目标物，提高了目标检测模型的普适性和检测精度。
继续参考图6，作为对上述各图所示方法的实现，本公开提供了一种目标检测模型的更新装置的一个实施例，该装置实施例与图3所示的方法实施例相对应，该装置具体可以应用于各种电子设备中。
如图6所示，目标检测模型的更新装置包括：构建单元601，被配置成根据目标物在多个角度下的图像数据，构建目标物的三维模型；图像生成单元602，被配置成根据三维模型，生成包括表征目标物的目标物对象的合成图像；样本生成单元603，被配置成以合成图像为样本图像，以目标物对象为标签，得到训练样本，生成训练样本集；更新单元604，被配置成通过训练样本集训练目标检测模型，得到更新后的目标检测模型，其中，目标检测模型用于表征输入图像与输入图像中的目标物对象对应的检测结果之间的对应关系。
在本实施例的一些可选的实现方式中，构建单元601，进一步被配置成：在根据目标检测模型对输入图像中的目标物对象的检测结果，控制拣选机器人执行对于目标物的拣选任务的过程中，通过第一图像采集装置采集处于预设信息采集位置的目标物在多个角度下的二维图像数据和三维图像数据；根据二维图像数据和三维图像数据，构建目标物的三维模型。
在本实施例的一些可选的实现方式中，图像生成单元602，进一步被配置成：确定拣选机器人对应的第二图像采集装置、预设信息采集位置和第一图像采集装置之间的坐标系对应关系；根据坐标系对应关系，确定三维模型在第二图像采集装置的视角下的调整后三维模型；根据调整后三维模型和表征拣选机器人的拣选场景的预设背景图像，生成包括目标物对应的目标物对象的合成图像。
在本实施例的一些可选的实现方式中，上述装置还包括：确定单元(图中未示出)，被配置成确定目标物的重量；以及图像生成单元602，进一步被配置成：根据调整后三维模型、预设背景图像、重量、合成图像的预设分辨率、第二图像采集装置的参数，生成包括目标物对应的目标物对象的合成图像。
在本实施例的一些可选的实现方式中，更新单元604，进一步被配置成：响应于确定目标检测模型的检测精度低于预设阈值，利用机器学习算法，通过训练样本集训练目标检测模型，得到更新后的目标检测模型。
在本实施例的一些可选的实现方式中，上述装置还包括：得到单元(图中未示出)，被配置成通过更新后的目标检测模型对后续的输入图像进行目标检测，得到检测结果；执行单元(图中未示出)，被配置成根据检测结果控制拣选机器人执行拣选任务。
本实施例中，目标检测模型的更新装置中的构建单元根据目标物在多个角度下的图像数据，构建目标物的三维模型；图像生成单元根据三维模型，生成包括表征目标物的目标物对象的合成图像；样本生成单元以合成图像为样本图像，以目标物对象为标签，得到训练样本，生成训练样本集；更新单元通过训练样本集训练目标检测模型，得到更新后的目标检测模型，其中，目标检测模型用于表征输入图像与输入图像中的目标物对象对应的检测结果之间的对应关系，从而提供了一种自动生成训练数据、自动更新目标检测模型的装置，提高了目标检测模型更新的便捷性和检测结果的准确度。
下面参考图7，其示出了适于用来实现本公开实施例的设备(例如图1所示的设备101、102、105、106)的计算机系统700的结构示意图。图7示出的设备仅仅是一个示例，不应对本公开实施例的功能和使用范围带来任何限制。
如图7所示，计算机系统700包括处理器(例如CPU，中央处理器)701，其可以根据存储在只读存储器(ROM)702中的程序或者从存储部分708加载到随机访问存储器(RAM)703中的程序而执行各 种适当的动作和处理。在RAM703中，还存储有系统700操作所需的各种程序和数据。处理器701、ROM702以及RAM703通过总线704彼此相连。输入/输出(I/O)接口705也连接至总线704。
以下部件连接至I/O接口705：包括键盘、鼠标等的输入部分706；包括诸如阴极射线管(CRT)、液晶显示器(LCD)等以及扬声器等的输出部分707；包括硬盘等的存储部分708；以及包括诸如LAN卡、调制解调器等的网络接口卡的通信部分709。通信部分709经由诸如因特网的网络执行通信处理。驱动器710也根据需要连接至I/O接口705。可拆卸介质711，诸如磁盘、光盘、磁光盘、半导体存储器等等，根据需要安装在驱动器710上，以便于从其上读出的计算机程序根据需要被安装入存储部分708。
特别地，根据本公开的实施例，上文参考流程图描述的过程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，其包括承载在计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通过通信部分709从网络上被下载和安装，和/或从可拆卸介质711被安装。在该计算机程序被处理器701执行时，执行本公开的方法中限定的上述功能。
需要说明的是，本公开的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本公开中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本公开中，计算机可读的信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中 承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读的信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：无线、电线、光缆、RF等等，或者上述的任意合适的组合。
可以以一种或多种程序设计语言或其组合来编写用于执行本公开的操作的计算机程序代码，程序设计语言包括面向目标的程序设计语言—诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言—诸如”C”语言或类似的程序设计语言。程序代码可以完全地在客户计算机上执行、部分地在客户计算机上执行、作为一个独立的软件包执行、部分在客户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络——包括局域网(LAN)或广域网(WAN)—连接到客户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。
附图中的流程图和框图，图示了按照本公开各种实施例的装置、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，该模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
描述于本公开实施例中所涉及到的单元可以通过软件的方式实现，也可以通过硬件的方式来实现。所描述的单元也可以设置在处理器中， 例如，可以描述为：一种处理器，包括构建单元、图像生成单元、样本生成单元和更新单元。其中，这些单元的名称在某种情况下并不构成对该单元本身的限定，例如，更新单元还可以被描述为“通过训练样本集训练目标检测模型，得到更新后的目标检测模型的单元”。
作为另一方面，本公开还提供了一种计算机可读介质，该计算机可读介质可以是上述实施例中描述的设备中所包含的；也可以是单独存在，而未装配入该设备中。上述计算机可读介质承载有一个或者多个程序，当上述一个或者多个程序被该装置执行时，使得该计算机设备：根据目标物在多个角度下的图像数据，构建目标物的三维模型；根据三维模型，生成包括表征目标物的目标物对象的合成图像；以合成图像为样本图像，以目标物对象为标签，得到训练样本，生成训练样本集；通过训练样本集训练目标检测模型，得到更新后的目标检测模型，其中，目标检测模型用于表征输入图像与输入图像中的目标物对象对应的检测结果之间的对应关系。
以上描述仅为本公开的较佳实施例以及对所运用技术原理的说明。本领域技术人员应当理解，本公开中所涉及的发明范围，并不限于上述技术特征的特定组合而成的技术方案，同时也应涵盖在不脱离上述发明构思的情况下，由上述技术特征或其等同特征进行任意组合而形成的其它技术方案。例如上述特征与本公开中公开的(但不限于)具有类似功能的技术特征进行互相替换而形成的技术方案。

Claims (15)

  1. 一种目标检测模型的更新方法，包括：
    根据目标物在多个角度下的图像数据，构建所述目标物的三维模型；
    根据所述三维模型，生成包括表征所述目标物的目标物对象的合成图像；
    以所述合成图像为样本图像，以所述目标物对象为标签，得到训练样本，生成训练样本集；
    通过所述训练样本集训练所述目标检测模型，得到更新后的目标检测模型，其中，所述目标检测模型用于表征输入图像与输入图像中的目标物对象对应的检测结果之间的对应关系。
  2. 根据权利要求1所述的方法，其中，所述根据目标物在多个角度下的图像数据，构建所述目标物的三维模型，包括：
    在根据所述目标检测模型对输入图像中的目标物对象的检测结果，控制拣选机器人执行对于所述目标物的拣选任务的过程中，通过第一图像采集装置采集处于预设信息采集位置的所述目标物在多个角度下的二维图像数据和三维图像数据；
    根据所述二维图像数据和所述三维图像数据，构建所述目标物的三维模型。
  3. 根据权利要求2所述的方法，所述根据所述三维模型，生成包括表征所述目标物的目标物对象的合成图像，包括：
    确定所述拣选机器人对应的第二图像采集装置、所述预设信息采集位置和所述第一图像采集装置之间的坐标系对应关系；
    根据所述坐标系对应关系，确定所述三维模型在所述第二图像采集装置的视角下的调整后三维模型；
    根据所述调整后三维模型和表征所述拣选机器人的拣选场景的预设背景图像，生成包括所述目标物对应的目标物对象的合成图像。
  4. 根据权利要求1所述的方法，其中，还包括：
    确定所述目标物的重量；以及
    所述根据所述调整后三维模型和表征所述拣选机器人的拣选场景的预设背景图像，生成包括所述目标物对应的目标物对象的合成图像，包括：
    根据所述调整后三维模型、所述预设背景图像、所述重量、所述合成图像的预设分辨率、所述第二图像采集装置的参数，生成包括所述目标物对应的目标物对象的合成图像。
  5. 根据权利要求1所述的方法，其中，所述通过所述训练样本集训练所述目标检测模型，得到更新后的目标检测模型，包括：
    响应于确定所述目标检测模型的检测精度低于预设阈值，利用机器学习算法，通过所述训练样本集训练所述目标检测模型，得到更新后的目标检测模型。
  6. 根据权利要求1-5中任一所述的方法，其中，还包括：
    通过更新后的目标检测模型对后续的输入图像进行目标检测，得到检测结果；
    根据所述检测结果控制拣选机器人执行拣选任务。
  7. 一种目标检测模型的更新装置，包括：
    构建单元，被配置成根据目标物在多个角度下的图像数据，构建所述目标物的三维模型；
    图像生成单元，被配置成根据所述三维模型，生成包括表征所述目标物的目标物对象的合成图像；
    样本生成单元，被配置成以所述合成图像为样本图像，以所述目标物对象为标签，得到训练样本，生成训练样本集；
    更新单元，被配置成通过所述训练样本集训练所述目标检测模型，得到更新后的目标检测模型，其中，所述目标检测模型用于表征输入 图像与输入图像中的目标物对象对应的检测结果之间的对应关系。
  8. 根据权利要求7所述的装置，其中，所述构建单元，进一步被配置成：
    在根据所述目标检测模型对输入图像中的目标物对象的检测结果，控制拣选机器人执行对于所述目标物的拣选任务的过程中，通过第一图像采集装置采集处于预设信息采集位置的所述目标物在多个角度下的二维图像数据和三维图像数据；根据所述二维图像数据和所述三维图像数据，构建所述目标物的三维模型。
  9. 根据权利要求8所述的装置，所述图像生成单元，进一步被配置成：
    确定所述拣选机器人对应的第二图像采集装置、所述预设信息采集位置和所述第一图像采集装置之间的坐标系对应关系；根据所述坐标系对应关系，确定所述三维模型在所述第二图像采集装置的视角下的调整后三维模型；根据所述调整后三维模型和表征所述拣选机器人的拣选场景的预设背景图像，生成包括所述目标物对应的目标物对象的合成图像。
  10. 根据权利要求7所述的装置，其中，还包括：
    确定单元，被配置成确定所述目标物的重量；以及
    所述图像生成单元，进一步被配置成：
    根据所述调整后三维模型、所述预设背景图像、所述重量、所述合成图像的预设分辨率、所述第二图像采集装置的参数，生成包括所述目标物对应的目标物对象的合成图像。
  11. 根据权利要求7所述的装置，其中，所述更新单元，进一步被配置成：
    响应于确定所述目标检测模型的检测精度低于预设阈值，利用机器学习算法，通过所述训练样本集训练所述目标检测模型，得到更新 后的目标检测模型。
  12. 根据权利要求7-11中任一所述的装置，其中，还包括：
    得到单元，被配置成通过更新后的目标检测模型对后续的输入图像进行目标检测，得到检测结果；
    执行单元，被配置成根据所述检测结果控制拣选机器人执行拣选任务。
  13. 一种计算机可读介质，其上存储有计算机程序，其中，所述程序被处理器执行时实现权利要求1-6中任一所述的方法。
  14. 一种电子设备，包括：
    一个或多个处理器；
    存储装置，其上存储有一个或多个程序，
    当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现权利要求1-6中任一所述的方法。
  15. 一种计算机程序产品，包括计算机程序，所述计算机程序在被处理器执行时实现权利要求1-6中任一项所述的方法。
PCT/CN2022/099510 2021-08-17 2022-06-17 目标检测模型的更新方法及装置 WO2023020103A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP22857416.6A EP4336385A1 (en) 2021-08-17 2022-06-17 Method and apparatus for updating target detection model

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110957524.XA CN113610968A (zh) 2021-08-17 2021-08-17 目标检测模型的更新方法及装置
CN202110957524.X 2021-08-17

Publications (1)

Publication Number Publication Date
WO2023020103A1 true WO2023020103A1 (zh) 2023-02-23

Family

ID=78341408

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/099510 WO2023020103A1 (zh) 2021-08-17 2022-06-17 目标检测模型的更新方法及装置

Country Status (3)

Country Link
EP (1) EP4336385A1 (zh)
CN (1) CN113610968A (zh)
WO (1) WO2023020103A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117911630B (zh) * 2024-03-18 2024-05-14 之江实验室 一种三维人体建模的方法、装置、存储介质及电子设备

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113610968A (zh) * 2021-08-17 2021-11-05 北京京东乾石科技有限公司 目标检测模型的更新方法及装置
CN115082795A (zh) * 2022-07-04 2022-09-20 梅卡曼德（北京）机器人科技有限公司 虚拟图像的生成方法、装置、设备、介质及产品

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111931836A (zh) * 2020-07-31 2020-11-13 上海商米科技集团股份有限公司 获取神经网络训练图像的方法和装置
CN112016630A (zh) * 2020-09-03 2020-12-01 平安科技（深圳）有限公司 基于图像分类模型的训练方法、装置、设备及存储介质
CN112287960A (zh) * 2019-07-24 2021-01-29 辉达公司 用于训练或再训练机器学习模型的地面真值数据的自动生成
US10909349B1 (en) * 2019-06-24 2021-02-02 Amazon Technologies, Inc. Generation of synthetic image data using three-dimensional models
CN112651881A (zh) * 2020-12-30 2021-04-13 北京百度网讯科技有限公司 图像合成方法、装置、设备、存储介质以及程序产品
CN113610968A (zh) * 2021-08-17 2021-11-05 北京京东乾石科技有限公司 目标检测模型的更新方法及装置

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109061643B (zh) * 2018-08-13 2022-05-27 南京理工大学 基于三维属性散射中心模型的多雷达融合高分辨成像方法
CN112465960B (zh) * 2020-12-18 2022-05-20 天目爱视（北京）科技有限公司 一种三维模型的尺寸标定装置及方法
CN112966742A (zh) * 2021-03-05 2021-06-15 北京百度网讯科技有限公司 模型训练方法、目标检测方法、装置和电子设备

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10909349B1 (en) * 2019-06-24 2021-02-02 Amazon Technologies, Inc. Generation of synthetic image data using three-dimensional models
CN112287960A (zh) * 2019-07-24 2021-01-29 辉达公司 用于训练或再训练机器学习模型的地面真值数据的自动生成
CN111931836A (zh) * 2020-07-31 2020-11-13 上海商米科技集团股份有限公司 获取神经网络训练图像的方法和装置
CN112016630A (zh) * 2020-09-03 2020-12-01 平安科技（深圳）有限公司 基于图像分类模型的训练方法、装置、设备及存储介质
CN112651881A (zh) * 2020-12-30 2021-04-13 北京百度网讯科技有限公司 图像合成方法、装置、设备、存储介质以及程序产品
CN113610968A (zh) * 2021-08-17 2021-11-05 北京京东乾石科技有限公司 目标检测模型的更新方法及装置

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117911630B (zh) * 2024-03-18 2024-05-14 之江实验室 一种三维人体建模的方法、装置、存储介质及电子设备

Also Published As

Publication number Publication date
CN113610968A (zh) 2021-11-05
EP4336385A1 (en) 2024-03-13

Similar Documents

Publication Publication Date Title
WO2023020103A1 (zh) 目标检测模型的更新方法及装置
CN109870983B (zh) 处理托盘堆垛图像的方法、装置及用于仓储拣货的系统
JP6551184B2 (ja) シミュレーション装置、シミュレーション方法、およびシミュレーションプログラム
CN108053473A (zh) 一种室内三维模型数据的处理方法
RU2700246C1 (ru) Способ и система захвата объекта с помощью роботизированного устройства
JP6671694B1 (ja) 機械学習装置、機械学習システム、データ処理システム及び機械学習方法
JP2017094407A (ja) シミュレーション装置、シミュレーション方法、およびシミュレーションプログラム
CN109910018B (zh) 具有视觉语义感知的机器人虚实交互操作执行系统及方法
CN112115607A (zh) 一种基于多维赛博空间的移动智能体数字孪生系统
US20220019939A1 (en) Method and system for predicting motion-outcome data of a robot moving between a given pair of robotic locations
CN111309035B (zh) 多机器人协同移动与动态避障方法、装置、设备及介质
CN113858217B (zh) 多机器人交互的三维视觉位姿感知方法及系统
CN111414508B (zh) 一种在设计模型中搜索并实现可视化的方法及终端
CN113561181B (zh) 目标检测模型的更新方法、装置及系统
Kim et al. Digital twin for autonomous collaborative robot by using synthetic data and reinforcement learning
CN116205561A (zh) 一种基于数字孪生的物料管理方法和系统
Maru et al. Internet of things based cyber-physical system framework for real-time operations
CN114897339A (zh) 任务评价方法、装置、存储介质及电子设备
CN117769724A (zh) 使用深度学习的对象检测和分类的合成数据集创建
Gomes et al. Deep Reinforcement learning applied to a robotic pick-and-place application
Zheng et al. Virtual Prototyping-Based Path Planning of Unmanned Aerial Vehicles for Building Exterior Inspection
WO2023100282A1 (ja) データ生成システム、モデル生成システム、推定システム、学習済みモデルの製造方法、ロボット制御システム、データ生成方法、およびデータ生成プログラム
CN114390270B (zh) 实时智能站点全景查勘方法、装置和电子设备
Horváth et al. A use case of the simulation-based approach to mobile robot algorithm development
US20220012462A1 (en) Systems and Methods for Remote Measurement using Artificial Intelligence

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 2022857416

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2023577672

Country of ref document: JP

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 2022857416

Country of ref document: EP

Effective date: 20231204

NENP Non-entry into the national phase

Ref country code: DE