CN110405810A - Automatic package registration arrangement, device and method - Google Patents

Abstract

It is a kind of to operate the first image data that the method for wrapping up processing system includes: the first surface image on reception expression package surface；A pair of intersecting edge on the package surface is identified based on the first image；Minimum feasible region is determined based on the pair of edge, and the minimum feasible region is for clamping and being promoted the package；Receive the second image data of the package after indicating the promotion；And registration data is created based on the third image data.

Description

Automatic package registration arrangement, device and method

The application is the divisional application application No. is 201910582988.X, and the applying date of the female case is 06 month 2019 28 Day, entitled automatic package registration arrangement, device and method.

Cross reference to related applications

This application claims the equity for the U.S. Provisional Patent Application Serial No. 62/752,756 that on October 30th, 2018 submits, The full content of the temporary patent application is hereby incorporated herein by.

Background technique

Under normal conditions, package is shipped to the destination of destacking package by stacking.Sometimes, package is unloaded by worker Pile, this may be with high costs, and has the risk for causing personal injury.In industrial environment, destacking is operated usually by industrial machine People's (such as mechanical arm) executes, and clamps, is promoted, transports and be delivered to point of release for package.Furthermore, it is possible to utilize imaging device To capture the image for the package stacking being loaded on pallet.System can handle image to ensure that mechanical arm efficiently handles package, Such as by the way that institute's captured image to be compared with the image that is registrated being stored in registration data source.

Sometimes, the captured images of package can be registrated images match.As a result, imaging object physical characteristic (for example, The measured value of the size of package, weight and/or mass center) it may be unknown.Fail correctly to identify that physical characteristic may cause respectively The undesirable result of kind.For example, such failure may cause shutdown, this may need manual registration to wrap up.In addition, such failure It may cause package to deal with improperly, especially if wrapping up relatively heavy and/or deflection.

Detailed description of the invention

Figure 1A shows the robot system configured with package registration mechanism of some embodiments according to this technology.

Figure 1B is the functional block diagram according to the robot system of some embodiments of this technology.

Fig. 2A shows the force-moment sensor of some embodiments according to this technology.

Fig. 2 B shows the clamper of some embodiments according to this technology.

Fig. 3 A shows the perspective view of exemplary package stacking.

Fig. 3 B shows the top view of exemplary package stacking.

Fig. 3 C shows the exemplary symbols of the registration package according to some embodiments of this technology.

Fig. 4 A shows the perspective view of the second exemplary package stacking of some embodiments according to this technology.

Fig. 4 B shows the top view of the second package stacking of some embodiments according to this technology.

Fig. 4 C shows the exemplary symbols of some embodiments according to this technology being registrated and be not registrated package.

Fig. 4 D shows the unintelligible edge of some embodiments and showing for minimum feasible region (MVR) according to this technology Example.

Fig. 4 E shows exemplary according to second group for being registrated and not being registrated package of some embodiments of this technology Symbol.

Fig. 4 F shows second example at unintelligible edge and MVR according to some embodiments of this technology.

Fig. 4 G shows the third group exemplary symbols for not being registrated package of some embodiments according to this technology.

Fig. 4 H shows clamper system the showing above the first MVR of Fig. 4 D according to some embodiments of this technology Example property is placed.

Fig. 4 I shows the 4th group of exemplary symbols for not being registrated package of some embodiments according to this technology.

Fig. 4 J shows the unintelligible edge and MVR of the package for newly handling of some embodiments according to this technology Example.

Fig. 4 K shows exemplary above the MVR of Fig. 4 F according to the clamper system of some embodiments of this technology It places.

Fig. 4 L shows the 5th group of exemplary symbols for not being registrated package of some embodiments according to this technology.

Fig. 4 M shows the unintelligible edge and MVR of the package for newly handling of some embodiments according to this technology Example.

Fig. 4 N shows exemplary above the MVR of Fig. 4 M according to the clamper system of some embodiments of this technology It places.

Fig. 4 O shows exemplary above the MVR of Fig. 4 J according to the clamper system of some embodiments of this technology It places.

Fig. 4 P shows clamper system the showing above the 2nd MVR of Fig. 4 D according to some embodiments of this technology Example property is placed.

Fig. 4 Q shows clamper system the showing above the 3rd MVR of Fig. 4 D according to some embodiments of this technology Example property is placed.

Fig. 5 shows the example of the robot system 100 for operation diagram 1A of some embodiments according to this technology The control flow of property method.

Specific embodiment

In the following description, several details are presented to provide to the embodiment of inventive concept disclosed herein It understands thoroughly.However, those skilled in the relevant art are not it will be recognized that the embodiment of this technology disclosed herein can have Implement in the case where there are one or more specific details, or implementation, etc. in conjunction with other component.In other cases, many Well known implementation or operation is not shown or described in detail, to avoid the various embodiment party for obscuring this technology disclosed herein The various aspects of case.

Figure 1A and 1B shows the system, robot configured with package registration mechanism of some embodiments according to this technology System 100.In some embodiments, robot system may include destacking platform 110, receiving platform 120, mechanical arm system 130, End effector 140, processing unit (PU) 150, picture system 160, registration data source (RDS) 170, height determine sensor (HDS) 180 and/or release point sensor (RPS) 190.

Destacking platform 110 may include any platform, surface and/or structure, multiple packages 112 (individually, " package 112 ") It stacking and/or arrangement and can be ready to transport receiving platform 120 on it.Term is used herein although should be pointed out that " package ", but the term includes any other word, and as discussed in detail below, container can be clamped, promoted, transport Defeated and delivering, such as, but not limited to " box ", " chest ", " carton " or any combination thereof.In addition, although disclosed herein attached Rectangular cases are shown in figure, but the shape of chest is not limited to such shape, but including that can be clamped, be promoted, be transported With any rule of delivering or irregular shape, as discussed in detail below.

Similar with destacking platform 110, receiving platform 120 may include specifying for receiving package 112 further to be appointed Any platform of business/operation, surface and/or structure.In some embodiments, receiving platform 120 may include for that will wrap up 112 transport to another location from a position (for example, point of release as described below) to carry out further operating (for example, classification And/or storage) conveyer system.

Mechanical arm system 130 may include being configured as manipulation (for example, transmit and/or rotate or redirect) package 112 One group of coupling structure, connector, motor/actuator, sensor or combinations thereof.Mechanical arm system 130 may include mechanical arm 132, For illustrative purposes herein rather than limitation, mechanical arm 132 can be hinged six shaft mechanicals arm configuration.Although the discussion of this paper will be by Mechanical arm system 130 is referred to, but embodiment disclosed herein is not limited to such system, but including can be configured to hold Any robot system of row movement disclosed herein.

End effector 140 may include any one or more components for being connected to the distal end of mechanical arm 132, be configured To interact with multiple packages 112.For example, end effector 140 may include the knot for being configured as clamping and keeping package 112 Structure (for example, clamper based on vacuum).In some embodiments, end effector 140 may include that force-moment (F-T) is passed Sensor 142, arm interface 144, clamper system 146 and/or clamper interface 148 (as shown in Figure 2 A and 2B).

The PU 150 of Figure 1B can be any unit or circuit, be programmed and/or be configured not only to guide and control machine The movement of tool arm system 130 (including end effector 140) and/or other movement but also handle other related datas.For example, PU 150 can receive the image data for indicating the surface image (SI) of package 112 from picture system 160, and be mentioned according to RDS 170 The registration record 172 of confession determines the registration state (being discussed below) of SI, that is, package 112 is registration example or the packet for wrapping up 112 Wrap up in 112 be not registrated example (for example, package 112 registration record 172 be not present or incomplete example).Moreover, for example, PU 150 can be directed to mechanical arm system 130 the registration example of package 112.In addition, PU 150 can be released receiving from RPS 190 Clamping control signal is applied to clamper system 146 when discharge signal, arm is directed to receiving platform 120, and/or is flat receiving The registration example of release package 112 on platform 120.

PU 150 may include any electronic data processing division, and executing for good and all or to be stored temporarily in digital storage Storage device or software or computer instruction generation in non-transitory computer-readable medium (the usually memory 152 of Figure 1B) Code, the digital storage storage device or non-transitory computer-readable medium include but is not limited to random access memory (RAM), disc driver, magnetic memory, read-only memory (ROM), CD (CD), solid-state memory, safe digital card and/or Compact flash.PU 150 can be driven by executing software or computer instruction code, the software or computer instruction Code is included as the specific function realized herein and the algorithm developed.In some embodiments, PU 150 can be for for herein The specific integrated circuit (ASIC) of disclosed embodiment customization.In some embodiments, PU 150 may include microprocessor, One in digital signal processor (DSP), programmable logic device (PLD), programmable gate array (PGA) and signal generator Or it is multiple；However, term " processor " is not limited to such exemplary processing unit, and it contains for embodiments herein Justice, which is not intended to, narrowly to be explained.For example, PU 150 may also include more than one electronic data processing division.In some embodiment party In case, PU 150 can for by robot system 100 any other system use or processor in connection, it is described its His system includes but is not limited to mechanical arm system 130, end effector 140 and/or picture system 160.

PU 150 can electronics connection (for example, via electric wire, bus and/or wireless connection) to system and/or source to help In reception input data.In some embodiments, operatively connection can be considered as that exchange can be coupled with electronics.It is not necessary to It is directly connected to；On the contrary, receive input data and provide output data can by bus, by wireless network offer or conduct It is received by PU 150 via physics or virtual machine port and/or the signal of transmission provides.PU 150 can be programmed or configure To execute method discussed in more detail below.In some embodiments, PU 150 can be programmed or configured to from various systems and/ Or unit receives data, the system and/or unit include but is not limited to picture system 160, RDS 170, HDS 180 and/or RPS 190.In some embodiments, PU 150 can be programmed or configured to provide output number to various systems and/or unit According to the system and/or unit include but is not limited to mechanical arm system 130, end effector 140 and RDS 170.

Picture system 160 may include one or more sensors 162, and being configured as capture indicates to be located at destacking platform The image data of one or more SI of package 112 on 110.In some embodiments, image data can indicate to appear in Vision Design pattern and/or label on one or more surfaces of package 112, from the Vision Design pattern and/or label It can determine the registration state of package 112.In some embodiments, picture system 160 may include one or more cameras, quilt Be designed to target (such as, it is seen that and/or it is infrared) work in electromagnetic spectrum bandwidth and for detecting light/energy in corresponding spectrum Amount.In some embodiments, image data can be from one or more three-dimensional (3D) cameras and/or one or more two dimensions One group of data point of (2D) camera capture, this group of data point form point cloud, depth map or combinations thereof.From these cameras, it may be determined that Between picture system 160 and one or more exposure (for example, sight relative to picture system 160) surface of package 112 Distance or depth.In some embodiments, distance or depth can be determined by using image recognition algorithm, such as context Image classification algorithms and/or edge detection algorithm.Once it is determined that distance/depth value can be used for manipulating via mechanical arm system 130 Package 112.For example, distance/depth value can be used to calculate and can promote and/or clamp packet in PU 150 and/or mechanical arm system 130 Wrap up in 112 position.It should be pointed out that data described herein such as image data may include any discrete or continuous mould Quasi- or digital signal may include information or instruction information.

Picture system 160 may include at least one display unit 164, is configured as presenting and is captured by sensor 162 The image of package 112, described image can be observed by one or more operators of robot system 100, be such as discussed further below 's.In addition, display unit 164 can be configured to present other information, such as, but not limited to indicate package 112 registration and/or It is not registrated the symbol of example, as discussed in detail below.

RDS 170 may include being configured as storing any database of the registration record 172 of multiple packages 112 and/or depositing Reservoir storage device (for example, non-transitory computer-readable medium).For example, RDS 170 may include read-only memory (ROM), light Disk (CD), solid-state memory, safe digital card, compact flash and/or data storage server or remote storage.

In some embodiments, registration record 172 can respectively include the physical features or attribute of corresponding package 112.Example Such as, each registration record 172 may include but be not limited to one or more templates SI, 2D or 3D size measurement, weight and/or matter The heart (CoM) information.Template SI can indicate package 112 known or previously determined visible features, including package 112 design, Label, appearance, outer shape/profile or combinations thereof.2D or 3D size measurement may include the length of known/expected package, width Degree, height or combinations thereof.

In some embodiments, RDS 170 can be configured to receive matching according to the creation of following public embodiment The new example (for example, the not previously known aspect for being directed to not previously known package and/or package) of quasi- record 172.Therefore, machine People's system 100 can be used to be registrated package 112 by the quantity of registration record 172 of the extension storage in RDS 170 to automate Process, to make destacking operation more efficiently, and wrap up 112 not to be registrated example less.By using in real time/operation data The registration record 172 in RDS 170 dynamically (for example, during operation/deployment) is updated, robot system 100 can be efficiently Realize that study's procedure of computer subject, the study's procedure of computer subject can solve not previously known or fortuitous event (for example, lighting condition, not Know that orientation and/or stacking are inconsistent) and/or the package that newly encounters.Therefore, robot system 100 can reduce by " unknown " situation/ Caused by package, associated human operator who intervention and/or associated mission failure (for example, the package lost and/or collision) Failure.

HDS 180 may include being configured to supply object (for example, package 112) relative to reference point (for example, and HDS 180 associated surfaces) the component measured vertically.For example shown in figure 1A, when the bottom of package 112/edge passes through When horizontal surface (for example, plane of scanning motion) associated with HDS 180, HDS 180 can send signal to the PU 150 of Fig. 1.PU 150 can be used the signal from HDS 180 to determine the height (for example, according to placement/close orientation) of package 112, such as root According to HDS 180 the plane of scanning motion (that is, previously known) and clamper system 146 signal generate when position between away from From.As described below, wrap up 112 height can be used as three-dimensional measurement value be added to be registered in registration record 172 in 2D survey Magnitude.In some embodiments, HDS 180 is mountable on receiving platform 120, as shown in Figure 1A.

RPS 190 may include component/circuit, which is configured as passing through or contacting and RPS when package 112 Triggering is supplied to the signal of PU 150 when 190 associated horizontal plane.In some embodiments, the letter triggered by RPS 190 Number it can be used for determining the position that package 112 is discharged on receiving platform 120 by clamper system 146.In some embodiments, RPS 190 is mountable on receiving platform 120, as shown in Figure 1A.

Referring now to shown in Fig. 2A, F-T sensor 142 can be to be configured as detecting the axis and/or axis along coordinate system Linear force and/or torque any sensor.In some embodiments, F-T sensor 142 may include that there are six axle powers to pass The component of sensor, the six-axis force sensor are configured as utilize up to three of robot system 100 of detection Figure 1A and Figure 1B Axle power (for example, the power detected along the x-axis, y-axis and z-axis of cartesian coordinate system) and/or three axle power squares are (for example, surround The torque that the x-axis, y-axis and z-axis of cartesian coordinate system detect).In some embodiments, F-T sensor 142 may include using In the built-in amplifier and microcomputer of signal processing, static and dynamically measurement the ability of progress, and/or based on sampling interval inspection Survey the ability changed immediately.In some embodiments, F-T sensor 142 can be with the PU 150 of Figure 1B via wired and/or nothing Couple to line communication.

Arm interface 144 can be to be configured as the distal end of the mechanical arm 132 of Figure 1A being connected to any of F-T sensor 142 Device.In some embodiments, the manufacturer of mechanical arm system 130 can be different from the manufacturer of F-T sensor 142.At this In the case of kind, the end of arm interface 144 can have different configurations: one end is configured for being connected to distal end, and the other end It is configured for being connected to F-T sensor 142.

Referring now to Fig. 2 B, clamper system 146 can be the destacking platform being configured as from the package 112 of Figure 1A in Figure 1A Any system or component of resting position clamping package 112 on 110, and when package 112 is by the mechanical arm system 130 of Figure 1B Clamping is kept when transporting and be delivered to the receiving platform 120 of Figure 1A.In some embodiments, clamper system 146 can be by true Empty clamper system composition, the holder fixed by vacuum system use vacuum source, for clamping package 112 simultaneously at receiving platform 120 The release package 112 at destacking platform 110.In some embodiments, holder fixed by vacuum system can be used sucker as with packet Wrap up in 112 engagement.

The clamper interface 148 of Fig. 2 B can be to be configured as F-T sensor 142 being connected to appointing for clamper system 146 What device.In some embodiments, the manufacturer of F-T sensor 142 can be different from the manufacturer of clamper system 146.In In this case, the opposite side of clamper interface 148 can have different configurations: side is configured for being connected to F-T sensing Device 142, and opposite side is configured for being connected to clamper system 146.

Fig. 3 A to 4Q shows the package 112 that Figure 1A how can be created using the robot system 100 of Figure 1A and Figure 1B The Figure 1B for not being registrated example registration record 172.Fig. 3 A shows the perspective of the exemplary stacking of package 112-1 to 112-24 Figure.Fig. 3 B shows the top view of the exemplary stacking of these packages 112.Fig. 3 B can correspond to overlook the field of these packages 112 The image and/or point cloud data of scape, as the sensor 162 of the picture system 160 by Figure 1A and Figure 1B captures.Merely for explanation Rather than the purpose of limitation, package 112-1 can indicate package 112 to 112-3,112-13 to 112-14 and 112-21 to 112-24 It is registrated example, each package has the registration record 172 being stored in the RDS 170 of Figure 1B.Only for discussion and for purpose of brevity, Package 112 is shown as rectangular shape, and respectively has the SI (being shown as diagonal line) being registrated in corresponding registration record 172.

Fig. 3 C shows the exemplary symbols of the registration package according to some embodiments of this technology.Fig. 3 C shows packet Wrap up in symbol (for example, visual indicia and/or covering of the registration state for indicating shown object) 112-1a to 112-3a, 112-13a to 112-14a and 112-21a to 112-24a, as shown in the display unit 164 of Figure 1B.In order to generate these symbols and Corresponding registration state, package 112-1 being captured to 112-3,112-13 to 112-14 and 112-21 to 112-24 of Fig. 3 B SI can be compared using image recognition algorithm with the SI for being registrated record 172 being stored in RDS 170, and if matched Standard is then shown as symbol.In some embodiments, symbol can be registered by manufacturer and/or end user using instruction SI Visual formats configure, wherein various shape and/or color can be used in such format.For illustrative and not limiting purpose, Manufacturer and/or end user are by symbol 112-1a to 112-3a, 112-13a to 112-14a and 112-21a to 112-24a It is configured to the shapes and sizes (that is, being assumed to be rectangle herein) of package profile.Although being not shown, symbol can be color (example Such as, green), instruction package 112-1 having to 112-3,112-13 to 112-14 and 112-21 to each of 112-24 Benefit registration state (that is, registration record 172 there are examples).Although being not shown, the image of scene shown in Fig. 3 B can It is presented on display unit 164, the symbol of Fig. 3 C can be covered on above described image.In some embodiments, instruction package Alphanumeric character of the 112-1 to 112-3,112-13 to 112-14 and 112-21 to 112-24 is (for example, in registration record 172 In one or more physical characteristics for listing) symbol can be replaced or presented as the supplement of symbol.

In some embodiments, robot system 100 can be from image data (by the figure that may include edge detection algorithm As recognizer) the 2D measurement (for example, length and width) wrapped up.For example, image recognition can be used in robot system 100 Algorithm (for example, edge detection algorithm and/or mapping algorithm) measures to carry out the 2D of package 112 at destacking platform 110, such as 112-1 is wrapped up to 112-3,112-13 to 112-14 and 112-21 to 112-24.Robot system 100 can be based on away from measurement surface Depth carry out 2D measurement.The 2D measurement of corresponding package 112 can be registrated record 172 with it and is compared by robot system 100 (for example, measured package once being identified, such as via the image recognition of its exposed surface), with confirmation registration record 172 Accuracy.

In some embodiments, the CoM information being stored in registration record 172 is provided to PU 150, to position The end effector 140 and/or clamper system 146 of Figure 1A and Figure 1B.Robot system 100 can by end effector 140 and/ Or clamper system 146 is placed on the CoM of package 112 and puts to grasp and be promoted package 112, to facilitate in package 112 It is related to what the clamping of balance package was promoted with horizontal wrap when the position on the destacking platform 110 of Figure 1A clamps and is promoted from it Operation.Package 112 can be by grasping, promotion, the receiving platform 120 of transport to Figure 1A and the release at receiving platform 120.

Fig. 4 A shows the second exemplary package stacking according to some embodiments of this technology (for example, package 112- 31 to 112-54) perspective view.Fig. 4 B shows the second example stacking of the package of some embodiments according to this technology Top view.Fig. 3 B can correspond to the field for overlooking these packages captured by the sensor 162 of the picture system 160 of Figure 1A and Figure 1B The image and/or point cloud data of scape.It is merely to illustrate and not restrictive, package 112-33,112-43 and 112-51 can be indicated The registration example of package 112, it includes the registration record 172 in the RDS 170 for being stored in Figure 1B of Figure 1B that each example, which has, Respective instance in SI.Only for discussing that package 112 is rectangular shape with for purpose of brevity.

In addition, package 112-31,112-32,112-44 and 112-52 to 112-54 can indicate package 112 be not registrated and/ Or error handle/matched example, the registration record 172 being stored in the RDS 170 of Figure 1B may not corresponded to.Robot System 100 can be used indicate these be not registrated/the capture image data of unmatched SI as the first image data and can be registrated SI, as described below.Only for discussing and for purpose of brevity, package 112 is rectangular shape, and each package have by it is vertical, SI shown in horizontal and/or cross-hauling.

Fig. 4 C, 4E, 4G, 4I and 4L can show the top view of the stacking when removing one or more packages.Fig. 4 C is shown According to the exemplary symbols of some embodiments of this technology being registrated and be not registrated package.Fig. 4 C is shown such as Figure 1B The symbol of rectangle 112-33a, 112-43a and 112-51a shown in display unit 164.As described above, the SI of package 112 can be by The picture system 160 of Figure 1B captures, and is compared with the SI for being registrated record 172 in the RDS170 for being stored in Figure 1B of Figure 1B Compared with.If registration, the symbol that instruction SI is registered can be distributed and/or be shown to robot system.As shown, symbol 112- 33a, 112-43a and 112-51a may include the rectangular profile shown by display unit 164.

As shown in Figure 4 C, inner rectangular 112-61a to 112-61d (including end value) outer rectangular 112-62a extremely Combination in 112-62d (including end value) can be for package 112-31,112-44 and 112-53's on display unit 164 It is not registrated SI and is shown as symbol 112-31a, 112-44a and 112-53a.In some embodiments, inner rectangular 112-61a is extremely The size of 112-61d and placement can be overlapped with minimum feasible region (MVR) discussed in more detail below.Symbol 112-31a, 112- The presence of 44a and 112-53a can indicate that the SI of package 112-31,112-44 and 112-53 do not have and/or mismatch registration note Record 172.In this case, image recognition algorithm (for example, limb recognition algorithm), which can be used for determining, forms at least the two of corner The presence of a sharp edge.

Fig. 4 D shows the example at unintelligible edge and MVR according to some embodiments of this technology.Fig. 4 D is shown Be not registrated/mismatch package two sharp edge 112-31b for being respectively formed corner 112-31c, 112-44c and 112-53c, 112-44b and 112-53b (for example, the edge for forming the outer edge of stacking/be overlapped with the outer edge of stacking).Moreover, two not Sharp edge 112-31d, 112-44d and 112-53d can by 150 identifications of PU/estimation, be respectively formed corner 112-31e, 112-44e and 112-53e.In some embodiments, two unintelligible edge 112-31d of the package 112-31 of Fig. 4 A can lead to It crosses and assumes that they are extended up to from end 112-31ba and 112-31bb with predetermined angle (for example, vertically) respectively Intersection is at the 112-31e of corner to handle.Similarly, wrap up 112-44 two unintelligible edge 112-44d can by assuming that it Extend up to respectively from end 112-44ba and 112-44bb with predetermined angle (for example, vertically) in corner 112- Intersection is at 44e to handle.Similarly, wrap up 112-53 two unintelligible edge 112-53d can by assuming that they respectively from End 112-53ba and 112-53bb extends up to the phase at the 112-53e of corner with predetermined angle (for example, vertically) It hands over to handle.In some embodiments, robot system 100 can according to the associated depth measurement of corresponding package, positioning/ Position, determination posture or combinations thereof select or adjust predetermined angle.

In addition, showing exemplary MVR 112-31f, 112-44f and 112-53f in Fig. 4 D.In some embodiments, It can not be registrated example calculation MVR for each of package 112, which can be used for creating the registration record 172 of Figure 1B.Robot System 100 can create registration record 172 according to its corresponding MVR.MVR 112-31f can be calculated, so that its side is clear with two Clear edge 112-31b or part thereof is conllinear.In other words, the size of MVR 112-31f is smaller than the surface of package 112-31.In In some embodiments, the size of MVR 112-31f can according to fig. 2 the size of clamper system 146 shown in B or with wrapping up 112 grasp when being elevated and keep other corresponding features of ability of grasping safely to set.When handling MVR, robot System 100 may be assumed that and using the default CoM of package 112.Therefore, using default CoM and calculated size, robot system 100 can calculate MVR 112-31f, prevent the mistake in the associated manipulation (for example, clamping, promotion and/or transport) of package 112 Accidentally/failure.For example, clamper system 146 may misplace position, and when package 112 is elevated when MVR is too small, clamping The lasting grasping of device system 146 may be impossible.In some embodiments, the size of MVR 112-31f may be set to Exclude the straight line of a part as surface designs and/or the label that occurs in SI, such as by image recognition algorithm it is true Fixed.In some embodiments, the size of MVR 112-31f may be set to exclude to correspond to the side compared with low confidence measure Edge.Although should be pointed out that the discussion of front is directed to MVR 112-31f, the function for calculating MVR 112-31f is suitable Any MVR for being discussed herein.

Fig. 4 E shows exemplary according to second group for being registrated and not being registrated package of some embodiments of this technology Symbol.Fig. 4 E can indicate the state of the package stacking after the stacking removal shown in Fig. 4 C/transport one or more package. The symbol 112-33a for lacking Fig. 4 C can be indicated from the grasping of the destacking platform 110 of Figure 1A, the package of registration for being promoted and being transported 112-33.Moreover, robot system 100 can on display unit 164 generation/display symbol 112-32a, thus the new identification of instruction To formed shown in Fig. 4 B wrap up 112-32 corner two sharp edges (for example, now exposure and no longer adjacent to any Adjacent edge/package edge).

Fig. 4 F shows second example at unintelligible edge and MVR according to some embodiments of this technology.Fig. 4 F shows The sharp edge 112-32b for forming corner 112-32c is gone out, said one or multiple processes has been used to identify.According to this Identification, robot system 100 (for example, PU 150) can estimate two unintelligible edge 112-32d and corresponding angles in the above described manner Portion 112-32e, and MVR 112-32f can be calculated in a manner of discussed above.

Fig. 4 G shows the third group exemplary symbols for not being registrated package of some embodiments according to this technology.Fig. 4 G It can indicate the state that stacking is wrapped up after the stacking removal shown in Fig. 4 E/transport one or more package.Lack Fig. 4 C and The symbol 112-43a and 112-51a of Fig. 4 E can be indicated from the registration of the grasping of the destacking platform 110 of Fig. 1, promotion and transport Wrap up 112-43 and 112-51.Although should be pointed out that the right hand edge of the package 112-52 of Fig. 4 B and left edge now can quilts Identify (for example, when they are exposed and no longer neighbouring adjacent edge/package), but these edges it is parallel to each other and It is non-intersecting to form corner；Therefore, in some embodiments, display unit 164, which may not be shown, corresponds to package 112-52 Symbol addition.

Fig. 4 H shows clamper system (for example, clamper system 146) according to some embodiments of this technology Exemplary placement above MVR (for example, MVR 112-44f of Fig. 4 D).Constitute the F- of the end effector 140 of Figure 1A and Figure 1B T sensor 142 and clamper system 146 can be oriented so that clamper system 146 is placed on above MVR 112-44f, and The sensor 162 of the picture system 160 of Fig. 1 will not be stopped to observe the unintelligible edge 112-44d of Fig. 4 D.Then, 112- is wrapped up 44 can check that distance arrives raised position by one section of promotion of grasping and vertical-lifting.Check that distance can indicate to be enough to supply for example, being promoted Figure 1B and Fig. 2A F-T sensor 142 capture package 112-44 one or more axle powers and/or axle power square measured value away from From.Check that distance can indicate the vertical distance greater than zero millimeter or the distance along z-axis for example, being promoted.For particular example, mention It rises and checks that distance can indicate 50 millimeters.

In some embodiments, with reference to cartesian coordinate system, along one or more axis (that is, F (x-axis), F (y-axis) And/or F (z-axis)) power measured value and/or power around one or more axis (that is, M (x-axis), M (y-axis) and/or M (z-axis)) Square measured value can be captured via F-T sensor 142.By applying CoM computational algorithm, the weight of package can according to power measured value come It calculates, and the CoM wrapped up can be calculated according to power measured value and torgue measurement value.These measured values can be added to Figure 1B Wrap up the new example of the registration record 172 of 112-44 creation.In some embodiments, if the CoM calculated falls in MVR Except 112-44f, then wrapping up 112-44 can discharge from clamper system 146.Then, clamper system 146 can be positioned on calculating Above CoM out, wrapping up 112-44 at this time can safely be clamped and be promoted again to raised position.

When in a raised position, robot system can the package that is promoted of reimaging so that previously unsharp side Edge is apparent from.Indicate the second image data of the part SI of 112-44 (that is, entire SI's is not stopped by clamper system 146 Part) can be captured by the sensor 162 of the picture system 160 of Figure 1A and Figure 1B, to obtain the unintelligible edge 112- of Fig. 4 D The sharp edge 112-44g of 44d.In some embodiments, it is indicated in the first image data being made of the first point cloud data Depth information can be compared with the depth information indicated in the second image data being made of second point cloud data, with determination The variation of depth information can determine sharp edge and 2D measured value from the variation of depth information.Once obtaining, so that it may from the first figure As the SI of data and the sharp edge of the second image data generate the third image data for indicating that SI can be registrated.From the second picture number According to the registration note for being registrated the 2D measured value that SI is determined and being added to package 112-44 creation now with third image data Record 172.For another example, third image data can indicate the template image data of the robot system 100 of Figure 1A.From this In, by can be registrated SI, 2D measured value and wrap up 112-44 the weight for not being registrated example and CoM measurement set at registration The new example of record 172 is storable in now in the RDS 170 of Figure 1B.Robot system 100 can by will wrap up 112 SI with Template image data is compared to determine whether package 112 is not registrated.

Fig. 4 I shows the 4th group of exemplary symbols for not being registrated package of some embodiments according to this technology.Fig. 4 I It can indicate the state that stacking is wrapped up after the stacking removal shown in Fig. 4 G/transport one or more package.Lack Fig. 4 C, figure The symbol 112-44a of 4E and Fig. 4 G can indicate establish registration record 172 new example after, wrap up 112-44 previously not Registration example is transported from destacking platform 110.Moreover, robot system 100 can on display unit 164 generation/display symbol 112-54a, so that instruction newly recognizes two sharp edges to form the corner for wrapping up 112-54 shown in Fig. 4 B (for example, In New exposed edge after the previously adjacent package of removal/transport).

Fig. 4 J shows the unintelligible edge of the package (for example, package 112-54) for newly handling and the example of MVR.Figure 4J shows the sharp edge 112-54b to form corner 112-54c, and said one or multiple processes has been used to identify.Root According to the identification, robot system 100 (for example, PU 150) can estimate two unintelligible edge 112-54d and right in the above described manner Corner 112-54e is answered, and MVR 112-54f can be calculated in a manner of discussed above.

Fig. 4 K shows clamper system (for example, clamper system 146) according to some embodiments of this technology Exemplary placement above the MVR (for example, MVR 112-32f) of Fig. 4 F.Constitute the F- of the end effector 140 of Figure 1A and Figure 1B T sensor 142 and clamper system 146 can be oriented so that clamper system 146 is placed on above MVR 112-32f, and The sensor 162 of the picture system 160 of Fig. 1 will not be stopped to observe unintelligible edge 112-32d.Then, at package 112-32 In the case where in raised position and in the above described manner, weight and CoM measured value can be determined and be added to package 112-32 The new example of the registration record 172 of creation；Indicate that the second image data of part SI of package 112-32 can be captured, from this Two image datas can determine 2D measured value and sharp edge 112-32g；And indicate that the third image data that can be registrated SI can be from The SI of first image data and the sharp edge of the second image data generate.From here, by SI, 2D measured value and packet can be registrated Wrap up in 112-32 the weight for not being registrated example and CoM measurement set at registration record 172 new example be storable in now In the RDS 170 of Figure 1B.

Fig. 4 L shows the 5th group of exemplary symbols for not being registrated package of some embodiments according to this technology.Fig. 4 L It can indicate the state that stacking is wrapped up after the stacking removal shown in Fig. 4 I/transport one or more package.Lack Fig. 4 E, 4G It can indicate to wrap up not being registrated previously for 112-32 after the new example for establishing registration record 172 with the symbol 112-32a of 4I Example is transported from destacking platform 110.Moreover, robot system 100 can on display unit 164 generation/display symbol 112- 52a, thus instruction newly recognize to be formed shown in Fig. 4 B wrap up 112-52 corner two sharp edges (for example, remove/ New exposed edge after the previously adjacent package of transport).

Fig. 4 M shows the unintelligible edge of the package (for example, package 112-52) for newly handling and the example of MVR.Figure 4M shows the sharp edge 112-52b to form corner 112-52c, and said one or multiple processes has been used to identify.Root According to the identification, robot system 100 (for example, PU 150) can estimate two unintelligible edge 112-52d and right in the above described manner Corner 112-52e is answered, and MVR 112-52f can be calculated in a manner of discussed above.

Fig. 4 N shows clamper system (for example, clamper system 146) according to some embodiments of this technology Exemplary placement above the MVR (for example, MVR 112-52f) of Fig. 4 M.Constitute the F- of the end effector 140 of Figure 1A and Figure 1B T sensor 142 and clamper system 146 can be oriented so that clamper system 146 is placed on above MVR 112-52f, and The sensor 162 of the picture system 160 of Fig. 1 will not be stopped to observe unintelligible edge 112-52d.Then, at package 112-52 In the case where in raised position and in the above described manner, weight and CoM measured value can be determined and be added to package 112-52 The new example of the registration record 172 of creation；Indicate that the second image data of part SI of package 112-52 can be captured, from this Two image datas can determine 2D measured value and sharp edge 112-52g；And indicate that the third image data that can be registrated SI can be from The SI of first image data and the sharp edge of the second image data generate.From here, by SI, 2D measured value and packet can be registrated Wrap up in 112-52 the weight for not being registrated example and CoM measurement set at registration record 172 new example be storable in now In the RDS 170 of Figure 1B.

Fig. 4 O shows clamper system (for example, clamper system 146) according to some embodiments of this technology Exemplary placement above the MVR (for example, MVR 112-54f) of Fig. 4 J.Constitute the F- of the end effector 140 of Figure 1A and Figure 1B T sensor 142 and clamper system 146 can be oriented so that clamper system 146 is placed on above MVR 112-54f, and The sensor 162 of the picture system 160 of Fig. 1 will not be stopped to observe unintelligible edge 112-54d.Then, at package 112-54 In the case where in raised position and in the above described manner, weight and CoM measured value can be determined and be added to package 112-54 The new example of the registration record 172 of creation；Indicate that the second image data of part SI of package 112-54 can be captured, from this Two image datas can determine 2D measured value and sharp edge 112-54g；And indicate that the third image data that can be registrated SI can be from The SI of first image data and the sharp edge of the second image data generate.From here, by SI, 2D measured value and packet can be registrated Wrap up in 112-54 the weight for not being registrated example and CoM measurement set at registration record 172 new example be storable in now In the RDS 170 of Figure 1B.

Fig. 4 P shows clamper system (for example, clamper system 146) according to some embodiments of this technology Exemplary placement above the 2nd MVR (for example, MVR 112-31f) of Fig. 4 D.Constitute the end effector 140 of Figure 1A and Figure 1B F-T sensor 142 and clamper system 146 can be oriented so that clamper system 146 is placed on above MVR112-31f, Sensor 162 without stopping the picture system 160 of Fig. 1 observes unintelligible edge 112-31d.Then, in package 112-31 In the case where in raised position and in the above described manner, weight and CoM measured value can be determined and be added to package 112- The new example of the registration record 172 of 31 creations；Indicate that the second image data of the part SI of package 112-31 can be captured, from this Second image data can determine 2D measured value and sharp edge 112-31g；And indicate that the third image data that can be registrated SI can The sharp edge of SI and the second image data from the first image data generate.From here, by can be registrated SI, 2D measured value and Be not registrated package 112-31 weight and CoM measurement set at registration record 172 new example be storable in Figure 1B now RDS 170 in.

Fig. 4 Q shows clamper system (for example, clamper system 146) according to some embodiments of this technology Exemplary placement above the 3rd MVR (for example, MVR 112-53) of Fig. 4 D.Constitute the end effector 140 of Figure 1A and Figure 1B F-T sensor 142 and clamper system 146 can be oriented so that clamper system 146 is placed on MVR 112-53f Side, without stopping the sensor 162 of picture system 160 of Fig. 1 to observe unintelligible edge 112-53d.Then, in package 112- 53 in raised position and in the above described manner in the case where, weight and CoM measured value can be determined and be added to package The new example of the registration record 172 of 112-53 creation；Indicate that the second image data of the part SI of package 112-53 can be captured, It can determine 2D measured value and sharp edge 112-53g from second image data；And indicate the third picture number that can be registrated SI According to can be generated from the SI of the first image data and the sharp edge of the second image data.From here, by SI, 2D measured value can be registrated And package 112-53 the weight for not being registrated example and CoM measurement set at registration record 172 new example now may be used It is stored in the RDS 170 of Figure 1B.

Fig. 5 shows the control flow 200 of the illustrative methods of the robot system 100 for operation diagram 1A.Control stream Journey 200 may include that the package 112 not being registrated is registrated during wrapping up processing operation, and wherein the PU 150 of Figure 1B can be programmed or match It is right with the module (for example, circuit, function, computer/device executable instruction or combinations thereof) that embodies in control flow 200 to be equipped with The instruction answered.In some embodiments, PU 150 can for the mechanical arm system 130 of Figure 1B, Figure 1B end effector 140, The picture system 160 of Figure 1B, the RDS 170 of Figure 1B, independent control and/or be adapted for carrying out in any other system of task are looked for The combination of the processor or processor that arrive such as, but not limited to has one or more moreover, PU 150 can be the processor of module The Printed Circuit Card of a input interface receives and provides data to facilitate the bidirectional data communication of PU 130.In order to realize Embodied in control flow 200 with lower module, receive data and obtain and/or retrieve data be it is synonymous and/or interchangeable, And it is synonymous and/or interchangeable for providing data with supply or supply data.

Robot system 100 may include trapping module 202.Trapping module 202 captures SI as the first image data.Example Such as, trapping module 202 can capture the first image data using the sensor 162 of Figure 1B.More specifically, as an example, capture mould Block 202 can operate picture system 160 and/or interact with picture system 160, from the capture of picture system 160 and/or reception and such as The corresponding imaging data of top view of stacking shown in Fig. 3 B and/or Fig. 4 B.Trapping module 202 can handle obtained imaging number The top surface of the package 112 of Figure 1A is captured accordingly as the first image data.In some embodiments, trapping module 202 can It is parsed into using one or more image recognition algorithms (for example, context image classification, pattern-recognition and/or edge detection) As data and identify it is therein package 112 edge and/or surface.Based on processing result (for example, the edge that is identified and/or Continuous surface), the part (for example, set of pixel value and/or depth reading) of imaging data can be identified as by trapping module 202 Indicate the top surface of each package.Trapping module 202 can will correspond to the first figure of one or more top surfaces individually wrapped up As data are transferred to regions module 204.

Robot system 100 may include regions module 204, can be connected to trapping module 202.Regions module 204 calculates MVR.For example, regions module 204 can record 172 or combinations thereof based on the registration of package 112, Figure 1B to calculate MVR.

In some embodiments, trapping module 202 can by received image data (for example, first image data) with match Quasi- record 172 is compared.For example, trapping module 202 can by the first image data and/or its any processing result (for example, from The estimation of sizes/dimensions derived from image data and/or visual indicia) with existing description/mould of package that is known or previously encountering Plate is compared.Based on the comparison, regions module 204 can determine the first image data whether be stored in be registrated record 172 in Known or the package that previously encounters corresponding informance matching.

Whether regions module 204 can calculate MVR in many ways, including be for example registered based on package 112 as registration note 172 are recorded to calculate MVR (for example, whether the first image data returns to matching compared with registration record 172 carries out).More specifically Ground, as an example, if the first image data be registrated record 172 in one matching (for example, package 112 be registered for One or more of quasi- record 172), then regions module 204 can avoid calculating MVR.On the contrary, regions module 204 can be first Image data calculates MVR with any be registrated when record 172 mismatches.For example, the first image data captured can indicate to wrap up 112 top surface for not being registrated example.Since package 112 is not registrated, some examples at the edge of package 112 may be unclear It is clear.More specifically, as an example, Fig. 4 B's is not registrated the unintelligible edge 112-31d that package 112-31 includes Fig. 4 D.Region mould Block 204 can recognize one group of clear and/or unsharp edge.In some embodiments, regions module 204 can recognize clear side Edge (for example, the peripheral edge of formation stacking/be overlapped with the peripheral edge of stacking and do not wrap up adjacent side with any other Edge) and use them as referring to.Therefore, sharp edge can be used to determine MVR as described above (for example, root in regions module 204 According to size/characteristic of clamper system 146).Regions module 204 can determine the package for being less than and capturing in the first image data The MVR of the area on 112 surface.

For particular example, the first image data for wrapping up 112-31 may include two sharp edge 112-31b of Fig. 4 D. Regions module 204 can make two clear sides by extending two sharp edge 112-31b and two unintelligible edge 112-31d The boundary that edge 112-31b intersects to predict the surface region of package 112-31 with two unintelligible edge 112-31d.Regions module 204 can be by determining that MVR is less than the boundary of surface region come the feature modeling MVR based on surface region.In some embodiments In, recognizable (for example, being based on image data) the package surface of regions module 204 intersects with each other to form a pair in the first corner Edge (for example, clear and/or unintelligible edge).This can be used to edge to determine MVR in regions module 204, such as pass through through Unintelligible edge is estimated by making to extend towards one another perpendicular to a pair of of line to edge.Therefore, regions module 204 can be by phase Pair corner be estimated as this to it is unintelligible/extend a pair of of line intersection point.Regions module 204 can from first pair of edge with clamping The corresponding point of one or more size/features of device system 146 extends unintelligible/vertical line.

For another example, appearing in layout on the SI of package 112 and/or label may include straight line.Straight line can It can be erroneously interpreted as the edge on the surface of package 112.In order to reduce the latent fault identification at edge, regions module 204 can be arranged MVR is calculated except the part with straight line on surface.More specifically, as an example, regions module 204 can be calculated less than including straight The MVR on the boundary of the surface region of line.

For different examples, regions module 204 can calculate MVR based on the position of package 112.For example, pallet can be It include 112 (have been registrated and/or have not been registrated) of more than one package on the destacking platform 110 of Figure 1A, as shown in Fig. 3 A/ Fig. 4 B. From group package, regions module 204 can position of the example based on package 112 relative to another example of package 112 It sets to calculate MVR.As discussed, wrap up all edges of 112 registration example be all it is known, such as based on will wrap up Exposed surface is actively matched with registration data and is measured using the previously stored size/dimension corresponding to registration data.

On the contrary, package 112 some edges for not being registrated example may be it is unknown, such as due to package 112-52 do not match It is quasi-.In addition, unknown parcels (for example, package 112-52) can be surrounded by other packages, such as package 112-31,112-32,112- 33,112-43,112-44,112-51,112-53 and/or 112-54, as shown in Figure 4 B.Since the edge of 112-52 is unknown , therefore the SI of 112-52 can be Chong Die with the surface image of other packages 112.

In some cases, according to registration record 172, one or more of surrounding package (for example, package 112-32) Can also for be not registrated/it is unmatched, this may introduce about package 112-52 residue/unknown edge does not know further Property.In the case where not establishing clear boundary between package 112-52 and package 112-32, the SI and package of 112-52 are wrapped up The SI of 112-32 can overlap each other.

On the contrary, although package 112-31 can also be not to be registrated/unmatched, the detectable package 112- of sensor 162 31 unique positions/state relative to destacking platform 110 and/or other packages.For example, regions module 204 can be in package 112- Not neighbouring/adjacent another in 31 at least one edge determines that package 112-31 meets predetermined position/shape when wrapping up 112 State.More specifically, as an example, the corner 112-31c of two edge 112-31b and Fig. 4 B of Fig. 4 B can be clear in the first image It is clear visible and/or correspond to and the depth measurement (example of different (for example, being located above) those of at surrounding/adjacent level position Such as, for continuous surface).Moreover, for example, regions module 204 can determine that package 112-31 and/or its outer edge correspond to destacking The corner of platform 110 and/or exterior section.

Therefore, regions module 204 can determine package 112 based on the visibility at the edge of package 112, corner or combinations thereof The example that is not registrated be located at or near the outer periphery of destacking platform 110.In some embodiments, regions module 204 can be into One step determination is (such as previous due to removing for example, adjacent not with other packages along the exposure of one or more horizontal directions Adjacent package) package 112 be not registrated example.Regions module 204 can preferentially calculate package 112 and not be registrated example MVR, this, which is not registrated example and is not registrated above package at other, exposes and/or external (for example, being located in stacking/layer level It is not registrated package 112-52112 at portion part).

In some embodiments, regions module 204 can preferentially calculate the MVR of the package with greater number edge, this A little edge clears are visible and/or expose above the package with less such edge.Therefore, robot system 100 can reduce Using the risk wrapped up with the SI crawl of another 112 overlapping of package and reduce corresponding clamping/promotion failure.For shown in 4B Example, regions module 204 can package 112-32, determine before 112-54 112-52 package 112-31,112-53 and/or The MVR of 112-44.MVR can be transferred to hoisting module 206 by regions module 204.

Regions module 204 can calculate MVR based on two unintelligible corners.More specifically, as an example, two unintelligible Corner can by least three unintelligible edges, two sharp edges and a unintelligible edge or two unintelligible edges and The group of one sharp edge is combined into.As described above, regions module 204 can by extend each edge with another edge phase Hand over form corner predict package 112 surface boundary.Regions module 204 can be by determining that it is unclear by two that MVR is less than The boundary that clear corner is formed calculates MVR based on the boundary of the surface region formed by three edge/two corners.

Robot system 100 calculates MVR dynamically and in real-time and provides improvement when not being registrated example in crawl package 112 Accuracy and performance.By calculating MVR, robot system 100 can estimate the surface region of package 112 (for example, corresponding side Edge/boundary), wherein the clamper system 146 of Fig. 4 H can safely grab package 112.As a result, robot system 100 can be not Package 112 will be registrated in the case where the process that stops working and is transmitted to robot system 100, will be unloaded with improving for 112 will to be wrapped up The performance of the workflow of pile.

Robot system 100 may include hoisting module 206, can be connected to regions module 204.Hoisting module 206 is realized (for example, via transmitting and/or executing) promotes the order of package 112 for the mechanical arm 132 of Figure 1A.For example, as described above, mentioning Rising mould block 206 can operating robotic arm 132 one section of promotions inspection distance promoted with the example that is not registrated that will wrap up 112.More specifically, As an example, mechanical arm 132 can promote package 112 by grabbing in the MVR of package 112 using clamper system 146, In at least one of unintelligible edge sensor 162 is visible.In other words, for not being registrated/unidentified packet Wrap up in, hoisting module 206 can operating robotic arm 132 clamper system 146 to be directly placed on MVR to/top/contact MVR.Cause This, hoisting module 206 can grab and be promoted corresponding package, while make one or more unintelligible edges not by clamper system 146 cover and are exposed to sensor 162/ and can be observed by sensor 162.

Hoisting module 206 can operate in many ways.For example, hoisting module 206 is executable to promote order, so that mechanical arm 132 grab package 112 in MVR, wherein unintelligible edge is visible sensor 162.For specific example, such as Fig. 4 D Shown, mechanical arm 132 can promote the package 112-44 of Fig. 4 D in the 112-44f of Fig. 4 D.The SI for wrapping up 112-44 may include two A sharp edge 112-44b and corner 112-44c.The SI of package 112-44 may also include two unintelligible edge 112-44d.Such as Shown in Fig. 4 H, mechanical arm 132 can promote package 112-44, the unintelligible edge 112- of two of them by crawl package 112-44 44b is visible sensor 162.In the case where not utilizing MVR, clamper system 146 can prevent edge from being sensed Package is grabbed at the position that device 162 detects.

For different examples, hoisting module 206 can determine the weight of package 112.More specifically, as an example, being promoted The weight for not being registrated example of the determining package 112 of the F-T sensor 142 of Figure 1B can be used in module 206.Hoisting module 206 can incite somebody to action The weight of package 112 is transferred to registration module 212.

In some embodiments, trapping module 202 can be mentioned after promotion and/or and grabbing on MVR The SI for not being registrated example of package 112 is further captured when rising as the second image data.More specifically, as an example, capture Module 202 can capture the second image data based on the package 112 for promoting inspection distance has been elevated, to include the present of Fig. 4 H Visible two sharp edge 112-44g.In other words, hoisting module 206 can be improved initially unclear by promoting package 112 The visibility and/or clarity at clear edge.Therefore, trapping module 202 can capture the second image data (for example, mentioning for being in Rise the package 112 of state/position), the clearly previous unsharp edge of identification/display.Trapping module 202 can be by the second figure As data are transferred to extraction module 208.

Robot system 100 may include extraction module 208, can be connected to hoisting module 206.Extraction module 208 extracts Third image data.For example, extraction module 208 can extract based on the first image data, the second image data or combinations thereof Three image datas.

Extraction module 208 can extract third image data in many ways.For example, extraction module 208 can be based on comparing the One image data and the second image data determine image difference.Image difference can indicate the phase for not being registrated example of package 112 With the difference between the first image data and the second image data of example.

More specifically, as an example, the first image data may include layout and/or label with package 112 SI.However, since package 112 is not registrated, wrapping up 112 edge may be unintelligible or not can determine that.Therefore, the first image data It may include the SI for wrapping up 112, blur margin is clear or Chong Die with the SI of another package 112.For another example, the second picture number According to may include wrap up 112 be elevated promoted check distance after SI.More specifically, as an example, the second image data It may include the SI for wrapping up 112, wherein after package 112 is elevated, previous unsharp edge (for example, edge 112-44b) Become sharp edge (for example, edge 112-44g).After package 112 is elevated, unintelligible edge can be apparent from, because Package 112 becomes package 112 adjacent with other and separates (for example, in higher height).Sensor 162 can distinguish different packets 112 are wrapped up in, because being elevated package 112 and adjacent package the distance between 112 or depth can be different from sensor 162.

Extraction module 208 can extract the based on the image difference between the first image data of combination and the second image data Three image datas.For example, the first image data of package 112-44 may include layout and/or label, two sharp edges 112-44b, corner 112-44c or combinations thereof.For another example, the second image data for wrapping up 112-44 may include two clear Clear edge 12-44g.Extraction module 208 can be by including layout and/or label, two sharp edge 112-44b, corners 112-44c, two sharp edge 112-44g or combinations thereof wrap up the third image data of 112-44 to extract.

For another example, extraction module 208 can be determined based on third image data the length of package 112, width or its Combination.More specifically, extraction module 208 can determine including length, width or combinations thereof as an example, being based on sharp edge Size.Third image data, length, width or combinations thereof can be transferred to registration module 212 by extraction module 208.

Third image data is extracted dynamically and in real-time provides the identification of robot system 100 and crawl package 112 not It is registrated the improved performance and accuracy of example.By extracting third image data, robot system 100 can recognize package 112 Edge be different from it is another package 112.By clearly identifying that the border/edge of package 112, robot system 100 can be high Clamper system 146 is placed on package 112 to grab and transmit package 112 securely by effect ground.As a result, robot system 100, which can continue not to be registrated package 112, is transmitted to robot system 100, to improve the workflow for that will wrap up 112 destackings Performance.

In order to schematically be illustrated, hoisting module 206 is described as executing the order for promoting package 112, but mentions Rising mould block 206 can operate differently.For example, hoisting module 206 can determine CoM based on package 112 is promoted.More specifically, making For example, hoisting module 206 can determine whether the position of the clamping of clamper system 146 and promotion package 112 is wrapping up in MVR On 112 CoM.For another example, whether hoisting module 206 can determine CoM in the xy axis for being expressed as MVR.

Hoisting module 206 can determine CoM in many ways.For example, hoisting module 206 can be determined using F-T sensor 142 CoM, as described above.For another example, hoisting module 206 can be determined using F-T sensor 142, CoM algorithm or combinations thereof Whether CoM is under the surface region for being expressed as MVR.

Using F-T sensor 142 and CoM algorithm, hoisting module 206 may further determine that whether position/part in MVR is wrapping Wrap up in 112 be overlapped with CoM or the position including CoM at contact and clamper or covered by clamper.For another example, if original Clip position is not on the CoM of package 112, then hoisting module 206 can determine the new position in MVR for clamper system 146 clamping packages 112.More specifically, as an example, CoM algorithm discussed above can be used to determine clamping for hoisting module 206 Newly position is relative to the home position of clamping on CoM.For example, hoisting module 206 can be based on the torque of measurement and/or its side Always direction vector is calculated.Based on torque, hoisting module 206 can estimate downward force relative to F-T sensor 142 and/or clamping The position/orientation of device system 146.Moreover, hoisting module 206 can magnitude based on measured torque, promote the survey of package 112 Relationship between the weight of amount, clip position and package border/edge or combinations thereof calculates distance.Hoisting module 206 can be examined The new position to be clamped (for example, direction vector and distance) is looked into whether in MVR.If new position is on CoM, Lifting Modules Block 206 can verify that the CoM of package 112 is correctly determined.

If hoisting module 206 determines that the position of the clamping in MVR is not CoM, 206 executable command of hoisting module with By package 112 put down or be reduced to clamper system 146 it is boosted package 112 position.In addition, hoisting module 206 can be held Line command, so that new position crawl package 112 of the clamper system 146 in MVR is (for example, by reducing and discharging package 112, clamper system 146 is relocated, then again 112) clamping wraps up and makes mechanical arm 132 that package 112 is promoted one Duan Tisheng checks distance.For other example, clamper system 146 can be grabbed in the new position in MVR, be passed without stopping The unintelligible edge of the detection of sensor 162 package 112.Hoisting module 206 can transmit registration module 212.

Robot system 100 determines that the CoM of package 112 provides the transmission package of robot system 100 dynamically and in real-time The 112 improvement performance for not being registrated example.It is not registrated/the CoM of unidentified package, clamper system 146 by accurately identifying The stability of crawl package 112 is improved.As a result, robot system 100, which can continue not to be registrated package 112, is transmitted to machine People's system 100, to improve the performance of the workflow for 112 destackings will to be wrapped up.

In order to schematically be illustrated, trapping module 202 is described as capturing the first image data, the second image data Or combinations thereof, but trapping module 202 can operate differently.For example, as described above, it is correct determine CoM after, trapping module 202 can capture the image data clamped again for indicating the SI of package 112.More specifically, as an example, mechanical arm 132 can be Clamper system 146 promotes package 112 after the new position crawl package 112 in MVR.Trapping module 202 can capture table Show the image data of the SI of the package 112 of the new position clamping in MVR and above CoM clamped again.Trapping module Again the image data clamped can be transferred to extraction module 208 by 202.

In order to schematically be illustrated, extraction module 208 is described as based on the first image data, the second image data Or combinations thereof extract third image data, but extraction module 208 can operate differently.For example, extraction module 208 can be based on First image data, the image data clamped again or combinations thereof extract third image data, are similar to above for extraction Module 208 is based on the first image data, the second image data or combinations thereof and extracts described in third image data.Institute as above It states, extraction module 208 can determine length, the size of width or combinations thereof including package 112.Extraction module 208 can be by third Image data, length, width or combinations thereof are transferred to registration module 212.

Robot system 100 may include delivery module 210, can be connected to extraction module 208.Delivery module 210 executes Order will be will wrap up 112 receiving platforms 120 for being transmitted to Figure 1A.For example, 210 executable command of delivery module, so that mechanical arm The example for being registrated or not being registrated for wrapping up 112 is transmitted to receiving platform 120 by 132.

Delivery module 210 can execute order in many ways.For example, delivery module 210 can be based on the registration shape of package 112 State executes order to transmit package 112.More specifically, as an example, then being transmitted if there is the registration record 172 for wrapping up 112 210 executable command of module, so that the transmission of mechanical arm 132 package 112 of the registration example of crawl package 112 is received with being placed on On platform 120.

For different examples, if registration record 172 is not present, by the third figure as described above for extracting package 112 As data.In addition, 210 executable command of delivery module, so that the mechanical arm 132 for not being registrated example of crawl package 112 will wrap It wraps up in 112 and is transmitted to receiving platform 120.For another example, when package 112 is reduced to receiving platform 120 by mechanical arm 132, The bottom range of package 112 can trigger the HDS 180 of Figure 1A.

HDS 180 can be predefined relative to the height on floor, because the height of receiving platform 120 can be predefined.It passes Send module 210 that can wrap up based on the bottom of package 112 across the time of HDS 180 and the height of clamper system 146 to determine 112 height.More specifically, as an example, delivery module 210 can be based on receiving signal (that is, indicating that package 112 passes through HDS 180) position of clamper system 146 or the distance between height and the predefined height of HDS 180/difference are wrapped up when to determine 112 height.Delivery module 210 can will wrap up 112 high transmission to registration module 212.

Robot system 100 determines that the height for not being registrated example of package 112 provides system, robot dynamically and in real-time System 100 will wrap up the improvement performance of 112 destackings.It is not registrated/the height of unidentified package by determining, robot system 100 can Accurately the attribute of identification package 112 is safely to grab package.As a result, robot system 100 can continuously transmit it is mutually similar The performance of workflow of the package 112 of type with improvement for 112 destackings will to be wrapped up.

Robot system 100 may include registration module 212, can be connected to delivery module 210.Registration module 212 is registrated The attribute of package 112.Do not match for example, registration module 212 can be registrated (for example, by bundling or being stored together) package 112 The third image data of quasi- example, length, width, height, weight, CoM or combinations thereof.More specifically, as an example, matching quasi-mode Block 212 produces registration record 172, and the instance transfer that is not registrated that will wrap up 112 is the registration example for wrapping up 112.

It should be pointed out that the step of above method, can in the computer being stored in non-transitory computer-readable medium It reads to realize in medium as computer instruction code.This method may include one or more steps as described herein, this Or multiple steps can be any desired sequence execute, including execute simultaneously with one another.For example, it is disclosed herein two or more Step can combine in a single step and/or one or more steps can be used as two or more sub-steps to execute.In addition, Not explicitly disclosed herein or the step of being inherently present, can intert or be added in steps described herein, or can replace herein The one or more steps, as benefit from the disclosure those skilled in the art will appreciate that.

Other examples:

Example 1: a method of for being registrated package during robot wraps up processing operation, which comprises

Receive the first image data for indicating the first surface image on package surface；

It is compared to determine that the package is not registrated with record is registrated based on by the first surface image；

A pair of of edge that the package surface is identified based on the first image data, wherein the pair of edge is along dampening Square to exposure and intersect with each other with formed it is described package surface corner；

Minimum feasible region is determined based on the pair of edge, wherein in the minimum feasible region and the pair of edge One or two overlapping and/or be overlapped；

One or more orders are generated, to execute for (1) using the end for being located in the minimum feasible overlying regions Device clamps the package, and (2) promote the package；

Receive the second image data of the package after indicating the promotion；

Third image data is determined according to the first image data and second image data；And

The registration data that the new registration record of the package is indicated based on third image data creation, for subsequent Identify other packages.

Example 2: in above example 1, further include:

Unintelligible edge is estimated based on the first image data, wherein the pair of edge and the unintelligible edge pair The boundary on surface is wrapped up described in Ying Yu；And

Wherein:

The minimum feasible region with the unintelligible imbricate, to clamp the package, while not making the packet The expose portion wrapped up in corresponds to the unintelligible edge of the estimation.

Example 3: in any of the above example, further include:

Based on the one or more new edges of second image data identification, wherein the new edge indicates the package The actual exposed edge at the unintelligible edge corresponding to the estimation；And

Wherein:

The third image data includes one or more of new edges.

Example 4: in any of the above example, wherein identifying that one or more of new edges include based in the promotion packet The difference of the depth data measured later is wrapped up in identify one or more of new edges.

Example 5: in any of the above example, wherein determining that the third image data includes:

Calculate the difference between the first image data and second image data；And

It combines the first image data with the difference of the calculating.

Example 6: in any of the above example, further include:

Straight line is identified based on the first image data；

Wherein:

The minimum feasible region is the continuum for excluding the straight line.

Example 7: in any of the above example, further include:

Calculate the height for indicating the size of the package along vertical direction；And

Wherein:

The registration data includes the height of the calculating.

Example 8: in any of the above example, wherein calculating the height and including:

Detection triggers, the trigger indicate that the package determines sensor across with the height being located at known altitude Time when associated horizontal detection plane；

Vertical position associated with the trigger is determined, wherein the vertical position indicates that the end effector exists The vertical position of the time when package is across the horizontal detection plane；And

The height is calculated based on the difference between the known altitude and the vertical position.

Example 9: in any of the above example, further include:

Receive data after promoting the package, wherein the tables of data show from be connected to the end effector or The power measured value of the F-T sensor integral with the end effector；

The weight of the package is determined based on the received data；And

Wherein:

The registration data includes the weight of the determination.

Example 10: in any of the above example, further include:

Receive data after promoting the package, wherein the tables of data show from be connected to the end effector or The torque measurement of the F-T sensor integral with the end effector；

The centroid position of the package is determined based on the received data；And

Wherein:

The registration data includes the centroid position.

Example 11: a kind of for being registrated the system for not being registrated package, the system during robot wraps up processing operation Include:

At least one processor；With

At least one processor, at least one processor are connected at least one described processor, the memory Including instruction, described instruction can by least one described processor execute with:

Receive the first image data for indicating the first surface image on package surface；

It is compared to determine that the package is not registrated with record is registrated based on by the first surface image；

A pair of of edge that the package surface is identified based on the first image data, wherein the pair of edge is along dampening Square to exposure and intersect with each other with formed it is described package surface corner；

Minimum feasible region is determined based on the pair of edge, wherein in the minimum feasible region and the pair of edge One or two overlapping and/or be overlapped；

One or more orders are generated, to execute for (1) using the end for being located in the minimum feasible overlying regions Device clamps the package, and (2) promote the package；

The second image data of the package is received, wherein the packet after being promoted described in second pictorial data representation It wraps up in；

Third image data is determined according to the first image data and second image data；And

The registration data that the new registration record of the package is indicated based on third image data creation, for subsequent Identify other packages.

Example 12: in above example 11, wherein at least one processor includes instruction, described instruction is used for:

Unintelligible edge is estimated based on the first image data, wherein the pair of edge and the unintelligible edge pair The boundary on surface is wrapped up described in Ying Yu；And

Wherein:

The minimum feasible region with the unintelligible imbricate, to clamp the package, while not making the packet The expose portion wrapped up in corresponds to the unintelligible edge of the estimation.

Example 13: in above example 11-12, wherein at least one processor includes instruction, described instruction is used for:

Based on the one or more new edges of second image data identification, wherein the new edge indicates the package The actual exposed edge at the unintelligible edge corresponding to the estimation；And

Wherein:

The third image data includes one or more of new edges.

Example 14: in above example 11-13, wherein at least one processor includes for based in the promotion packet The difference of the depth data measured later is wrapped up in identify the instruction at one or more of new edges.

Example 15: in above example 11-14, wherein at least one processor includes instruction, described instruction is used for base The third image data is determined in the following terms:

Calculate the difference between the first image data and second image data；And

It combines the first image data with the difference of the calculating.

Example 16: a kind of non-transient storage media storing computer executable instructions, described instruction is by computing system The computing system is set to execute computer implemented method when execution, described instruction includes:

For receiving the instruction for indicating first image data of first surface image on package surface；

For being compared to determine the finger that the package is not registrated with record is registrated based on by the first surface image It enables；

For identifying the instruction at a pair of of edge on the package surface based on the first image data, wherein the pair of Edge is along horizontal direction exposure and intersects to form the corner for wrapping up surface；

For determining the instruction in minimum feasible region based on the pair of edge, wherein the minimum feasible region with it is described The overlapping of one or two of a pair of of edge and/or coincidence；

For generating the instruction of one or more orders, the order is utilized for (1) is located in the minimum feasible area End effector above domain clamps the package, and (2) promote the package；

For receiving the instruction of the second image data of the package, wherein being promoted described in second pictorial data representation The package afterwards；

For determining the instruction of third image data according to the first image data and second image data；With

For indicating the new registration data for being registrated record of the package to be used for based on third image data creation Then identify the instruction of other packages.

Example 17: in above example 16, wherein described instruction further include:

For estimating the instruction at unintelligible edge based on the first image data, wherein the pair of edge and it is described not Sharp edge corresponds to the boundary on the package surface；And

Wherein:

The minimum feasible region with the unintelligible imbricate, to clamp the package, while not making the packet The expose portion wrapped up in corresponds to the unintelligible edge of the estimation.

Example 18: in above example 16-17, wherein described instruction further include:

For the instruction based on the one or more new edges of second image data identification, wherein the new edge indicates The actual exposed edge at the unintelligible edge corresponding to the estimation of the package；And

Wherein:

The third image data includes one or more of new edges.

Example 19: in above example 16-18, wherein the described instruction at one or more of new edges includes for identification For identifying the finger at one or more of new edges based on the difference of the depth data measured after promoting the package It enables.

Example 20: in above example 11-15, wherein the described instruction for determining the third image data includes being used for Execute the following instruction operated:

Calculate the difference between the first image data and second image data；And

It combines the first image data with the difference of the calculating.

Example 21: a method of for being registrated package during robot wraps up processing operation, which comprises

Receive the first image data for indicating the first surface image on package surface；

It is compared to determine that the package is not registrated with record is registrated based on by the first surface image；

A pair of of edge that the package surface is identified based on the first image data, wherein the pair of edge is along dampening Square to exposure and intersect with each other with formed it is described package surface corner；

Minimum feasible region is determined based on the pair of edge, wherein in the minimum feasible region and the pair of edge One or two overlapping and/or be overlapped；

One or more orders are generated, to execute for (1) using the end for being located in the minimum feasible overlying regions Device clamps the package, and (2) promote the package；

Receive the second image data of the package after indicating the promotion；

Third image data is determined according to the first image data and second image data；

The registration data that the new registration record of the package is indicated based on third image data creation, for subsequent Identify other packages；And

Unintelligible edge is estimated based on the first image data, wherein the pair of edge and the unintelligible edge pair The boundary on surface is wrapped up described in Ying Yu；

Wherein:

The minimum feasible region with the unintelligible imbricate, to clamp the package, while not making the packet The expose portion wrapped up in corresponds to the unintelligible edge of the estimation.

Example 22: in above example 21, further include:

Based on the one or more new edges of second image data identification, wherein the new edge indicates the package The actual exposed edge at the unintelligible edge corresponding to the estimation；And

Wherein:

The third image data includes one or more of new edges.

Example 23: in above example 21-22, wherein identifying that one or more of new edges include based on described in the promotion The difference of the depth data measured later is wrapped up to identify one or more of new edges.

Example 24: in above example 21-23, wherein determining that the third image data includes:

Calculate the difference between the first image data and second image data；And

It combines the first image data with the difference of the calculating.

Example 25: in above example 21-24, further include:

Straight line is identified based on the first image data；

Wherein:

The minimum feasible region is the continuum for excluding the straight line.

Example 26: in above example 21-25, further include:

Calculate the height for indicating the size of the package along vertical direction；And

Wherein:

The registration data includes the height of the calculating.

Example 27: in above example 21-26, wherein calculating the height and including:

Detection triggers, the trigger indicate that the package determines sensor across with the height being located at known altitude Time when associated horizontal detection plane；

Vertical position associated with the trigger is determined, wherein the vertical position indicates that the end effector exists The vertical position of the time when package is across the horizontal detection plane；And

The height is calculated based on the difference between the known altitude and the vertical position.

Example 28: in above example 21-27, further include:

Receive data after promoting the package, wherein the tables of data show from be connected to the end effector or The power measured value of the F-T sensor integral with the end effector；

The weight of the package is determined based on the received data；And

Wherein:

The registration data includes the weight of the determination.

Example 29: in above example 21-28, further include:

Receive data after promoting the package, wherein the tables of data show from be connected to the end effector or The torque measurement of the F-T sensor integral with the end effector；

The centroid position of the package is determined based on the received data；And

Wherein:

The registration data includes the centroid position.

Example 30: a kind of for being registrated the system for not being registrated package, the system during robot wraps up processing operation Include:

At least one processor；With

At least one processor, at least one processor are connected at least one described processor, the memory Including instruction, described instruction can by least one described processor execute with:

Receive the first image data for indicating the first surface image on package surface；

It is compared to determine that the package is not registrated with record is registrated based on by the first surface image；

A pair of of edge that the package surface is identified based on the first image data, wherein the pair of edge is along dampening Square to exposure and intersect with each other with formed it is described package surface corner；

Minimum feasible region is determined based on the pair of edge, wherein in the minimum feasible region and the pair of edge One or two overlapping and/or be overlapped；

One or more orders are generated, to execute for (1) using the end for being located in the minimum feasible overlying regions Device clamps the package, and (2) promote the package；

The second image data of the package is received, wherein the packet after being promoted described in second pictorial data representation It wraps up in；

Third image data is determined according to the first image data and second image data；And

The registration data that the new registration record of the package is indicated based on third image data creation, for subsequent Identify other packages；And

Unintelligible edge is estimated based on the first image data, wherein the pair of edge and the unintelligible edge pair The boundary on surface is wrapped up described in Ying Yu；

Wherein:

The minimum feasible region with the unintelligible imbricate, to clamp the package, while not making the packet The expose portion wrapped up in corresponds to the unintelligible edge of the estimation.

Example 31: in above example 30, wherein at least one processor includes instruction, described instruction is used for:

Based on the one or more new edges of second image data identification, wherein the new edge indicates the package The actual exposed edge at the unintelligible edge corresponding to the estimation；And

Wherein:

The third image data includes one or more of new edges.

Example 32: in above example 30-31, wherein at least one processor includes for based in the promotion packet The difference of the depth data measured later is wrapped up in identify the instruction at one or more of new edges.

Example 33: in above example 30-32, wherein at least one processor includes instruction, described instruction is used for base The third image data is determined in the following terms:

Calculate the difference between the first image data and second image data；And

It combines the first image data with the difference of the calculating.

Example 34: a kind of non-transient storage media storing computer executable instructions, described instruction is by computing system The computing system is set to execute computer implemented method when execution, described instruction includes:

For receiving the instruction for indicating first image data of first surface image on package surface；

For being compared to determine the finger that the package is not registrated with record is registrated based on by the first surface image It enables；

For identifying the instruction at a pair of of edge on the package surface based on the first image data, wherein the pair of Edge is along horizontal direction exposure and intersects to form the corner for wrapping up surface；

For determining the instruction in minimum feasible region based on the pair of edge, wherein the minimum feasible region with it is described The overlapping of one or two of a pair of of edge and/or coincidence；

For generating the instruction of one or more orders, the order is utilized for (1) is located in the minimum feasible area End effector above domain clamps the package, and (2) promote the package；

For receiving the instruction of the second image data of the package, wherein being promoted described in second pictorial data representation The package afterwards；

For determining the instruction of third image data according to the first image data and second image data；

For indicating the new registration data for being registrated record of the package to be used for based on third image data creation Then identify the instruction of other packages；And

For estimating the instruction at unintelligible edge based on the first image data, wherein the pair of edge and it is described not Sharp edge corresponds to the boundary on the package surface；

Wherein:

The minimum feasible region with the unintelligible imbricate, to clamp the package, while not making the packet The expose portion wrapped up in corresponds to the unintelligible edge of the estimation.

Example 35: in above example 34, wherein described instruction further include:

For the instruction based on the one or more new edges of second image data identification, wherein the new edge indicates The actual exposed edge at the unintelligible edge corresponding to the estimation of the package；And

Wherein:

The third image data includes one or more of new edges.

Example 36: in above example 34-35, wherein the described instruction at one or more of new edges includes for identification For identifying the finger at one or more of new edges based on the difference of the depth data measured after promoting the package It enables.

Example 37: in above example 30-33, wherein the described instruction for determining the third image data includes being used for Execute the following instruction operated:

Calculate the difference between the first image data and second image data；And

It combines the first image data with the difference of the calculating.

Above to the exemplary detailed description of disclosed technology be not intended to exhaustion or by disclosed technical restriction be it is upper Precise forms disclosed in face.Although disclosed for illustrative purposes the specific example of disclosed technology above, disclosed In the range of technology, the modification of various equivalent forms be it is possible, as one skilled in the relevant art will recognize that.Though for example, Right process or module are presented with given sequence, but substitute implementation can be executed with different order the routine with step or Using the system with module, and some processes or module can be deleted, moved, added, subdivided, combined, and or modified with Substitution or sub-portfolio are provided.Each of these processes or module can be implemented in various different ways.Although moreover, sometimes Process or module are shown as serially executing, but these processes or module are alternatively executed or realized parallel, or can be Different time executes.In addition, any specific number being mentioned herein is only example；Substitution implementation can be used different value or Range.

As used herein, term " embodiment " means the embodiment for illustrating rather than limited.This field skill Art personnel will be understood that aforementioned exemplary and embodiment are exemplary the extensive model without limitation inventive concept disclosed herein It encloses.It will be apparent upon all modifications, displacement, increasing in reading specification and research attached drawing to those skilled in the art By force, equivalent and improvement are intended to be included in the broad range of inventive concept disclosed herein.Therefore, claims appended below All such modifications for being intended to include in the broad range for falling into inventive concept disclosed herein, enhancing, equivalent and change displacement Into.

Claims (17)

1. a kind of method for being registrated package during robot wraps up processing operation, which comprises

Receive the first image data for indicating the first surface image on package surface；

It is compared to determine that the package is not registrated with record is registrated based on by the first surface image；

A pair of of edge that the package surface is identified based on the first image data, wherein the pair of edge is along level side To exposure and intersect with each other to form the corner on the package surface；

Minimum feasible region is determined based on the pair of edge, wherein one in the minimum feasible region and the pair of edge A or two overlappings and/or coincidence；

One or more orders are generated, to press from both sides for (1) using the end effector for being located in the minimum feasible overlying regions The package is held, and (2) promote the package；

Receive the second image data of the package after indicating the promotion；

Third image data is determined according to the first image data and second image data；

The registration data that the new registration record of the package is indicated based on third image data creation, for then identifying Other packages；And

Unintelligible edge is estimated based on the first image data, wherein the pair of edge and the unintelligible edge correspond to The boundary on the package surface；

Wherein:

The minimum feasible region with the unintelligible imbricate, to clamp the package, while not making the package Expose portion corresponds to the unintelligible edge of the estimation.

2. the method as described in claim 1, further include:

Based on the one or more new edges of second image data identification, wherein the new edge indicates the correspondence of the package In the actual exposed edge at the unintelligible edge of the estimation；And

Wherein:

The third image data includes one or more of new edges.

3. method according to claim 2, wherein identifying that one or more of new edges include based in the promotion packet The difference of the depth data measured later is wrapped up in identify one or more of new edges.

4. the method as described in claim 1, wherein determining that the third image data includes:

Calculate the difference between the first image data and second image data；And

It combines the first image data with the difference of the calculating.

5. the method as described in claim 1, further include:

Straight line is identified based on the first image data；

Wherein:

The minimum feasible region is the continuum for excluding the straight line.

6. the method as described in claim 1, further include:

Calculate the height for indicating the size of the package along vertical direction；And

Wherein:

The registration data includes the height of the calculating.

7. method as claimed in claim 6, wherein calculating the height and including:

Detection triggers, the trigger indicate that the package determines that sensor is related across to the height being located at known altitude Time when the horizontal detection plane of connection；

Vertical position associated with the trigger is determined, wherein the vertical position indicates the end effector described The vertical position of time when wrapping up across the horizontal detection plane；And

The height is calculated based on the difference between the known altitude and the vertical position.

8. the method as described in claim 1, further include:

Receive data after promoting the package, wherein the tables of data show from be connected to the end effector or with institute State the power measured value of the integral F-T sensor of end effector；

The weight of the package is determined based on the received data；And

Wherein:

The registration data includes the weight of the determination.

9. the method as described in claim 1, further include:

Receive data after promoting the package, wherein the tables of data show from be connected to the end effector or with institute State the torque measurement of the integral F-T sensor of end effector；

The centroid position of the package is determined based on the received data；And

Wherein:

The registration data includes the centroid position.

10. it is a kind of for being registrated the system for not being registrated package during robot wraps up processing operation, the system comprises:

At least one processor；With

At least one processor, at least one processor are connected at least one described processor, and the memory includes Instruction, described instruction can by least one described processor execute with:

Receive the first image data for indicating the first surface image on package surface；

It is compared to determine that the package is not registrated with record is registrated based on by the first surface image；

A pair of of edge that the package surface is identified based on the first image data, wherein the pair of edge is along level side To exposure and intersect with each other to form the corner on the package surface；

Minimum feasible region is determined based on the pair of edge, wherein one in the minimum feasible region and the pair of edge A or two overlappings and/or coincidence；

One or more orders are generated, to press from both sides for (1) using the end effector for being located in the minimum feasible overlying regions The package is held, and (2) promote the package；

The second image data of the package is received, wherein the package after being promoted described in second pictorial data representation；

Third image data is determined according to the first image data and second image data；And

The registration data that the new registration record of the package is indicated based on third image data creation, for then identifying Other packages；And

Unintelligible edge is estimated based on the first image data, wherein the pair of edge and the unintelligible edge correspond to The boundary on the package surface；

Wherein:

The minimum feasible region with the unintelligible imbricate, to clamp the package, while not making the package Expose portion corresponds to the unintelligible edge of the estimation.

11. described instruction is used for system as claimed in claim 10, wherein at least one processor includes instruction:

Based on the one or more new edges of second image data identification, wherein the new edge indicates the correspondence of the package In the actual exposed edge at the unintelligible edge of the estimation；And

Wherein:

The third image data includes one or more of new edges.

12. device as claimed in claim 11, wherein at least one processor includes for based in the promotion packet The difference of the depth data measured later is wrapped up in identify the instruction at one or more of new edges.

13. device as claimed in claim 10, wherein at least one processor includes instruction, described instruction is for being based on The following terms determines the third image data:

Calculate the difference between the first image data and second image data；And

It combines the first image data with the difference of the calculating.

14. a kind of non-transient storage media for storing computer executable instructions, described instruction make when being executed by computing system The computing system executes computer implemented method, and described instruction includes:

For receiving the instruction for indicating first image data of first surface image on package surface；

For being compared to determine the instruction that the package is not registrated with record is registrated based on by the first surface image；

For identifying the instruction at a pair of of edge on the package surface based on the first image data, wherein the pair of edge Along horizontal direction exposure and intersect to form the corner for wrapping up surface；

For determining the instruction in minimum feasible region based on the pair of edge, wherein the minimum feasible region with it is the pair of The overlapping of one or two of edge and/or coincidence；

For generating the instruction of one or more orders, the order is utilized for (1) is located in the minimum feasible region The end effector of side clamps the package, and (2) promote the package；

For receiving the instruction of the second image data of the package, wherein after being promoted described in second pictorial data representation The package；

For determining the instruction of third image data according to the first image data and second image data；

For indicating the new registration data for being registrated record of the package for subsequent based on third image data creation Identify the instruction of other packages；And

For estimating the instruction at unintelligible edge based on the first image data, wherein the pair of edge and described unintelligible Edge corresponds to the boundary on the package surface；

Wherein:

The minimum feasible region with the unintelligible imbricate, to clamp the package, while not making the package Expose portion corresponds to the unintelligible edge of the estimation.

15. non-transient storage media as claimed in claim 14, wherein described instruction further include:

For the instruction based on the one or more new edges of second image data identification, wherein described in the new edge expression The actual exposed edge at the unintelligible edge corresponding to the estimation of package；And

Wherein:

The third image data includes one or more of new edges.

16. non-transient storage media as claimed in claim 15, wherein the institute at one or more of new edges for identification Stating instruction includes for one or more of to identify based on the difference of the depth data measured after promoting the package The instruction at new edge.

17. system as claimed in claim 10, wherein the described instruction for determining the third image data includes being used for Execute the following instruction operated:

Calculate the difference between the first image data and second image data；And

It combines the first image data with the difference of the calculating.