JP6704362B2 - Camera system for cloth gripping device - Google Patents

Description

The present invention relates to a camera system for a cloth gripping device regarding the start and end of the operation of a pair of gripping hands for a cloth gripping device that grips a cloth such as a towel and a towel.

In the so-called linen supply business, we supply towels, sheets, and other cloth pieces to hotels, restaurants, medical facilities, and the like. A large number of cloth pieces collected from the customer are washed and dried, and the cloth pieces are supplied to the customer again in a folded state. The cloth pieces are hand towels, towels, sheets and the like having various sizes, and are often rectangular shapes such as squares and rectangles.

A device for automation is introduced in a series of working steps of washing, drying and folding the large number of cloth pieces. However, automation does not cover the whole process with one device, but is often divided into a washing device, a drying device, a folding device and the like in terms of cost and compactness.

Further, an operator who delivers a piece of cloth from a device for each process to a device for the next process is often interposed between the processes. In particular, in the process of folding each piece of cloth, it may be necessary to unfold each piece of cloth in a rolled or overlapped state and align it in a certain direction.

Therefore, the applicant has developed a device for automatically detecting the edge portion of the suspended cloth piece (for example, refer to Patent Document 1). By using this device, the edge portion of the cloth piece can be detected, and the robot grips the edge portion to automate the work of spreading the cloth piece.

JP, 2014-45801, A

However, the above-mentioned device relates to a device for automatically detecting the edge portion of the suspended cloth piece, and is premised on the start of the operation, and the determination of the start and end of the operation is not particularly considered.

An object of the present invention is to provide a camera system for a cloth piece gripping device, which relates to the start and end of the operation of a pair of gripping hands for a cloth piece gripping device that grips a cloth piece.

A cloth piece gripping device camera system according to the present invention holds a suspended cloth piece by a pair of gripping hands, and linearly develops the sides of the cloth piece gripped by the pair of gripping hands. In the cloth piece gripping device camera system for starting and ending the operation of the pair of gripping hands of the cloth piece gripping device,
The suspended piece of cloth is photographed before the operation of the pair of gripping hands of the cloth piece gripping device is started, and the state in which the cloth piece is gripped by the pair of gripping hands of the cloth piece gripping device after the operation is started is shown. A camera to shoot
Based on the image of the suspended piece of cloth, an operation start determination for confirming whether or not the piece of cloth is within a predetermined range and determining whether to start the operation of the pair of gripping hands of the cloth piece gripping device Department,
Based on the image of the state of gripping the cloth piece by the pair of gripping hands of the cloth piece gripping device, it is determined whether the cloth piece gripped by the pair of gripping hands has been deployed. An operation end determination unit that confirms and determines the operation end of the pair of gripping hands of the cloth piece gripping device,
Equipped with.

According to the cloth piece grasping device camera system of the present invention, the start and end of the operation of the pair of grasping hands can be clearly determined, and thus the cloth piece grasping device incorporating the same can be appropriately controlled.

1 is a block diagram showing a configuration of a camera system for a cloth gripping device according to a first embodiment. FIG. 1 is a schematic perspective view showing an example of the configuration of a cloth piece gripping device including the cloth piece gripping device camera system according to the first embodiment. It is the schematic which shows the cloth piece suspended from the conveyor. 5 is a flowchart regarding determination of operation start of a pair of gripping hands of the cloth piece gripping device by the cloth piece gripping device camera system according to the first embodiment. 5 is a detailed flowchart of the operation start availability determination of FIG. 4. It is a schematic diagram showing a mode that a side of a piece of cloth is developed straight after grasping two corners of a piece of cloth with a pair of grasping hands. FIG. 6B is a schematic view showing the relationship between the pair of gripping hands and the cloth piece after being linearly developed in FIG. 6A. 5 is a flowchart regarding determination of the end of operation of a pair of gripping hands of the cloth piece gripping device by the cloth piece gripping device camera system according to the first embodiment. 8 is a detailed flowchart of the operation end availability determination of FIG. 7. FIG. 3 is a schematic diagram of a dual-arm robot in which a camera is provided on the upper part of the body.

A cloth piece gripping device camera system according to a first aspect holds a suspended cloth piece by a pair of gripping hands, and a side of the cloth piece gripped by the pair of gripping hands is linear. A camera system for a cloth piece gripping device, which relates to the start and end of the operation of the pair of gripping hands of the cloth piece gripping device that is developed into
Before the operation of the pair of grasping hands of the cloth piece grasping device, the suspended cloth piece is photographed and an image is acquired, and after the operation is started, the pair of grasping hands of the cloth piece grasping device grasps the cloth piece. A camera that captures images while capturing
Based on the image of the suspended piece of cloth, an operation start determination for confirming whether or not the piece of cloth is within a predetermined range and determining whether to start the operation of the pair of gripping hands of the cloth piece gripping device Department,
Based on the image of the state of gripping the cloth piece by the pair of gripping hands of the cloth piece gripping device, it is determined whether the cloth piece gripped by the pair of gripping hands has been deployed. An operation end determination unit that confirms and determines the operation end of the pair of gripping hands of the cloth piece gripping device,
Equipped with.

A cloth grip device camera system according to a second aspect is the camera system according to the first aspect, wherein the operation start determination unit is
A lowest point detection unit that detects the lowest point of the cloth piece from the image of the suspended cloth piece;
An operation start possibility determination unit that determines whether or not the pair of gripping hands of the cloth piece gripping device can start operating, based on whether the lowest point of the cloth piece is within a predetermined range,
May be provided.

A cloth gripping device camera system according to a third aspect is the above-described first or second aspect, wherein the operation end determination unit is
The operation end determination unit,
From the image of the state in which the gripping hand of the cloth piece gripping device is gripping the cloth piece, a straight line connecting the tip portions of the pair of gripping hands that grip the cloth piece, and the cloth piece. A distance detection unit that detects the maximum distance from the side curve of
An operation end availability determination unit that determines whether the maximum distance of the side is less than or equal to a predetermined value, and determines whether or not the operation ends of the pair of gripping hands of the cloth piece gripping device.
May be provided.

In the camera system for a cloth gripping device according to a fourth aspect, in any one of the first to third aspects, the camera may be provided on an intermediate surface of the pair of gripping hands.

Hereinafter, a camera for a cloth gripping device according to an embodiment will be described with reference to the accompanying drawings. In the drawings, substantially the same members are designated by the same reference numerals.

(Embodiment 1)
FIG. 1 is a block diagram showing a configuration of a cloth piece grasping device camera system 30 according to the first embodiment. FIG. 2 is a schematic perspective view showing an example of the configuration of the cloth piece gripping device 20 including the cloth piece gripping device camera system 30 according to the first embodiment. FIG. 3 is a schematic diagram showing the cloth piece 22 suspended from the conveyor 23. FIG. 6A is a schematic view showing a state in which the side 29 of the cloth piece 22 is linearly expanded after the two corners 26a and 26b of the cloth piece 22 are gripped by the pair of gripping hands 10a and 10b. FIG. 6B is a schematic view showing the relationship between the pair of gripping hands 10 a and 10 b and the cloth piece 22 after being linearly developed in FIG. 6A.

The camera system 30 for the cloth piece gripping device grips the suspended cloth piece 22 with the pair of gripping hands 10a and 10b, and the side 24 of the cloth piece 22 gripped by the pair of gripping hands 10a and 10b. The start and end of the operation of the pair of gripping hands 10a and 10b of the cloth piece gripping device 20 that linearly expands is determined. The cloth gripping device camera system 30 includes a sub camera 32, an operation start determination unit 34, and an operation end determination unit 37. The camera 32 photographs the cloth piece 22 (FIG. 3) that is hung before the operation of the pair of gripping hands 10a and 10b of the cloth piece gripping device 20 starts, and acquires an image. Further, the camera 32 captures an image by capturing a state (FIG. 6B) of gripping the cloth piece 22 with the pair of gripping hands 10a and 10b of the cloth piece gripping device 20 after the operation is started. Further, the operation start determination unit 34 confirms whether or not the vertically downward side 24 of the cloth piece 22 is within a predetermined range on the basis of the image of the cloth piece 22 that has been suspended, and the pair of cloth piece gripping devices 20 is checked. The start of the operation of the gripping hands 10a, 10b is determined. The "start of operation" means "start of the gripping operation of the cloth piece 22 by the pair of gripping hands 10a and 10b". Further, the operation end determination unit 37 recognizes whether or not the side 29 of the held cloth piece 22 is linear based on the image of the state in which the cloth piece 22 is being held, and thus the cloth piece 22 is grasped. By confirming whether or not 22 has been deployed, it is judged whether the operation of the pair of gripping hands 10a and 10b of the cloth piece gripping device 20 has ended. The "operation end" means "end of the unfolding operation of the cloth piece 22 by the pair of gripping hands 10a and 10b".
According to the cloth piece grasping device camera system 30, the start and end of the operation of the pair of grasping hands can be clearly determined, so that the cloth piece grasping device incorporating the same can be appropriately controlled.

This cloth piece grasping device camera system 30 can be used, for example, as a part of the cloth piece grasping device 20 shown in the schematic perspective view of FIG. 2 is a conveyor 23 for suspending the cloth piece 22, a camera system 30, a three-dimensional measuring device 25, a pair of gripping hands 10a and 10b, and control for controlling each device. The unit 40 is provided. In the present embodiment, the operation start determination unit 34 and the operation end determination unit 37 are mounted in the control unit 40 of the cloth gripping device 20, but not limited to this, and may be provided separately from the control unit, An image processing function may be mounted in the camera 32, and the camera 32 may be integrated with the camera 32. In the cloth piece gripping device 20, the camera system 30 detects the vertically downward side 24 of the cloth piece 22 suspended on the conveyor 23 to determine the start of the operation. When it is determined that the operation has started, the three-dimensional measuring device 25 acquires the three-dimensional shape of the cloth piece 22. The pair of gripping hands 10a and 10b are attached to the first and second robot arms 27a and 27b, respectively, and grip the two corner portions 26a and 26b of the cloth piece 22. Next, while pulling out the cloth piece 22 from the conveyor 23 while gripping the two corners 26a, 26b of the cloth piece 22 with the pair of gripping hands 10a, 10b, as shown in FIG. It is supported by the gripping hands 10a and 10b. In this case, the vertically lower side is turned upside down from FIG. 2 to become the vertically upper side 29. Next, as shown in FIGS. 6A and 6B, the pair of gripping hands 10a and 10b are relatively moved in a direction away from each other (arrows in FIG. 6A) to be gripped by the pair of gripping hands 10a and 10b. The side 29 of the piece 22 is developed into a straight line. If it can be confirmed that the side 29 of the cloth piece 22 has been developed in a straight line, it can be determined that the operation of the pair of gripping hands 10a and 10b of the cloth piece gripping device 20 has ended.

Each component that constitutes the camera system 30 for the cloth gripping device will be described.

<Camera>
The camera 32 photographs the cloth piece 22 suspended before the pair of gripping hands 10a and 10b of the cloth piece gripping device 20 start to operate, and acquires an image. Further, the camera 32 captures an image of a state in which the cloth piece 22 is gripped by the pair of gripping hands 10a and 10b of the cloth piece gripping device 20 after the operation is started. The camera 32 may be a two-dimensional camera. The camera 32 may be provided on the intermediate surface of the pair of gripping hands 10a and 10b. By providing it on the intermediate surface of the pair of gripping hands 10a and 10b, it is possible to acquire an image in which the pair of gripping hands 10a and 10b are arranged on both sides with respect to the sides 24 and 29 of the cloth piece 22. Specifically, the camera 32 may be provided between the first and second robot arms 27a and 27b to which the pair of gripping hands 10a and 10b are attached. Alternatively, the robot arms 27a and 27b may be configured by using a dual-arm robot, and a camera 32 may be provided on the upper part of the body of the dual-arm robot as shown in the schematic view of FIG. 9 (FIG. 9). In this case, since the bases of the two robot arms and the shooting direction of the camera are always linked, it is easy to shoot the image (FIGS. 6A and 6B) holding the cloth piece 22.

<Operation start determination unit>
The operation start determination unit 34 includes a lowest point detection unit 35 and an operation start possibility determination unit 36. The lowest point detection unit 35 detects the lowest point 21 of the cloth piece 22 from the image of the hanging cloth piece 22. Whether or not the pair of gripping hands 10a, 10b of the cloth piece gripping device 20 can be started based on whether or not the lowest point 21 of the cloth piece 22 is within a predetermined range by the operation start possibility determination unit 36. To judge. Specifically, for example, if the length L of the cloth piece 22 to the lowest point is within a predetermined range, it is determined that the operation can be started. On the other hand, when the length L is out of the predetermined range, it is determined that the operation cannot be started. The predetermined range of the length L may be set to an appropriate value depending on the size of the cloth piece 22, the range of motion of the robot arm, and the like.

<Flow chart for determining operation start>
FIG. 4 is a flowchart regarding determination of the operation start of the pair of gripping hands 10a and 10b of the cloth piece gripping device 20 by the cloth piece gripping device camera system 30 according to the first embodiment. FIG. 5 is a detailed flowchart of the operation start availability determination of FIG.

The determination of the start of operation of the pair of gripping hands 10a and 10b is performed as shown in the flowchart of FIG.
(1) The suspended cloth piece 22 is photographed to acquire an image (S01). For example, as shown in FIG. 2, the cloth piece 22 is photographed from below vertically to obtain an image including the side 24 of the cloth piece 22.
(2) Based on the image of the suspended cloth piece 22, it is confirmed whether or not the vertically downward side 24 of the cloth piece 22 is within a predetermined range, and the pair of gripping hands 10a of the cloth piece gripping device 20. It is determined whether the operation of 10b is started (S02).
Specifically, it is shown in the flowchart of FIG.
2-1) The lowest point 21 of the cloth piece 22 is detected from the image of the hanging cloth piece 22 (S11). The lowest point can be detected by normal edge detection. For example, in the image of the cloth piece 22, the lowest point 21 is determined by binarizing the cloth piece 22 and other portions, and examining the coordinates of the pixel at the lowest point among the pixels recognized as the cloth piece 22. Can be detected.
2-2) It is determined whether the lowest point 21 of the cloth piece 22 is within a predetermined range (S12). When the lowest point 21 of the cloth piece 22 is within the predetermined range (YES), it is determined that the pair of gripping hands 10a and 10b can start operation (S13). On the other hand, when the lowest point 21 of the cloth piece 22 is outside the predetermined range (NO), it is determined that the pair of gripping hands 10a and 10b cannot start the operation (S14).
Based on the above, whether or not the operation of the pair of gripping hands 10a and 10b can be started is determined.

<Operation end determination unit>
The operation end determination unit 37 includes a distance detection unit 38 and an operation end availability determination unit 39. From the image of the state in which the distance detecting unit 38 grips the cloth piece 22 with the gripping hands 10a and 10b of the cloth piece gripping device 20, the tips of the pair of gripping hands 10a and 10b gripping the cloth piece 22. The maximum distances d ₁ and d ₂ between the straight line 28 connecting the portions and the curve of the side 29 of the cloth piece 22 are detected. The operation end availability determination unit 39 confirms whether or not the maximum distances d ₁ and d _{2 of} the side 29 are within a predetermined range (equal to or less than a predetermined value), and the pair of gripping hands 10 a of the cloth piece gripping device 20. It is determined whether or not the operation of 10b is completed.
Specifically, the maximum distance d2 between the straight line 28 connecting the tip portions of the pair of gripping hands 10a and 10b that grip the cloth piece 22 and the curve of the side 29 of the cloth piece 22 is within a predetermined range (a predetermined value). In the following case, it is determined that the curve of the side 29 of the cloth piece 22 is substantially developed into a straight line. As a result, it is determined that the operation of the pair of gripping hands 10a and 10b can be completed. After that, the pair of gripping hands 10a and 10b can be used to place the linearly developed side 29 of the cloth piece 22 at a predetermined position to align the cloth pieces 22 in a certain direction.
On the other hand, the maximum distance d2 between the straight line 28 connecting the tips of the pair of gripping hands 10a and 10b that grip the cloth piece 22 and the curve of the side 29 of the cloth piece 22 is outside the predetermined range (exceeds a predetermined value). If it is, it is determined that the curve of the side 29 of the cloth piece 22 is insufficiently expanded, and the expanding operation of the cloth piece 22 by the gripping hand is continued.

<Flow chart for determining operation end>
FIG. 7 is a flowchart regarding determination of the operation end of the pair of gripping hands 10a and 10b of the cloth piece gripping device 20 by the cloth piece gripping device camera system 30 according to the first embodiment. FIG. 8 is a detailed flowchart of the operation end availability determination of FIG. 7. Actually, the “imaging→judgment” process shown in the flowchart of FIG. 7 is repeatedly executed in real time until the gripping hand finishes developing the cloth piece 22.

The judgment of the operation end of the pair of gripping hands 10a and 10b is performed as shown in the flowchart of FIG.
(1) An image is acquired by photographing the state in which the cloth piece 22 is held by the pair of gripping hands 10a and 10b of the cloth piece gripping device 20 (S21). As shown in FIG. 6A, the side 24 of the cloth piece 22 is linearly expanded from the state where the two corners 26a and 26b of the cloth piece 22 are gripped by the pair of gripping hands 10a and 10b. Then, as shown in FIG. 6B, the cloth piece 22 gripped by the pair of gripping hands 10a and 10b after being linearly developed is photographed.
(2) Based on the image of the state in which the cloth piece 22 is gripped by the pair of gripping hands 10a and 10b of the cloth piece gripping device 20 (FIG. 6B), the cloth piece is gripped by the pair of gripping hands 10a and 10b. Whether or not the side 29 of the cloth piece 22 is linear is checked to determine whether the pair of gripping hands 10a and 10b of the cloth piece gripping device 20 have finished operating (S22).
Specifically, it is shown in the flowchart of FIG.
2-1) From the image of the state in which the gripping hands 10a and 10b of the cloth piece gripping device 20 are gripping the cloth piece 22, the tip portions of the pair of gripping hands 10a and 10b that grip the cloth piece 22 are shown. a straight line 28 connecting to detect the maximum distance _{d 2} between the curve of the side 29 of the fabric piece 22 (S31). The straight line 28 may be set as predetermined coordinates as long as the relative positional relationship between the camera and the gripping hand is the same, or a marker may be placed on the hand and extracted from the coordinates. The side 29 may be extracted by binarizing the cloth piece 22 portion and the background portion from the image.
2-2) It is determined whether the maximum distance d ₂ of the side 29 is within a predetermined range (below a predetermined value) (S32). For example, it is determined whether the maximum distance d ₂ of the side 29 is 10 mm or less. When the maximum distance d ₂ of the side 29 is within the predetermined range (equal to or less than the predetermined value) (YES), it is determined that the operation of the pair of gripping hands 10a and 10b can be completed. On the other hand, when the maximum distance d ₂ of the side 29 is outside the predetermined range (exceeds the predetermined value) (NO), it is determined that the operation of the pair of gripping hands 10a and 10b cannot be completed. In this case, for example, if the pair of gripping hands 10a and 10b are in the middle of movement and the side 29 can be further linearly expanded, the side is expanded. Alternatively, when the side 29 cannot be further developed by the pair of gripping hands 10a and 10b, the gripping of the cloth piece 22 is released and the cloth piece 22 is discarded. For example, when the pair of gripping hands 10a and 10b is deployed, when a load larger than a predetermined value is detected by the gripping hands 10a and 10b, the deployment may be stopped and the grip of the cloth piece 22 may be released.
As described above, it is determined whether or not the operation of the pair of gripping hands 10a and 10b can be completed.

Note that among the above-mentioned components, the operation start determination unit 34, the lowest point detection unit 35, the operation start possibility determination unit 36, the operation end determination unit 37, the distance detection unit 38, the operation termination possibility determination unit 39, and the like are functional. The constituent members are not limited to being realized by a mechanical structure as a whole, and at least a part thereof may be realized by a computer program. In this case, the computer may include normal physical components such as a CPU, a memory, a storage unit, a display unit, an input unit, and an interface.

It should be noted that the present disclosure includes appropriate combination of any of the various embodiments and/or examples described above, and each of the embodiments and/or The effects of the embodiment can be achieved.

According to the cloth piece grasping device camera system of the present invention, the start and end of the operation of the pair of grasping hands can be clearly determined, and thus the cloth piece grasping device incorporating the same can be appropriately controlled.

10a, 10b gripping hand 20 cloth piece gripping device 22 towel (cloth piece)
23 Cloth hanging section (conveyor)
24 side 25 three-dimensional measuring device 26a, 26b corner part 27a first robot arm 27b second robot arm 28 straight line between gripping hands 29 curve of side of cloth piece 30 camera system for cloth piece gripping device 32 camera 34 operation start judgment Unit 35 lowest point detection unit 36 operation start availability determination unit 37 operation end determination unit 38 distance detection unit 39 operation end availability determination unit 40 control unit

Claims (4)

Of the pair of gripping hands of the cloth piece gripping device that grips the suspended cloth piece with the pair of gripping hands and linearly expands the sides of the cloth piece gripped by the pair of gripping hands A camera system for a cloth gripping device regarding the start and end of an operation,
Before the operation of the pair of grasping hands of the cloth piece grasping device, the suspended cloth piece is photographed and an image is acquired, and after the operation is started, the pair of grasping hands of the cloth piece grasping device grasps the cloth piece. A camera that captures images while capturing
Based on the image of the suspended piece of cloth, an operation start determination for confirming whether or not the piece of cloth is within a predetermined range and determining whether to start the operation of the pair of gripping hands of the cloth piece gripping device Department,
Based on the image of the state of gripping the cloth piece by the pair of gripping hands of the cloth piece gripping device, it is determined whether the cloth piece gripped by the pair of gripping hands has been deployed. An operation end determination unit that confirms and determines the operation end of the pair of gripping hands of the cloth piece gripping device,
A camera system for a cloth gripping device. The operation start determination unit,
A lowest point detection unit that detects the lowest point of the cloth piece from the image of the suspended cloth piece;
An operation start possibility determination unit that determines whether or not the pair of gripping hands of the cloth piece gripping device can start operating, based on whether the lowest point of the cloth piece is within a predetermined range,
The camera system for the cloth piece gripping device according to claim 1, further comprising: The operation end determination unit,
From the image of the state in which the gripping hand of the cloth piece gripping device is gripping the cloth piece, a straight line connecting the tip portions of the pair of gripping hands that grip the cloth piece, and the cloth piece. A distance detection unit that detects the maximum distance from the side curve of
An operation end availability determination unit that determines whether the maximum distance of the side is less than or equal to a predetermined value, and determines whether or not the operation ends of the pair of gripping hands of the cloth piece gripping device.
The camera system for the cloth piece gripping device according to claim 1 or 2, further comprising: The camera system for a cloth gripping device according to claim 1, wherein the camera is provided on an intermediate surface of the pair of gripping hands.

Images