JP2017130003A - Clothing type determination device and clothing type determination method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a clothing type determination device capable of more accurately determining the type of clothing.SOLUTION: A clothing type determination device 3 comprises: a featured value calculation part 33 for calculating a featured value from an image obtained by imaging clothing C by an imaging part 2; a contour pattern extraction part 34 for extracting a contour pattern in accordance with the featured value from a plurality of contour patterns preliminarily created by using a clothing virtual model; a type-categorized image data creation part 35 for creating type-categorized image data by using the contour pattern extracted by the contour pattern extraction part; a contour line data acquisition part 36 for acquiring contour line data of the clothing C from the image captured by the imaging part 2; a coincidence degree calculation part 37 for calculating a coincidence degree between the type-categorized image data created by the type-categorized image data creation part 35 and the contour line data acquired by the contour line data acquisition part 36; and a determination part 38 for determining the type corresponding to the type-categorized image data with the highest coincidence degree calculated by the coincidence degree calculation part 37 as the type of the clothing C.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an apparatus and a method for determining the type of clothing.

  So far, research has been conducted on how to make robots determine the type of clothing, but there is a first problem that the shape of clothing changes due to deformation, and there are various designs for the same type. Since there is a second problem that there is a characteristic, it is difficult to determine the type.

  In such a situation, the inventor according to the present application has published the following Non-Patent Document 1, and for the first problem, by devising the procedure when the robot lifts and holds the clothing, A solution has been shown that transitions to a limited number of shapes each time.

  Also, in the same document, the second problem is unified to cover the differences in the design for each type by expressing each type as a “likelihood image” with a certain width. A solution to generate a good image was shown. In addition, the following patent documents 1 and 2 are disclosed by the inventors which concern on this application about the technique which estimates the state where the clothes were hung.

JP 2004-5361 A JP 2010-61431 A

Jingyu Hu, Yasuyo Kita, “Classification of the category of clothing item after bringing it into limited shapes”, Proc. Of IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids 2015) 2015.11.3-5, pp.588-594

  However, in the technique disclosed in Non-Patent Document 1, depending on the type of clothing, the width of the uniqueness image becomes wider and includes other clothing, and as a result, the type of clothing is erroneously determined. There is a problem of end up.

  The present invention has been made to solve such a problem, and an object of the present invention is to provide a clothing type determination device and a clothing type determination method for determining the type of clothing more accurately.

  In order to solve the above-mentioned problem, the present invention was created in advance using a feature amount calculation means for calculating a feature amount from an image obtained by imaging clothing supported at least at two points, and a clothing virtual model. A contour pattern extracting means for extracting a contour pattern from a plurality of contour patterns in accordance with a feature quantity; a type-specific image data creating means for creating type-specific image data using the contour pattern extracted by the contour pattern extracting means; , Contour data acquisition means for acquiring the contour line data of the clothing from the image obtained by imaging, and the type-specific image data created by the type-specific image data creation means and the contour acquired by the contour line data acquisition means A degree of coincidence calculating means for calculating the degree of coincidence with the line data, and corresponding to the type-specific image data calculated by the degree of coincidence calculating means. Providing clothing type determination device and a determining means and the garment of the type of classification.

  In order to solve the above-mentioned problem, the present invention images the clothing supported in the first step, the first step of supporting the clothing to be observed at at least two points according to a predetermined procedure, A second step of calculating a feature amount from the obtained image, a third step of extracting a contour pattern according to the feature amount from a plurality of contour patterns created in advance using a clothing virtual model, A fourth step of creating type-specific image data using the contour pattern extracted in step 3, a fifth step of acquiring clothing contour data from the image obtained in the second step, The sixth step for calculating the degree of coincidence between the type-specific image data created in step 4 and the contour line data obtained in the fifth step, and the degree of coincidence calculated in the sixth step are the highest. Providing clothing type determination method and a seventh step of determining the type of a clothing type corresponding to the type-specific image data.

  ADVANTAGE OF THE INVENTION According to this invention, the clothing type determination apparatus and clothing type determination method which determine the classification of clothing more accurately can be provided.

It is a block diagram which shows the structure of the clothing type determination system containing the clothing type determination apparatus 3 which concerns on embodiment of this invention. It is a flowchart for demonstrating the outline | summary of operation | movement of the clothing classification determination system shown by FIG. It is a flowchart for demonstrating concretely the operation | movement of the clothing type determination system shown by FIG. It is a figure for demonstrating concretely the expression method of the specific shape in step S1 of FIG. It is a figure for demonstrating the calculation of the feature-value in step S3a of FIG. 3, and the extraction operation | movement of the two-dimensional outline in step S3b. It is a figure for demonstrating the process from creating the reference | standard outline pattern in step S12 of the figure through the simulation of the specific shape in step S11 of FIG. 3 to creating the likelihood image in step S4 of the figure. It is a figure which shows the outline | summary of the creation process of the uniqueness image L2 produced when the clothing made into an observation object is pants. It is a figure for demonstrating the calculation operation | movement of the matching degree R in step S5 of FIG.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the drawings, the same reference numerals indicate the same or corresponding parts.
FIG. 1 is a block diagram showing a configuration of a clothing type determination system including a clothing type determination device 3 according to an embodiment of the present invention. As shown in FIG. 1, the clothing type determination system includes a support unit 1 that supports the clothing C to be observed, an imaging unit 2 that captures the clothing C, and a clothing type that controls the support unit 1 and the imaging unit 2. And a determination device 3.

  Here, the support unit 1 includes a robot arm 101 that supports the clothing C at the point P1 and a robot arm 102 that supports the clothing C at the point P2. In FIG. 1, clothing C is trousers, and a state in which a specific shape described later is expressed by support by robot arms 101 and 102 is illustrated.

  The imaging unit 2 is configured by a camera for imaging the clothing C. The clothing type determination device 3 includes a bus 4, an imaging unit 2 and a storage unit 10 connected to the bus 4, a display unit 20 connected to the bus 4, a support unit 1 and an imaging unit connected to the bus 4. 2 and a control unit 30 that controls 2.

  Further, the control unit 30 includes a robot arm control unit 31, an imaging control unit 32, a feature amount calculation unit 33, a contour pattern extraction unit 34, a type-specific image data creation unit 35, and a contour line data acquisition unit 36. , A degree-of-matching calculation unit 37 and a determination unit 38 are included.

  Below, with reference to FIG. 2, the outline | summary of operation | movement of the clothing classification determination system which has the above structures is demonstrated.

  In step S1, the support unit 1 supports the clothing to be observed at at least two points according to a predetermined procedure.

  Next, in step S2, the clothing supported in step S1 is imaged by the imaging unit 2, and the control unit 30 calculates a feature amount from the obtained image.

  Next, in step S3, the control unit 30 extracts a contour pattern according to the feature amount from a plurality of contour patterns created in advance using the clothing virtual model, and in step S4, the contour extracted in step S3. Create image data by type using patterns.

  Next, in step S5, the control unit 30 acquires clothing outline data from the image obtained in step S2. In step S6, the type-specific image data created in step S4 and the outline obtained in step S5. The degree of coincidence with the line data is calculated.

  In step S7, the control unit 30 determines that the type corresponding to the type-specific image data having the highest degree of coincidence calculated in step S6 is the type of the clothing to be observed. The “type” means not only shirts, trousers, skirts, and towels, but also long-sleeved shirts and short-sleeved shirts, and long-sleeved shirts and shorts.

  Hereinafter, the operation of the clothing type determination system shown in FIG. 1 will be described in detail with reference to FIG.

  In step S1, the robot arm control unit 31 controls the two-armed robot arms 101 and 102 to lift the clothing C to be observed from the desk according to a predetermined procedure, so that a small number of limited specific shapes can be obtained. To express.

  Here, the expression method of the specific shape will be specifically described with reference to FIG. FIG. 4 shows a case where the clothing to be observed is a long sleeve shirt.

  As shown in FIG. 4, when the garment C10 is randomly folded and placed on a desk, the robot arm control unit 31 causes the robot arm to grip an arbitrary point Pa of the garment C10. Next, the robot arm control unit 31 becomes a convex point when the garment C20 is rotated about the lowest point Pb of the garment C20 with the robot arm gripping the point Pa and the vertical axis passing through the point Pa. The robot arm 101, 102 supports the point Pc closest to the gripped point Pa, respectively. Thus, the outline of the garment C30 in a state of being spread by a predetermined procedure corresponds to the specific shape.

  Next, in step S2, the imaging control unit 32 causes the imaging unit 2 to image the clothing C supported in step S1, and causes the clothing type determination device 3 to acquire three-dimensional observation data. At this time, the storage unit 10 stores the region image of the clothing C that is the observation target, so that the main region image is extracted.

  Next, in step S3a, the feature amount calculation unit 33 calculates feature amounts K1 and K2. Here, the feature amounts K1 and K2 are calculated by taking the ratio represented by the following equation, respectively.

  In the above equation, as shown in FIG. 5, LA has a specific shape by supporting the clothing to be observed (here, pants are shown as an example) at two points P 1 and P 2. In this case, the distance between the points P1 and P2 is shown. Further, as shown in FIG. 5, LB indicates the height of the image area extracted in step S2, and LC indicates the maximum width of the image area.

  In step S3b, as shown in FIG. 5, the contour data acquisition unit 36 extracts a two-dimensional contour CD from the extracted image region.

  On the other hand, in step S11, the contour pattern extraction unit 34 simulates the specific shape using clothing virtual models of various designs and materials for each clothing type, and is calculated as a result of the simulation in step S12. Create a reference contour pattern for each type of clothing by extracting a two-dimensional contour line on a three-dimensional distance image and normalizing the size using the distance between the grip points of both hands or the support points. . Note that, as described above, a plurality of specific shapes are generated depending on the type of clothing to be observed, and therefore, for such a type, a plurality of reference contour patterns are created for the same type.

  Here, the operation in steps S11 and S12 of the contour pattern extraction unit 34 will be specifically described with reference to FIG.

  As described above, the contour pattern extraction unit 34 creates a reference contour pattern for each type of clothing, but FIG. 6 illustrates a method of creating a reference contour pattern CK taking a long sleeve shirt as an example.

  The contour pattern extraction unit 34 creates a plurality of clothing virtual models Ca0, Cb0, Cc0 having various designs and softness (materials) in the type of long-sleeved shirt, and specifies each clothing virtual model as a three-dimensional distance image. The shape images Ca1, Cb1, and Cc1 are simulated.

  Then, the contour pattern extraction unit 34 extracts two-dimensional contour lines Ca2, Cb2, and Cc2 for the specific shape images Ca1, Cb1, and Cc1, respectively, and sets the sizes of the extracted two-dimensional contour lines Ca2, Cb2, and Cc2 to long sleeves. A reference contour pattern CK for the long-sleeved shirt is created by normalizing and superimposing the distance between the two hand grip points or the support points at the upper part of the shirt.

  Here, in step S4 in FIG. 3, the contour pattern extraction unit 34 selects a reference contour pattern according to the feature amounts K1 and K2 from the population having the reference contour pattern CK created in advance in step S12 as a constituent element. CK is selected, and a two-dimensional contour line of the selected reference contour pattern CK is extracted. And the outline pattern extraction part 34 produces the reference | standard outline pattern according to observation object clothing for every classification using only the two-dimensional outline extracted by such a method. Further, the type-specific image data creation unit 35 creates a unique image based on the reference contour pattern created in this way, and FIG. 6 illustrates a long-sleeved shirt unique image L1.

  Here, “selecting a reference contour pattern according to the feature values K1 and K2” means not only selecting a reference contour pattern that completely matches the feature values K1 and K2, but also supporting the same clothing. In consideration of the fact that the height LB and the maximum width LC may vary somewhat, the reference contour pattern in which the feature values K1 and K2 fall within the width (K1 ± α1) and the width (K2 ± α2), respectively, is also selected. Means. Here, α1 and α2 are constants and are predetermined for each type of clothing.

  Further, the type-specific image data creation unit 35 sets the portion surrounded by the reference contour pattern to the pixel value 1 and sets the other portion to the pixel value 0, and then applies a Gaussian filter to smooth the appearance image L1. Create

  Note that, as described above, FIG. 6 shows the process of creating the uniqueness image L1 for the type of long-sleeved shirt, but for other types, the uniqueness image for each type is created by the same method. FIG. 7 shows an outline of the process of creating the uniqueness image L2 for the type of long pants.

  Next, in step S5, the degree of coincidence calculation unit 37 calculates the degree of coincidence R between the two-dimensional contour CD extracted in step S3b and each likelihood image. Here, the degree of coincidence R is calculated by the following equation.

  In the above equation, N represents the total number of pixels of the contour CD, n represents each pixel point constituting the contour CD, and n takes a value from 1 to N. In addition, (in, jn) means the coordinates of the nth pixel point, and L (in, jn) means the pixel value (having a probability value between 0 and 1) at the nth pixel point.

  Here, as shown in FIG. 8A and FIG. 8B, the pixel value L (in, jn) is obtained when the contour line CD is superimposed on each of the likelihood images L1 and L2. Is 1 when completely overlapping with the likelihood images L1 and L2, 0 when not completely overlapping, that is, when the pixels are positioned in the peripheral portion of the likelihood images L1 and L2. Takes a value between 0 and 1 depending on the degree of overlap.

  From these, the calculated degree of coincidence R means the average value of the pixel values of each pixel constituting the contour line CD. Therefore, the larger the average value, the more the contour line CD uniqueness images L1 and L2 are as a whole. The degree of overlap will be large.

  In FIG. 8, the contour line CD for the long pants shown in FIG. 5 is shown in FIG. 8 (a) as the long pants appearance image L2 shown in FIG. 7, and in FIG. 8 (b) as shown in FIG. 8 shows a state in which the image of the long-sleeved shirt is overlapped with the appearance image L1, but since the degree of overlap is larger as a whole in FIG. R is larger in FIG. 8A than in FIG. 8B.

  In step S <b> 6, the determination unit 38 determines the type corresponding to the likelihood image where the calculated matching degree R is the maximum as the type of clothing to be observed. Accordingly, in the case of FIG. 8, the determination unit 38 determines that the type of clothing to be observed is long pants. The determination unit 38 can also display the determination result on the display unit 20.

  It should be noted that the operations in step S11 and step S12 shown in FIG. 3 need only be executed once in advance in the determination of the type of observation target, and it is sufficient to repeatedly use the obtained results in all observations. Absent.

  In addition, the above-described clothing type determination device 3 according to the embodiment of the present invention includes the control unit 30 illustrated in FIG. 1 by a central processing unit (CPU) connected to the bus 4. The same function can be realized by causing the CPU described above to execute the software described in the programming language with the procedure shown in FIGS.

  As described above, according to the clothing type determination system including the clothing type determination device 3 and the clothing type determination device 3 and the clothing type determination method according to the embodiment of the present invention, the uniqueness image of each type of clothing is observed. Since it is created using only two-dimensional contour lines adapted to the acquired data, the function to determine the type of clothing to be observed is enhanced as the first step for autonomous handling of clothing automatically by a dual-arm robot etc. It can be made accurate.

  And with such high precision, it is possible to realize an autonomous robot that is useful even in the handling of everyday items such as nursing care, home, and tertiary industry through automation of clothing handling.

DESCRIPTION OF SYMBOLS 1 Support part 2 Imaging part 3 Clothing classification determination apparatus 31 Robot arm control part 32 Imaging control part 33 Feature-value calculation part 34 Contour pattern extraction part 35 Type-specific image data creation part 36 Contour line data acquisition part 37 Matching degree calculation part 38 Determination Part 101, 102 Robot arm LA Distance between support points LB Height LC Maximum width.

Claims (9)

A feature amount calculating means for calculating a feature amount from an image obtained by imaging clothing supported by at least two points;
Contour pattern extraction means for extracting the contour pattern according to the feature amount from a plurality of contour patterns created in advance using a clothing virtual model;
Type-specific image data creation means for creating type-specific image data using the contour pattern extracted by the contour pattern extraction means;
Contour data acquisition means for acquiring contour data of the clothing from the image obtained by imaging,
A degree of coincidence calculating means for calculating a degree of coincidence between the image data classified by type created by the image data creating means classified by type and the contour line data obtained by the contour line data obtaining means;
A clothing type determination device comprising: a determination unit that determines a type corresponding to the type-specific image data having the highest degree of matching calculated by the matching level calculation unit as the type of clothing. The apparatus further comprises support means for supporting the garment, and the support means includes
A robot arm,
The clothing type determination apparatus according to claim 1, further comprising a robot arm control unit configured to support the clothing to be observed by the robot arm according to a predetermined procedure. The feature amount is
A first feature that is a function of the height of the area in which the clothing is the subject in the image;
The clothing type determination apparatus according to claim 1, further comprising: a second feature amount that is a function of a maximum width of an area in which the clothing is a subject in the image.   The clothing type determination apparatus according to claim 1, wherein the type-specific image data creation unit creates the type-specific image data by smoothing an image region surrounded by the contour pattern. A first step of supporting the clothing to be observed at least two points according to a predetermined procedure;
A second step of imaging the clothing supported in the first step and calculating a feature value from the obtained image;
A third step of extracting the contour pattern according to the feature amount from a plurality of contour patterns created in advance using a clothing virtual model;
A fourth step of creating type-specific image data using the contour pattern extracted in the third step;
A fifth step of acquiring contour data of the clothing from the image obtained in the second step;
A sixth step of calculating a degree of coincidence between the image data classified by type created in the fourth step and the contour line data obtained in the fifth step;
A clothing type determination method comprising: a seventh step of determining a type corresponding to the type-specific image data having the highest degree of matching calculated in the sixth step as the type of clothing. The predetermined procedure is:
A grasping step for grasping an arbitrary portion of the clothing;
A first support step for supporting, as a first support point, a point that is the lowest point in the clothing by the gripping step;
The clothing is rotated using a vertical axis passing through the first support point as a rotation axis, and a point closest to the first support point among points where the distance from the vertical axis is a maximum is defined as a second support point. The clothing type determination method according to claim 5, further comprising a second supporting step of supporting. The feature amount is
A first feature that is a function of the height of the area in which the clothing is the subject in the image;
The clothing type determination method according to claim 5, further comprising: a second feature amount that is a function of a maximum width of an area in which the clothing is a subject in the image.   The clothing type determination according to claim 5, wherein the contour pattern is created by superimposing the two-dimensional contour line of the clothing virtual model after aligning and normalizing the distances between the support points in the first step. Method. The clothing type determination method according to claim 5, wherein the type-specific image data is created by smoothing an image region surrounded by the contour pattern.

Images