WO2022137494A1 - Shoe appearance inspection system, shoe appearance inspection method, and shoe appearance inspection program - Google Patents
Shoe appearance inspection system, shoe appearance inspection method, and shoe appearance inspection program Download PDFInfo
- Publication number
- WO2022137494A1 WO2022137494A1 PCT/JP2020/048681 JP2020048681W WO2022137494A1 WO 2022137494 A1 WO2022137494 A1 WO 2022137494A1 JP 2020048681 W JP2020048681 W JP 2020048681W WO 2022137494 A1 WO2022137494 A1 WO 2022137494A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- shoe
- image
- inspected
- reference point
- product
- Prior art date
Links
- 238000007689 inspection Methods 0.000 title claims abstract description 70
- 238000000034 method Methods 0.000 title claims description 45
- 238000000605 extraction Methods 0.000 claims abstract description 90
- 239000000284 extract Substances 0.000 claims abstract description 43
- 238000010801 machine learning Methods 0.000 claims abstract description 25
- 230000006870 function Effects 0.000 claims description 9
- 238000012549 training Methods 0.000 abstract description 3
- 238000005516 engineering process Methods 0.000 abstract description 2
- 230000002708 enhancing effect Effects 0.000 abstract 1
- 238000012545 processing Methods 0.000 description 24
- 210000000452 mid-foot Anatomy 0.000 description 16
- 210000002683 foot Anatomy 0.000 description 12
- 239000000463 material Substances 0.000 description 10
- 210000003371 toe Anatomy 0.000 description 9
- 238000010586 diagram Methods 0.000 description 8
- 238000013500 data storage Methods 0.000 description 6
- 238000004519 manufacturing process Methods 0.000 description 6
- 238000012986 modification Methods 0.000 description 5
- 230000004048 modification Effects 0.000 description 5
- 238000004804 winding Methods 0.000 description 4
- 239000010985 leather Substances 0.000 description 3
- 238000011179 visual inspection Methods 0.000 description 3
- DQXBYHZEEUGOBF-UHFFFAOYSA-N but-3-enoic acid;ethene Chemical compound C=C.OC(=O)CC=C DQXBYHZEEUGOBF-UHFFFAOYSA-N 0.000 description 2
- 238000003708 edge detection Methods 0.000 description 2
- 239000005038 ethylene vinyl acetate Substances 0.000 description 2
- 239000002657 fibrous material Substances 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 238000000465 moulding Methods 0.000 description 2
- 229920001200 poly(ethylene-vinyl acetate) Polymers 0.000 description 2
- 229920005989 resin Polymers 0.000 description 2
- 239000011347 resin Substances 0.000 description 2
- 239000000853 adhesive Substances 0.000 description 1
- 230000001070 adhesive effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 238000010030 laminating Methods 0.000 description 1
- 239000002184 metal Substances 0.000 description 1
- 229910052751 metal Inorganic materials 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/95—Investigating the presence of flaws or contamination characterised by the material or shape of the object to be examined
Definitions
- the present invention relates to a technique for inspecting the appearance of shoes.
- Patent Document 1 Conventionally, for standard products such as metal products whose shape is fixed while remaining constant, inspection efficiency and inspection accuracy may be improved by performing visual inspection by sensors or image processing (see, for example, Patent Document 1). Even in the case of amorphous products whose shapes can change significantly, there is also known a technique of determining the correspondence and consistency between parts between products having completely different shapes by image processing (for example, non-patent documents). 1). Both the technique of Patent Document 1 and the technique of Non-Patent Document 1 are common in that they are characterized in that the identity of the product is determined, and the determination content is whether or not the image processing substantially matches 100%. There is.
- the manufacturing process of shoe products includes an inspection process in order to maintain the quality of the product.
- the shoe product is a product whose shape is fixed to some extent
- the upper material is a mesh fiber material or a leather material, and the shape is not completely fixed and is easily deformed. Depending on the situation, there may be slight differences in shape.
- the manufacturing process such as the attachment of the upper to the sole and the application of the adhesive is performed manually by a human being, the attachment position and the application position may vary slightly. Due to these properties of shoes, the appearance inspection of shoe products has traditionally been performed by human visual inspection. However, visual inspection cannot deny the possibility of work variation and inspection omission, and the load of inspection is heavy. Therefore, it is desired to establish a technique capable of improving inspection efficiency and inspection accuracy.
- the present invention has been made in view of these problems, and an object thereof is to provide a shoe appearance inspection technique capable of improving inspection efficiency and inspection accuracy.
- the shoe appearance inspection system is an image acquisition unit that acquires an image of the shoe to be inspected, and an appearance feature point in the image of the shoe to be inspected.
- a reference point extraction unit that extracts a reference point by a predetermined reference point extraction method, and an element that extracts a plurality of element points that are appearance feature points in an image of shoes to be inspected by a predetermined element point extraction method.
- the point extraction unit, the virtual line extraction unit that extracts multiple virtual lines connecting the reference point and multiple element points from the image of the shoe to be inspected, and the virtual line extracted from the images of multiple shoes that are acceptable products.
- the model storage unit that stores the learning model generated by machine learning using lines as teacher data, and the shoes to be inspected by inputting multiple virtual lines extracted from the image of the shoes to be inspected into the training model. It is provided with a pass / fail judgment unit for determining whether or not the product has passed.
- the acquired image of the shoe is an image of the shoe in a state of being suspended by the last, and the reference point extraction unit extracts the appearance feature points of the last exposed from the shoe as the reference point in the image. You may.
- the acquired shoe image is composed of a plurality of images taken from a plurality of angles, and the model storage unit uses machine learning using virtual lines extracted from the plurality of images for one shoe as teacher data.
- the generated learning model is stored, and the pass / fail judgment unit determines whether or not the shoe to be inspected is a pass product by inputting virtual lines extracted from a plurality of images for the shoe to be inspected into the learning model. You may.
- a contour extraction unit that extracts the contour of the shoe from the image of the shoe to be inspected may be further provided.
- the model storage unit stores a learning model generated by machine learning using virtual lines and contours extracted from images of a plurality of acceptable shoes as teacher data, and the pass / fail judgment unit stores the learning model of the shoe to be inspected.
- the pass / fail judgment unit stores the learning model of the shoe to be inspected.
- Another aspect of the present invention is a shoe appearance inspection method.
- any combination of the above components, or the components and expressions of the present invention are mutually replaced between a method, a device, a program, a temporary or non-temporary storage medium in which the program is stored, a system, and the like. Is also effective as an aspect of the present invention.
- FIG. 1 is a configuration diagram of a shoe appearance inspection system 100 according to the present embodiment.
- the shoe appearance inspection system 100 includes a shoe appearance inspection device 110 and a shoe appearance inspection learning device 112.
- the shoe appearance inspection device 110 and the shoe appearance inspection learning device 112 are derived from a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a RAM (RandomAccessMemory), a ROM (ReadOnlyMemory), an auxiliary storage device, a communication device, and the like. It may consist of a computer.
- the shoe appearance inspection device 110 and the shoe appearance inspection learning device 112 may be configured by separate computers, or may be realized by one computer having both functions. In this embodiment, an example realized by a separate computer will be described.
- the shoe appearance inspection device 110 is communicated and connected with a plurality of photographing devices that capture images of a plurality of shoe products 10. Since the shoe product 10 is easily deformed when held by the operator and it is difficult to perform an accurate inspection, for example, the shoe product 10 is photographed while being placed on a table as shown in the figure. Not limited to.
- the plurality of photographing devices include a left side photographing device 50 for photographing the shoe product 10 from the left side, a right side photographing device 52 for photographing from the right side, an upper photographing device 54 for photographing from directly above, and a front photographing device 56 for photographing from the front. It is composed of a rear photographing device 58 for photographing from the rear and a lower photographing device 59 for photographing the bottom surface from below.
- the left side photographing device 50 photographs the outer instep side of the shoe product 10 for the left foot and photographs the inner instep side of the shoe product 10 for the right foot.
- the right side photographing device 52 photographs the inner instep side of the shoe product 10 for the left foot and photographs the outer instep side of the shoe product 10 for the right foot.
- the images taken by the left side photographing device 50, the right side photographing device 52, the upper photographing device 54, the front photographing device 56, the rear photographing device 58, and the lower photographing device 59 are transmitted to the shoe appearance inspection device 110.
- the shoe appearance inspection device 110 inspects the appearance of the shoe product 10 based on the received image.
- the shoe appearance inspection learning device 112 communicates with and connects with the shoe appearance inspection device 110, and machine-learns an image of the shoe product 10 to generate a learning model.
- the learning model is used for inspection by the shoe appearance inspection device 110.
- FIG. 2 is a diagram comparing the upper shape of the accepted product and the upper shape of the rejected product with images taken from the side of the outer instep side of the shoe product.
- FIG. 2A is an example of a accepted product
- FIG. 2B is an example of a rejected product.
- the sole is formed by laminating the outsole 12, the lower midsole 14, and the upper midsole 16 in this order from the bottom to the top.
- the shoe product 10 is configured in a so-called suspended state in which the upper 20 is attached to the vicinity of the instep of the last (foot type) 30 placed on the upper midsole 16 and adhered to the sole.
- the lower midsole 14, upper midsole 16, and outsole 12 made of resin such as EVA (Ethylene-Vinyl Acetate) are hard to deform after manufacturing and the shape is almost fixed, except for manufacturing variations.
- the upper 20 is made of a material such as a mesh fiber material or leather whose shape is not necessarily fixed even after manufacturing.
- the last 30 is pulled out from the shoe product 10
- the height of the toes is slightly lowered due to the repulsive force of the sole, so that the upper 20 is easily deformed and the shape of each individual is likely to vary. In this state, it is easy to keep the shape of the upper 20 constant. Therefore, the state with the last 30 is more suitable for the inspection, but the inspection accuracy is improved by machine learning even without the last 30.
- the upper is attached to the sole in the suspended state, but since the attaching process is done manually by the operator, there are variations in the work, and the variations may cause variations in the shape.
- the upper shape 22a has a curved shape that slightly warps from the instep to the toe
- the upper shape 22b has an instep to the toe. It has a shape that is almost straight.
- the error range of the ratio of the length to the accepted product is modeled by image processing and machine learning, and if the error is judged to be within the allowable range by the learning model, it is estimated as the accepted product, and if it exceeds the allowable range, it is not acceptable. Estimated to be a passing product.
- FIG. 3 is a diagram comparing the tilt of the axis in the accepted product and the tilt of the axis in the rejected product in the rear image.
- FIG. 3A is an example of a accepted product
- FIG. 3B is an example of a rejected product.
- the horizontal axis 18a is almost horizontal and the vertical axis 24a is almost vertical
- the horizontal axis 18b is horizontal. It is tilted slightly downward to the left, and the vertical axis 24b is also tilted slightly downward from the vertical.
- the range of the inclination ratio with the accepted product is modeled by image processing and machine learning, and if the error is judged to be within the allowable range by the learning model, it is estimated as a accepted product, and if it exceeds the allowable range, it is rejected. I presume.
- running shoes are exemplified and described as shoe products 10, and various sports shoes including running shoes, leather shoes, and various other shoe products such as those manufactured by attaching an upper to a sole are used.
- FIG. 4 shows the misalignment of the midfoot high hardness material in the midsole.
- a high-hardness material may be partially used at the position corresponding to the midfoot of the foot to ensure rigidity, but at the position where the high-hardness material is used in the sole molding process. Errors can occur. However, since the portion where the high hardness material is used does not appear in the appearance of the lower midsole 14, it is difficult for the operator to visually detect the error.
- the figure shows the outline of the sole.
- the midfoot portion M1 shown by a diagonal line pattern from the upper left to the lower right indicates the high hardness material portion in the accepted product.
- the midfoot portion M2 shown by a diagonal line pattern from the upper right to the lower left indicates a high hardness material portion in the rejected product.
- the difference in position is clear when comparing the two as shown in the figure, but when viewed from the inside of the shoe (left side of the figure), there is no difference in the position between the midfoot part M1 and the midfoot part M2, and the outside of the shoe (figure). There is a difference in the positions of the midfoot portion M1 and the midfoot portion M2 only when viewed from the right side of). However, when a worker visually inspects, it is difficult to find a positional deviation such as a difference in the positions of the midfoot portion M1 and the midfoot portion M2 just by looking at a single shoe.
- the virtual lines La and La'from the reference point Ra at the apex on the toe side to the start points of the midfoot portions M1 and M2 have the same length.
- the virtual line Lb from the reference point Ra to the start point of the midfoot portion M1 and the virtual line Lb'from the reference point Ra to the start point of the midfoot portion M2 have different lengths.
- the width Lc of the midfoot portion M1 and the width Lc'of the midfoot portion M2 are substantially the same length in the example of this figure.
- the magnitude of the error can be expressed by the ratio (Ln'/ Ln) of the lengths of the virtual lines of the accepted product and the rejected product.
- the ratio of the length of the virtual line extracted from each image and the length of the virtual line extracted from the image of the accepted product is calculated and learned.
- FIG. 5 is a functional block diagram showing the basic configuration of the shoe appearance inspection device 110.
- the shoe appearance inspection device 110 includes an image acquisition unit 120, an image storage unit 122, a reference point extraction unit 124, an element point extraction unit 126, a virtual line extraction unit 128, a contour extraction unit 130, an extraction data storage unit 132, and a pass / fail determination unit 134.
- the model storage unit 136 is included.
- the image acquisition unit 120 acquires an image of the shoe product 10 to be inspected from each of the left side photographing device 50, the right side photographing device 52, the upper photographing device 54, the front photographing device 56, and the rear photographing device 58, and stores the image.
- Store in unit 122 The image storage unit 122 is classified and stored together with attribute information such as the product model name and size of the shoe product 10 to be inspected, and whether it is for the left foot or the right foot.
- the reference point extraction unit 124 extracts a reference point, which is an appearance feature point in the image of the shoe product 10 to be inspected, by a predetermined reference point extraction method.
- the element point extraction unit 126 extracts a plurality of element points, which are appearance feature points in the image of the shoe product 10 to be inspected, by a predetermined element point extraction method.
- the virtual line extraction unit 128 extracts a plurality of virtual lines connecting the reference point and the plurality of element points from the image of the shoe product 10 to be inspected.
- the contour extraction unit 130 extracts the contour of the shoe product 10 from the image of the shoe product 10 to be inspected.
- FIG. 6 schematically shows a method of extracting reference points, element points, and virtual lines in an image taken from the side of the outer instep side of a shoe product with a last.
- a reference point and an element point are extracted as feature points from the image of the shoe product 10, and a virtual line connecting the reference point and the element point is extracted.
- Multiple such virtual lines are extracted, and the data of these multiple virtual lines are input to a predetermined machine learning model, and the position, inclination, inclination ratio of each virtual line to the accepted product, the length, and the accepted product are used.
- the length ratio is within the allowable error range, it is estimated whether or not the shoe product 10 is a acceptable product.
- the shoe It is estimated whether or not the product 10 is a acceptable product. Even in the contour inspection, it is possible to determine when an error exceeding the permissible range occurs in the positional relationship and balance of the contour between a plurality of images taken from a plurality of directions.
- Reference points and element points are feature points on the appearance with a stable shape that can be extracted based on a predetermined extraction method by image processing.
- the reference point extraction unit 124 extracts the end point on the heel side of the upper edge, which is a part of the last 30 exposed from the opening of the shoe product 10 and whose edge is detected by image processing for the last 30, as the first reference point R1. do.
- the upper edge of the last 30 has little shape change in the shoemaking process, and the last 30 used even when inspecting other product models has a common shape, so that it is suitable as a reference point for easy extraction.
- the number of feature points to be extracted is determined. It is possible to avoid an increase in processing load without increasing it indiscriminately. In this respect, it is more advantageous in terms of processing load than the method of extracting a plurality of virtual lines by connecting a plurality of element points from a plurality of different feature points in a many-to-many manner.
- a feature point that can be extracted by using a pattern or a character attached to a part of the last 30 as a mark may be extracted as the first reference point R1.
- the element point extraction unit 126 extracts the tip of the toe side of the outsole 12 whose edge is detected by image processing on the outsole 12 as the first element point P1.
- the first element point P1 is the apex that overhangs the most forward in the arcuate contour on the toe side of the outsole 12.
- the virtual line extraction unit 128 extracts the first virtual line L1 connecting the first reference point R1 and the first element point P1.
- the element point extraction unit 126 is a point at which the curvature of the winding portion on the toe side whose edge is detected by image processing with respect to the outsole 12 changes, that is, a starting point for winding from the ground plane of the outsole 12 toward the toe (“toe spring start point”). ”) As the second element point P2.
- the virtual line extraction unit 128 extracts the second virtual line L2 connecting the first reference point R1 and the second element point P2.
- the element point extraction unit 126 is a point where the curvature of the heel-side winding portion detected by image processing on the outsole 12 changes, that is, a starting point of winding from the ground plane of the outsole 12 toward the heel (“heel cut start point”). ”) As the third element point P3.
- the virtual line extraction unit 128 extracts the third virtual line L3 connecting the first reference point R1 and the third element point P3.
- the element point extraction unit 126 extracts the rearmost end portion on the heel side whose edge is detected by image processing on the sole as the fourth element point P4.
- the fourth element point P4 is the apex that protrudes most rearward in the arcuate contour on the heel side of the lower midsole 14.
- the virtual line extraction unit 128 extracts the fourth virtual line L4 connecting the first reference point R1 and the fourth element point P4.
- the element point extraction unit 126 extracts the uppermost end portion whose edge is detected by image processing for the shoe opening as the fifth element point P5.
- the shoe product 10 has a wavy shape on both the outside and the inside, and the element point extraction unit 126 extracts the apex or the highest point in the arc of the wave shape as the fifth element point P5.
- the virtual line extraction unit 128 extracts the fifth virtual line L5 connecting the first reference point R1 and the fifth element point P5. If the shoe product 10 to be inspected does not have a wavy shape such as boots, for example, the front end or the rearmost end of the shoe opening may be extracted as the fifth element point P5.
- FIG. 7 schematically shows a method of extracting reference points, element points, and virtual lines in an image taken from the side of the outer instep side of a shoe product without a last.
- the shoe product 10 in this figure is an example in which the state after removing the last 30 is inspected. Unlike the case with the last 30, the reference point is extracted from a part of the shoe product 10.
- the reference point extraction unit 124 extracts the lowermost end portion whose edge is detected by image processing for the wearing opening as the second reference point R2.
- the opening of the shoe product 10 has a wave shape both on the outside and inside, and the reference point extraction unit 124 extracts the lowest point or the lowest point in the arc of the wave shape as the second reference point R2.
- the rearmost end or the frontmost end of the shoe opening may be used as a reference point for extraction.
- a specification may be made in which any one of a plurality of element points is set as a reference point.
- the virtual line extraction unit 128 extracts the first virtual line L1 connecting the second reference point R2 and the first element point P1.
- the virtual line extraction unit 128 extracts the second virtual line L2 connecting the second reference point R2 and the second element point P2.
- the virtual line extraction unit 128 extracts the third virtual line L3 connecting the second reference point R2 and the third element point P3.
- the virtual line extraction unit 128 extracts the fourth virtual line L4 connecting the second reference point R2 and the fourth element point P4.
- the virtual line extraction unit 128 extracts the fifth virtual line L5 connecting the second reference point R2 and the fifth element point P5.
- FIG. 8 schematically shows a method of extracting reference points, element points, and virtual lines in a rear image of a shoe product with a last.
- the reference point extraction unit 124 extracts the apex or the highest point at the upper edge detected by the image processing for the last 30 as the third reference point R3.
- the third reference point R3 in the rear image is set to a point different from the first reference point R1 in the side image. For example, even in the side image, the upper edge is detected by the image processing for the last 30.
- common feature points may be used as reference points.
- the element point extraction unit 126 extracts the leftmost end detected by the image processing for the lower midsole 14 as the sixth element point P6.
- the sixth element point P6 is the leftmost apex of the arcuate contour on the left side of the lower midsole 14.
- the virtual line extraction unit 128 extracts the sixth virtual line L6 connecting the third reference point R3 and the sixth element point P6.
- the element point extraction unit 126 extracts the lowermost end where the edge is detected by the image processing for the outsole 12 as the seventh element point P7.
- the seventh element point P7 is the lowest point or the lowest point in the outsole 12 arcuate contour.
- the virtual line extraction unit 128 extracts the seventh virtual line L7 connecting the third reference point R3 and the seventh element point P7.
- the element point extraction unit 126 extracts the rightmost edge detected by image processing on the lower midsole 14 as the eighth element point P8.
- the eighth element point P8 is the apex that overhangs to the right in the arcuate contour on the right side of the lower midsole 14.
- the virtual line extraction unit 128 extracts the eighth virtual line L8 connecting the third reference point R3 and the eighth element point P8.
- reference points and element points are also extracted from the right side, front, and upper images of the shoe product 10, and virtual lines are extracted.
- FIG. 9 schematically shows a method of extracting contours in an image taken from the side of the outer instep side of a shoe product with a last.
- the contour extraction unit 130 extracts the contour SL of the entire shoe product 10 by edge detection in the image processing for the shoe product 10. Since only one contour SL can be obtained from one image of the shoe product 10, the number of items that can be inspected is less than that of inspection using virtual lines, and the number of objects to be machine-learned is smaller than that of virtual lines. There is an aspect that it is difficult to improve the detection accuracy by learning compared to the virtual line that can learn teacher data.
- the edge detection of the contour SL does not require the feature of the shape unlike the extraction of the reference point and the element point, so that it can be extracted more easily.
- the contour inspection can be used not only in the inspection of the finished product but also in the inspection in each of the plurality of processes included in the manufacturing process.
- the contour is extracted from the right side, front, rear, upper, and lower images of the shoe product 10 and the image of the shoe product 10 without the last.
- the model storage unit 136 stores a learning model that has been pre-generated and learned by machine learning using a plurality of virtual lines and contours extracted from images of a plurality of passing shoe products as teacher data.
- the model storage unit 136 stores a learning model generated by machine learning using a plurality of virtual lines and contours extracted from a plurality of images for each shoe product 10 as teacher data.
- This training model has a tolerance for data errors such as virtual line position, slope, slope ratio with pass product, length, length ratio with pass product, contour position, and balance in many pass products.
- the learning model is generated in advance by the shoe appearance inspection learning device 112 and stored in the model storage unit 136 as described later.
- the pass / fail determination unit 134 inputs a plurality of virtual lines and contours extracted from the image of the shoe product 10 to be inspected into the learning model, and the position, inclination, ratio of inclination to the accepted product, and length of the virtual line, By comparing the length ratio, the position of the contour, and the balance with the accepted product, it is possible to estimate whether the error is within the allowable range, that is, whether the shoe product 10 to be inspected is the accepted product.
- the pass / fail determination unit 134 inputs a plurality of virtual lines and contours extracted from a plurality of images for the shoe product 10 to be inspected into the learning model, and determines the position, inclination, and inclination ratio of the virtual lines to the accepted products.
- the pass / fail determination unit 134 outputs the estimation result by a method such as screen display, and feeds it back to the learning model stored in the model storage unit 136 as data of a pass product or a fail product.
- FIG. 10 is a functional block diagram showing the basic configuration of the shoe appearance inspection learning device 112.
- the image acquisition unit 220, the image storage unit 222, the reference point extraction unit 224, the element point extraction unit 226, the virtual line extraction unit 228, the contour extraction unit 230, and the extraction data storage unit 232 are the image acquisition unit 120 and the image storage unit 122, respectively.
- Reference point extraction unit 124, element point extraction unit 126, virtual line extraction unit 128, contour extraction unit 130, and extraction data storage unit 132 each of which has the same function.
- the machine learning unit 234 uses the data of a plurality of virtual lines and the contour data stored in the extracted data storage unit 232 as teacher data, and uses a learning model for determining whether or not the error between the virtual lines and the contour falls within an allowable range. It is generated by machine learning and stored in the model storage unit 236. The learning model is transmitted to the shoe appearance inspection device 110 and used for the appearance inspection of the shoe product 10.
- the teacher data includes information on a plurality of virtual lines and contours extracted from shoes as shown in FIGS. 6 to 9.
- the virtual line data is the position and inclination of the virtual line obtained from a plurality of images of a large number of accepted products, the ratio of the inclination to the accepted product, the length, and the ratio of the length to the accepted product.
- the contour data is the position and balance of the contour obtained from a plurality of images of a large number of accepted products. Since the virtual lines and contours have variations in position, inclination, length, balance, etc., machine learning is performed to model the allowable range as an error.
- the shoe product 10 has a plurality of types of product models, and even in one product model, there are a plurality of sizes, and the shoe product 10 is divided into one for the left foot and one for the right foot.
- the machine learning unit 234 machine-learns a plurality of virtual lines and contours for each attribute such as product model, size, left foot and right foot. By performing machine learning separately for each attribute, the determination accuracy can be further improved.
- a learning model by machine learning only the images of the passing shoe products 10 has been described.
- the virtual lines and contours extracted from the image of the accepted product are learned and labeled as pass, and the virtual lines and contours extracted from the image of the rejected product are further learned and labeled as rejected. It may be that. Although more teacher data is required than when only passing products are trained, the accuracy of judgment for classifying pass and fail can be improved accordingly.
- a learning model may be generated only by machine learning of virtual lines without using machine learning of contours.
- FIG. 11 is a flowchart showing a procedure for extracting a plurality of virtual lines and contours from an image of a shoe product and estimating whether or not the product is acceptable based on a learning model.
- the image acquisition unit 120 captures images of the shoe product 10 from a plurality of imaging directions by a plurality of imaging devices such as the left side photographing device 50 and the right side photographing device 52 (S10), and the image acquisition unit 120 captures a plurality of images.
- Acquire (S11).
- the reference point extraction unit 124 and the element point extraction unit 126 extract the reference point and a plurality of element points from the image (S12), and the virtual line extraction unit 128 extracts a plurality of virtual lines from the image based on the reference point and the plurality of element points. Is extracted and stored in the extracted data storage unit 132 (S14).
- the pass / fail determination unit 134 inputs virtual line data into the learning model stored in the model storage unit 136, and determines whether the error of the virtual line is within the allowable range (S16).
- the contour extraction unit 130 extracts a contour from the image of the shoe product 10 and stores it in the extraction data storage unit 132 (S18).
- the pass / fail determination unit 134 inputs contour data into the learning model stored in the model storage unit 136, and determines whether the contour error is within the permissible range (S19).
- the pass / fail determination unit 134 estimates whether the shoe product 10 is a pass product or not by comprehensively determining whether the error of the virtual line is within the allowable range and the error of the contour is within the allowable range. (S20).
- the present invention can provide a shoe appearance inspection technique that can improve inspection efficiency and inspection accuracy.
Landscapes
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Biochemistry (AREA)
- General Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Immunology (AREA)
- Pathology (AREA)
- Footwear And Its Accessory, Manufacturing Method And Apparatuses (AREA)
Abstract
Description
Claims (6)
- 検査対象である靴の画像を取得する画像取得部と、
前記検査対象である靴の画像における外観上の特徴点である基準点を、所定の基準点抽出方法により抽出する基準点抽出部と、
前記検査対象である靴の画像における外観上の特徴点である複数の要素点を、所定の要素点抽出方法により抽出する要素点抽出部と、
前記基準点と前記複数の要素点をそれぞれ結ぶ複数の仮想線を前記検査対象である靴の画像から抽出する仮想線抽出部と、
合格品である複数の靴の画像から抽出された前記仮想線を教師データとする機械学習により生成された学習モデルを記憶するモデル記憶部と、
前記検査対象である靴の画像から抽出された複数の仮想線を前記学習モデルに入力することにより前記検査対象の靴が合格品か否かを判定する合否判定部と、
を備えることを特徴とする靴外観検査システム。 The image acquisition unit that acquires the image of the shoes to be inspected,
A reference point extraction unit that extracts a reference point, which is an appearance feature point in the image of the shoe to be inspected, by a predetermined reference point extraction method, and a reference point extraction unit.
An element point extraction unit that extracts a plurality of element points, which are appearance feature points in the image of the shoe to be inspected, by a predetermined element point extraction method.
A virtual line extraction unit that extracts a plurality of virtual lines connecting the reference point and the plurality of element points from the image of the shoe to be inspected, and a virtual line extraction unit.
A model storage unit that stores a learning model generated by machine learning using the virtual lines extracted from images of a plurality of acceptable shoes as teacher data, and a model storage unit.
A pass / fail determination unit that determines whether or not the shoe to be inspected is a pass product by inputting a plurality of virtual lines extracted from the image of the shoe to be inspected into the learning model.
A shoe appearance inspection system characterized by being equipped with. - 前記取得される靴の画像は、足型による吊り込みがされた状態の靴の画像であり、
前記基準点抽出部は、前記画像において靴から露出する足型の外観上の特徴点を前記基準点として抽出することを特徴とする請求項1に記載の靴外観検査システム。 The acquired image of the shoe is an image of the shoe in a state of being suspended by the last.
The shoe appearance inspection system according to claim 1, wherein the reference point extraction unit extracts the appearance feature points of the foot shape exposed from the shoes in the image as the reference points. - 前記取得される靴の画像は、複数通りの角度から撮影された複数の画像で構成され、
前記モデル記憶部は、一つの靴につき複数の画像からそれぞれ抽出された仮想線を教師データとする機械学習により生成された学習モデルを記憶し、
前記合否判定部は、前記検査対象である靴につき複数の画像からそれぞれ抽出された仮想線を前記学習モデルに入力することにより前記検査対象の靴が合格品か否かを判定することを特徴とする請求項1または2に記載の靴外観検査システム。 The acquired shoe image is composed of a plurality of images taken from a plurality of angles.
The model storage unit stores a learning model generated by machine learning using virtual lines extracted from a plurality of images for one shoe as teacher data.
The pass / fail determination unit is characterized in that it determines whether or not the shoe to be inspected is a pass product by inputting virtual lines extracted from a plurality of images of the shoe to be inspected into the learning model. The shoe appearance inspection system according to claim 1 or 2. - 前記検査対象である靴の画像から前記靴の輪郭を抽出する輪郭抽出部をさらに備え、
前記モデル記憶部は、合格品である複数の靴の画像から抽出された前記仮想線および前記輪郭を教師データとする機械学習により生成された学習モデルを記憶し、
前記合否判定部は、前記検査対象である靴の画像から抽出された複数の仮想線および輪郭を前記学習モデルに入力することにより前記検査対象の靴が合格品か否かを判定することを特徴とする請求項1から3のいずれかに記載の靴外観検査システム。 Further, a contour extraction unit for extracting the contour of the shoe from the image of the shoe to be inspected is provided.
The model storage unit stores a learning model generated by machine learning using the virtual line and the contour as teacher data extracted from images of a plurality of shoes that are acceptable products.
The pass / fail determination unit is characterized in that it determines whether or not the shoe to be inspected is a pass product by inputting a plurality of virtual lines and contours extracted from the image of the shoe to be inspected into the learning model. The shoe appearance inspection system according to any one of claims 1 to 3. - 検査対象である靴の画像を所定の画像取得手段により取得する過程と、
前記検査対象である靴の画像における外観上の特徴点である基準点を、コンピュータによる所定の基準点抽出方法により抽出する過程と、
前記検査対象である靴の画像における外観上の特徴点である複数の要素点を、コンピュータによる所定の要素点抽出方法により抽出する過程と、
前記基準点と前記複数の要素点をそれぞれ結ぶ複数の仮想線を前記検査対象である靴の画像からコンピュータにより抽出する過程と、
合格品である複数の靴の画像から抽出された前記仮想線を教師データとする機械学習により生成された学習モデルを所定の記憶手段から読み出す過程と、
前記検査対象である靴の画像から抽出された複数の仮想線を前記学習モデルに入力することにより前記検査対象の靴が合格品か否かをコンピュータにより判定する過程と、
を備えることを特徴とする靴外観検査方法。 The process of acquiring an image of shoes to be inspected by a predetermined image acquisition means, and
A process of extracting a reference point, which is an appearance feature point in the image of the shoe to be inspected, by a predetermined reference point extraction method using a computer, and a process of extracting the reference point.
A process of extracting a plurality of element points, which are appearance feature points in the image of the shoe to be inspected, by a predetermined element point extraction method by a computer, and
A process of extracting a plurality of virtual lines connecting the reference point and the plurality of element points from the image of the shoe to be inspected by a computer, and a process of extracting the plurality of virtual lines from the image of the shoe to be inspected.
A process of reading a learning model generated by machine learning using the virtual lines extracted from images of a plurality of acceptable shoes as teacher data from a predetermined storage means, and
A process of determining whether or not the shoe to be inspected is a pass product by inputting a plurality of virtual lines extracted from the image of the shoe to be inspected into the learning model, and a process of determining whether or not the shoe to be inspected is a pass product.
A shoe appearance inspection method characterized by comprising. - 検査対象である靴の画像を取得する機能と、
前記検査対象である靴の画像における外観上の特徴点である基準点を、所定の基準点抽出方法により抽出する機能と、
前記検査対象である靴の画像における外観上の特徴点である複数の要素点を、所定の要素点抽出方法により抽出する機能と、
前記基準点と前記複数の要素点をそれぞれ結ぶ複数の仮想線を前記検査対象である靴の画像から抽出する機能と、
合格品である複数の靴の画像から抽出された前記仮想線を教師データとする機械学習により生成された学習モデルを記憶する機能と、
前記検査対象である靴の画像から抽出された複数の仮想線を前記学習モデルに入力することにより前記検査対象の靴が合格品か否かを判定する機能と、
をコンピュータに実現させることを特徴とする靴外観検査プログラム。 The function to acquire the image of the shoe to be inspected and
A function of extracting a reference point, which is an appearance feature point in the image of the shoe to be inspected, by a predetermined reference point extraction method, and
A function of extracting a plurality of element points, which are appearance feature points in the image of the shoe to be inspected, by a predetermined element point extraction method, and
A function of extracting a plurality of virtual lines connecting the reference point and the plurality of element points from the image of the shoe to be inspected, and
A function to store a learning model generated by machine learning using the virtual lines extracted from images of multiple shoes that are acceptable products as teacher data, and
A function of determining whether or not the shoe to be inspected is a pass product by inputting a plurality of virtual lines extracted from the image of the shoe to be inspected into the learning model.
A shoe appearance inspection program characterized by the realization of a computer.
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202080108063.2A CN116802483A (en) | 2020-12-25 | 2020-12-25 | Shoe appearance inspection system, shoe appearance inspection method, and shoe appearance inspection program |
JP2022570939A JP7520149B2 (en) | 2020-12-25 | 2020-12-25 | Shoe appearance inspection system, shoe appearance inspection method, and shoe appearance inspection program |
PCT/JP2020/048681 WO2022137494A1 (en) | 2020-12-25 | 2020-12-25 | Shoe appearance inspection system, shoe appearance inspection method, and shoe appearance inspection program |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/JP2020/048681 WO2022137494A1 (en) | 2020-12-25 | 2020-12-25 | Shoe appearance inspection system, shoe appearance inspection method, and shoe appearance inspection program |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2022137494A1 true WO2022137494A1 (en) | 2022-06-30 |
Family
ID=82157646
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2020/048681 WO2022137494A1 (en) | 2020-12-25 | 2020-12-25 | Shoe appearance inspection system, shoe appearance inspection method, and shoe appearance inspection program |
Country Status (3)
Country | Link |
---|---|
JP (1) | JP7520149B2 (en) |
CN (1) | CN116802483A (en) |
WO (1) | WO2022137494A1 (en) |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107114861A (en) * | 2017-03-22 | 2017-09-01 | 青岛小步科技有限公司 | A kind of customization footwear preparation method and system based on imaging pressure and dimensional Modeling Technology |
US20180322623A1 (en) * | 2017-05-08 | 2018-11-08 | Aquifi, Inc. | Systems and methods for inspection and defect detection using 3-d scanning |
WO2019044870A1 (en) * | 2017-09-04 | 2019-03-07 | 日本電産コパル株式会社 | Visual inspection device and product manufacturing system |
US20190096135A1 (en) * | 2017-09-26 | 2019-03-28 | Aquifi, Inc. | Systems and methods for visual inspection based on augmented reality |
JP2019074525A (en) * | 2017-10-13 | 2019-05-16 | マネスキ、アレッサンドロMANNESCHI,Alessandro | Inspection of shoes using thermal camera |
US20200175669A1 (en) * | 2018-12-04 | 2020-06-04 | General Electric Company | System and method for work piece inspection |
CN111340098A (en) * | 2020-02-24 | 2020-06-26 | 安徽大学 | STA-Net age prediction method based on shoe print image |
-
2020
- 2020-12-25 WO PCT/JP2020/048681 patent/WO2022137494A1/en active Application Filing
- 2020-12-25 CN CN202080108063.2A patent/CN116802483A/en active Pending
- 2020-12-25 JP JP2022570939A patent/JP7520149B2/en active Active
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107114861A (en) * | 2017-03-22 | 2017-09-01 | 青岛小步科技有限公司 | A kind of customization footwear preparation method and system based on imaging pressure and dimensional Modeling Technology |
US20180322623A1 (en) * | 2017-05-08 | 2018-11-08 | Aquifi, Inc. | Systems and methods for inspection and defect detection using 3-d scanning |
WO2019044870A1 (en) * | 2017-09-04 | 2019-03-07 | 日本電産コパル株式会社 | Visual inspection device and product manufacturing system |
US20190096135A1 (en) * | 2017-09-26 | 2019-03-28 | Aquifi, Inc. | Systems and methods for visual inspection based on augmented reality |
JP2019074525A (en) * | 2017-10-13 | 2019-05-16 | マネスキ、アレッサンドロMANNESCHI,Alessandro | Inspection of shoes using thermal camera |
US20200175669A1 (en) * | 2018-12-04 | 2020-06-04 | General Electric Company | System and method for work piece inspection |
CN111340098A (en) * | 2020-02-24 | 2020-06-26 | 安徽大学 | STA-Net age prediction method based on shoe print image |
Also Published As
Publication number | Publication date |
---|---|
JPWO2022137494A1 (en) | 2022-06-30 |
CN116802483A (en) | 2023-09-22 |
JP7520149B2 (en) | 2024-07-22 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7409256B2 (en) | Footwear measurement and footwear manufacture systems and methods | |
US10013803B2 (en) | System and method of 3D modeling and virtual fitting of 3D objects | |
JP4137942B2 (en) | Shoe selection support system and shoe selection support method | |
KR102028563B1 (en) | Method of measuring foot size and shape using image processing | |
US11176738B2 (en) | Method for calculating the comfort level of footwear | |
KR101624203B1 (en) | Realtime quality inspection method for shoes sole | |
CN106714914B (en) | Movement posture analytical equipment and movement posture analyze information generating method | |
US20210267315A1 (en) | Markerless foot size estimation device, markerless foot size estimation method, and markerless foot size estimation program | |
JP2022177028A (en) | Information processing device, information processing method, and program | |
CN105976406A (en) | Measurement system, measurement device, foot type measurement method, and foot type measurement system | |
CN108348045A (en) | Method and system for the size for determining apparatus for correcting | |
WO2016067573A1 (en) | Orientation estimation method and orientation estimation device | |
Sarghie et al. | Anthropometric study of the foot using 3D scanning method and statistical analysis | |
WO2022137494A1 (en) | Shoe appearance inspection system, shoe appearance inspection method, and shoe appearance inspection program | |
CN110664409B (en) | Arch type identification method based on pressure acquisition | |
Luximon et al. | Sizing and grading methods with consideration of footwear styles | |
TWI664547B (en) | Insole design method and insole design system | |
KR101781359B1 (en) | A Method Of Providing For Searching Footprint And The System Practiced The Method | |
JP2020018365A (en) | Foot state analysis method | |
KR101678166B1 (en) | System and method for manufacturing the custom made-to-order golf insole using golf swing analyzing | |
JP7356665B2 (en) | Shoe data generation device, shoe data generation method, shoe data generation program | |
JP2020012667A (en) | Identification apparatus, identification method and program | |
Xiong et al. | Foot measurements from 2D digital images | |
US20240212270A1 (en) | Representations of foot features | |
CN111274948B (en) | Method for detecting key points of human feet and shoes in e-commerce image |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20966980 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2022570939 Country of ref document: JP Kind code of ref document: A |
|
WWE | Wipo information: entry into national phase |
Ref document number: 202080108063.2 Country of ref document: CN |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 20966980 Country of ref document: EP Kind code of ref document: A1 |