CN109457398B

CN109457398B - Sweater automatic sewing method based on machine vision perception

Info

Publication number: CN109457398B
Application number: CN201811479720.5A
Authority: CN
Inventors: 明五一; 都金光; 何文斌; 贾豪杰; 刘兴; 胡怀谷; 马军; 李晓科; 曹阳; 侯俊剑; 沈帆; 陈志君
Original assignee: Zhengzhou University of Light Industry
Current assignee: Zhengzhou University of Light Industry
Priority date: 2018-12-05
Filing date: 2018-12-05
Publication date: 2020-08-21
Anticipated expiration: 2038-12-05
Also published as: CN109457398A

Abstract

The sweater automatic sewing method based on machine vision perception comprises the following steps: (1) starting the sweater automatic sewing equipment; (2) the sweater woven piece is conveyed into the stretching and freezing device from front to back in the input flattening device; (3) the sweater woven piece is stretched in a stretching and freezing device and then is frozen into a rigid body convenient for sewing; (4) the sweater fabric piece which is stretched and frozen into a rigid body is conveyed to a sewing preparation device; (5) and the conveying robot finishes turning over, conveying and sewing the frozen sweater woven pieces to assist pushing and sorting, and meanwhile, the sewing device sews the upper sweater woven piece and the lower sweater woven piece which are correspondingly arranged up and down. The invention can replace the traditional manual wool weaving sewing, improve the processing efficiency, realize the improvement of the sewing quality, realize the more stable consistency of products under the batch production, thereby also improving the added value of the products; has certain significance for promoting the development of equipment in the wool weaving industry in China.

Description

Sweater automatic sewing method based on machine vision perception

Technical Field

The invention belongs to the technical field of wool weaving, and particularly relates to an automatic sweater sewing method based on machine vision perception.

Background

China is a big country in wool weaving industry, and the wool weaving process flow mainly comprises four processes of weaving, sewing, washing and inspecting, wherein the weaving, washing and inspecting processes are all automatically operated, and only the sewing is needed to be manually operated, so the wool weaving industry is still labor-intensive. With the continuous rising of labor cost, enterprises hope to realize robot replacement urgently, and product quality assurance realized by depending on personal experience is reduced.

In order to realize automatic linking (sewing) of the sweater, the invention discloses a sweater automatic sewing method based on machine vision perception, which converts a flexible sweater woven piece into a rigid frozen sweater woven piece after stretching and freezing operations, thereby facilitating the realization of turning, carrying and sewing operations and realizing automatic sweater sewing.

Some relevant patent documents are found through the search of domestic patent documents, and the following are mainly found:

the publication number is CN201711299867, which is named as a high-efficiency sweater sewing device. Compared with the traditional sweater sewing machine, the sewing machine can realize that one motor drives six sewing devices to work, thereby greatly improving the production efficiency; the combination of the brake structure, the idle wheel and the real rotating wheel can realize that the belt is pulled to the idle wheel to idle, so that the sewing equipment is suspended; similarly, the belt may be pulled to a solid rotating wheel to allow the suturing apparatus to continue to operate. However, the six sewing devices are connected together through the transmission device in general, and a great deal of manpower is still needed in the production process.

The publication number CN201810318495 is named as a needle checking device and a needle checking method for sweater production, the essence of the patent is the optimization of a conventional needle checking machine, the traditional needle checking machine which is operated by two persons and adopts the electromagnetic induction as the detection principle is changed, and higher automatic needle checking operation is realized. The main realization process is that the conveyer belt with the baffles on two sides sends clothes into the mounting frame, double-sided detection is realized under the action of the infrared detector and the turner under the mounting frame, and the clothes are conveyed to the next station by the mechanical arm after detection is finished, so that manpower reduction, cost reduction and productivity improvement are realized. However, in the detection process, only the front and the back of the sweater are detected without the two inner side surfaces, so that the detection is insufficient, the detection is missed, and the detected sweater with broken needles still needs to be manually checked and removed.

The above-mentioned patent documents relating to sewing machines, which more or less ignore the problems relating to the improvement of production efficiency and the reduction of manpower, while relying on a higher degree of automation for both, provide a highly automated method for automatically sewing sweaters.

Disclosure of Invention

In order to overcome the defects in the prior art, the invention provides the sweater automatic sewing method based on machine vision perception.

In order to solve the technical problems, the invention adopts the following technical scheme: the automatic sweater sewing method based on machine vision perception is operated by adopting automatic sweater sewing equipment; the sweater automatic sewing equipment comprises an input flattening device, a stretching and freezing device, a sewing preparation device, a sewing device, a carrying robot 7 and a control box 14;

the input flattening device, the stretching and freezing device, the sewing preparation device and the sewing device are sequentially connected from front to back, rails 6 are arranged on the left sides of the lower portions of the sewing preparation device and the sewing device along the front-back direction, the bottom of the transfer robot is connected to the rails 6 in a rolling mode, and the working portion of the transfer robot 7 extends to the upper portions of the sewing preparation device and the sewing device;

a control module and a machine vision perception identification module are arranged in the control box 14, the control module is respectively connected with the input flattening device, the stretching and freezing device, the suture preparation device, the suture device and the transfer robot 7 through industrial Ethernet, and the machine vision perception identification module is connected with the control module through a control cable;

the automatic sweater sewing method comprises the following steps;

1. operating a button on the control box 14 to start the sweater automatic sewing device;

2. an operator places the sweater woven sheet on an input flattening device for unfolding, and the sweater woven sheet is conveyed into a stretching and freezing device under the guide of the operator and the forward and backward conveying of the input flattening device;

3. the sweater woven piece is stretched in a stretching and freezing device and then is frozen into a rigid body convenient for sewing;

4 the sweater fabric piece which is stretched and frozen into a rigid body is conveyed to a sewing preparation device;

5, the carrying robot 7 moves back and forth on the track 6 to complete turning, carrying and sewing auxiliary pushing and sorting of the frozen sweater woven pieces, meanwhile, the sewing device sews the upper sweater woven piece and the lower sweater woven piece which are correspondingly placed up and down, the qualified sweater woven piece is carried to the rear output end of the sewing device by the carrying robot 7, and the unqualified product is placed in a waste product frame.

The input flattening device comprises a flattening conveying motor and two supporting plates 1 which are vertically arranged, a plurality of flattening rollers 51 are rotatably arranged between the two supporting plates 1, and the flattening conveying motor is connected with all the flattening rollers 51 through a belt transmission mechanism; the flattening roller 51 at the rearmost side is equal in height and connected with the lower roller 2 at the foremost side;

the stretching freezing device comprises an upper box body 3, a lower box body 5, a conveying stepping motor 30 and lower rollers 2, wherein the top of the lower box body 5 is open, the left side and the right side of the bottom of the lower box body 5 are respectively provided with supporting legs 4, the left side and the right side of the top of the lower box body 5 are respectively provided with a mounting plate 24, the bottom in the lower box body 5 is uniformly provided with a plurality of refrigerating devices 20 which spray cold air upwards and a plurality of water spray pipelines 19, the upper parts of the water spray pipelines 19 are provided with a row of nozzles which spray water mist upwards, the lower rollers 2 are provided with a plurality of lower rollers 2, the plurality of lower rollers 2 are rotatably arranged in the lower box body 5 along the front-back direction, the lower rollers 2 are positioned above the refrigerating devices 20, the water spray pipelines 19 and the lower rollers 2 are horizontally; the conveying stepping motor 30 is in transmission connection with all the lower rollers 2 through a synchronous belt 23;

the bottom of the upper box body 3 is open, the upper box body 3 is buckled on the lower box body 5, a front and back through conveying channel is arranged in the middle of the bottom of the upper box body 3, a plurality of sweater woven piece stretching mechanisms positioned in the upper box body 3 are arranged between the mounting plates 24 on the left side and the right side, the number of the sweater woven piece stretching mechanisms is the same as that of the lower rollers 2, each sweater woven piece stretching mechanism is positioned right above one lower roller 2, a plurality of high-definition cameras 9-1 for shooting downwards are uniformly arranged at the top of the inner side of the upper box body 3, an annular LED lamp is arranged on the periphery of each high-definition camera 9-1, and the high-definition cameras 9-1 are used for detecting the shapes of the sweater woven pieces before and after stretching;

each sweater woven piece stretching mechanism comprises an upper elastic roller 18 which is parallel to the lower roller 2 and is positioned above the lower roller 2, a gap which is smaller than the thickness of a sweater woven piece is formed between the lower roller 2 and the upper elastic roller 18, the upper elastic roller 18 comprises two rigid guide rods 18-3 which are coaxially arranged, a plurality of sections of elastic rubber columns 18-1 and rigid cylinders 18-2 are arranged between the two rigid guide rods 18-3, and the plurality of sections of elastic rubber columns 18-1 and the rigid cylinders 18-2 are arranged at intervals and are fixedly connected with each other in the same axial direction; the two rigid guide rods 18-3 are respectively connected with synchronous stretching driving mechanisms which have the same structure and are symmetrically arranged left and right; an infrared sensor 25 is arranged on the mounting plate 24 between two synchronous stretching driving mechanisms;

the left synchronous stretching driving mechanism comprises a supporting block 32, a stretching stepping motor 22, a supporting piece 26, a rack 21, a rack guide rail 31 and a spring 18-4, wherein the rack guide rail 31 and the supporting piece 26 are fixedly arranged on a mounting plate 24 on the left side, the rack guide rail 31 is parallel to the upper elastic roller 18, the rack 21 is connected to the rack guide rail 31 in a sliding manner, the stretching stepping motor 22 is fixedly arranged on the supporting piece 26, a gear meshed with the rack 21 for transmission is arranged on a main shaft of the stretching stepping motor 22, the supporting block 32 is fixedly arranged at the right end part of the rack 21, a left-right through guide hole is formed in the supporting block 32, a rigid guide rod 18-3 at the left end is inserted in the guide hole in a sliding manner, a limiting cap with the diameter larger than that of the rigid guide rod 18-3 is fixedly arranged at the left end of the rigid guide rod 18-3, the, the left end and the right end of the spring 18-4 are respectively matched with the limiting cap and the supporting block 32 in a pressing mode.

The sewing preparation device comprises a bracket 13, a horizontal transmission belt mechanism 17 is arranged on the bracket 13 along the front-back direction, and the front side of the horizontal transmission belt mechanism 17 is connected with a lower roller 2 at the rearmost side; a first L-shaped bracket 8 is arranged on the right side of the bracket 13, a pre-sewing camera 9-2 positioned above a horizontal conveying belt mechanism 17 is arranged on the first L-shaped bracket 8, the horizontal conveying belt mechanism 17 conveys the sweater woven piece which is stretched and frozen by a stretching and freezing device to an input port of the sewing device, and the movement, pause and stop actions are executed under the visual perception of the pre-sewing camera 9-2;

the sewing device comprises a sewing workbench 15 horizontally arranged on a support 13, a plurality of balls 16 are arranged on the sewing workbench 15, the balls 16 are uniformly embedded on the sewing workbench 15 at intervals of 5-10 mm, a second L-shaped support 50 is arranged on the right side of the sewing workbench 15, a guide camera 9-4 located above the sewing workbench 15 is arranged on the second L-shaped support 50, a sewing machine support column 10 is respectively arranged on the left side and the right side of the sewing workbench 15, a sewing machine 11 located above the left side and the right side of the sewing workbench 15 is arranged at the upper end of each sewing machine support column 10, and a sewing camera 9-3 used for guiding the sewing process of the sewing machine 11 is arranged on each sewing machine 11.

The control module comprises an industrial control computer, an embedded ARM single chip microcomputer, an input/output sub-module, an industrial Ethernet communication sub-module, a display sub-module and a light warning sub-module; the industrial control computer is connected with the embedded ARM single chip microcomputer through an industrial Ethernet communication sub-module, and the embedded ARM single chip microcomputer is connected through an input/output sub-module; the input/output sub-module is respectively connected with a conveying stepping motor control signal of the input flattening module, a conveying stepping motor and a stretching stepping motor control signal of the stretching freezing device and a conveying stepping motor control signal of the suture preparing device, the corresponding stepping motors are sequentially controlled to act in order according to a control flow, the input/output sub-module is also connected with a water spray pipeline 19 of the stretching freezing device and an electric control valve of the refrigerating device 20, and the electric control valve is opened or closed according to the action flow; the input/output sub-module is also connected with and controls the opening and closing of the annular LED lamp and the brightness of the annular LED lamp; the input/output sub-module is also connected with an infrared sensor of the stretching and freezing device to sense the sweater knitted piece; the high-definition camera 9-1 of the stretching freezing device, the pre-sewing camera 9-2 of the sewing preparation device, the sewing camera 9-3 of the sewing device and the guiding camera 9-4 of the sewing device are all connected with an industrial control computer in a USB bus mode, the industrial control computer starts all the high-definition cameras 9-1 of the stretching freezing device to take pictures, a machine vision perception identification module calculates the position information of the sweater woven piece in the stretching freezing device after the pictures are obtained, the outline shape of the sweater woven piece is calculated, furthermore, the industrial control computer also starts the cameras of the sewing preparation device and the guiding camera 9-4 of the sewing device, the machine vision perception identification module identifies the front and back surfaces and the position information of the sweater woven piece after the pictures are obtained, and machine vision perception parameters are provided for the carrying robot 7, further, the industrial control computer also starts a sewing camera 9-3 in the sewing device, and after the picture is obtained, the machine vision perception identification module identifies the current sewing position information of the sweater woven piece and provides information for the transfer robot 7 to sew and assist in pushing the sweater woven piece; the carrying robot 7 is connected with an industrial control computer through an industrial Ethernet;

the machine vision perception identification module comprises an image enhancement sub-module, a monocular vision positioning sub-module, an image segmentation sub-module and a depth convolution network sub-module; the image enhancement submodule performs enhancement processing on the picture acquired by the camera to meet the requirements of sweater woven pieces with different colors and shapes, and the processed image is sent to the monocular vision positioning submodule and the image segmentation submodule to be processed respectively; the monocular vision positioning sub-module obtains position information and outline edges of the sweater woven piece through a characteristic point matching algorithm, wherein the characteristic points are 18-1 in a sweater woven piece stretching mechanism, 18-2 of a rigid cylinder, mark points on a horizontal transmission belt mechanism 17 of a sewing preparation device and balls 16 of the sewing device; the image segmentation submodule performs segmentation processing on the edge image of the sweater piece, performs segmentation and scaling to a fixed size of 128 x 128 pixels, and then sends the edge image to the deep convolution network submodule; the deep convolution network submodule detects the front and back information of the sweater knitted piece and the sewing quality of the sweater knitted piece and feeds the detection result back to the control module.

The specific process of the step 2 is as follows: an operator stands on the left side or the right side of the input flattening device to place a sweater woven sheet to be sewn on a flattening roller 51 for flattening, a flattening conveying motor drives the flattening roller 51 to rotate, the flattening roller 51 drives the sweater woven sheet to move backwards, and when the sweater woven sheet moves to a gap between a lower roller 2 and an upper elastic roller 18 on the foremost side, the sweater woven sheet is slightly extruded under the guidance of the operator and is sent into the gap between the lower roller 2 and the upper elastic roller 18, namely, the sweater woven sheet enters the stretching and freezing device;

the specific process of the step 3 is as follows: one piece of sweater woven piece can be contained in the stretching and freezing device at one time, and when the infrared sensor in the stretching and freezing device detects the sweater woven piece, whether the sweater woven piece can move forwards subsequently is judged; if the stretching and freezing device can move, the control module controls the start and stop of a conveying stepping motor 30 of the stretching and freezing device to move the conveying stepping motor to the proper position for stretching and freezing, and if the conveying stepping motor cannot move, the conveying stepping motor is stretched and frozen at the current position; the execution of the stretching action is completed by the sweater piece stretching mechanism, and the stretching stepping motor 22 of the sweater piece stretching mechanism stretches the sweater piece in place according to the high-definition camera 9-1 under the control of the control module and then transmits the stretching stepping motor to the recognition result of the machine vision perception recognition module so as to start and stop one or more stretching stepping motors 22; the sweater panel is then frozen.

The sweater woven piece is stretched by the following steps: the conveying stepping motor 30 drives the lower roller 2 to rotate, the lower roller 2 drives the sweater woven piece to move backwards until the sweater woven piece completely enters the upper box body 3, then the conveying stepping motor 30 stops, all the stretching stepping motors 22 are started, the stretching stepping motor 22 on the left side moves leftwards along the rack guide rail 31 through the gear driving rack 21, the stretching stepping motor 22 on the right side moves rightwards along the rack guide rail 31 through the gear driving rack 21, a supporting block 32 fixedly connected with the end part of the rack 21 compresses a spring 18-4 and drives a limiting cap to move axially outwards through the compression spring 18-4, the limiting cap axially pulls the rigid guide rod 18-3, the elastic rubber column 18-1 and the rigid cylinder 18-2, the elastic rubber column 18-1 is stretched, the rigid cylinder 18-2 also moves axially, the lower part of the rigid cylinder 18-2 pulls the sweater woven piece to move leftwards, the sweater woven piece is stretched and unfolded left and right until the contour of the sweater woven piece at the current position is consistent with the preset stretching state image information in the industrial control computer or the error is less than 0.1-0.2 cm, and then the stretching stepping motor 22 is stopped.

The process of freezing the sweater woven piece comprises the following steps: the freezing action is completed by the cooperation of the water spray pipeline 19 and the refrigerating device 20, when the sweater woven piece is stretched in place, the control module opens the electromagnetic valve on the water spray pipeline 19 through the input/output module, and independently controls each nozzle in the water spray pipeline 19 according to the recognition result of the sweater woven piece profile of the machine vision perception recognition module, so that unnecessary water spraying is reduced, and water mist is prevented from being jetted onto the high-definition camera 9-1; after the water mist spraying is finished, the control module starts the refrigerating device 20 through the input/output module, the refrigerating device 20 sends cold air upwards towards the sweater woven piece, the refrigerating device 20 adopts low-temperature nitrogen or dry ice for refrigerating, the wet sweater woven piece is quickly frozen, and the wet sweater woven piece is converted into a rigid body from a flexible body, so that automatic sewing is facilitated.

After the sweater woven piece is frozen, starting a conveying stepping motor 30, driving a lower roller 2 to rotate by the conveying stepping motor 30, driving the sweater woven piece to move backwards to a horizontal conveying belt mechanism 17 of a sewing preparation device by the lower roller 2, then stretching a stepping motor 22 to rotate reversely, driving a rack 21 to reset, and contracting and resetting an upper elastic roller 18; stretching the next sweater woven piece conveyed by the input flattening device;

the step 4 specifically comprises the following steps: outputting the stretched and frozen sweater knitted piece to a sewing preparation device, and identifying the position and the front and back surfaces of the current sweater knitted piece by a machine vision perception identification module; when the subsequent sewing device requires the current piece of sweater fabric to be turned over, it is handled by the transfer robot 7 under the direction of the pre-sewing camera 9-2 of the sewing preparation device.

The specific process of the step 5 is as follows: the transfer robot 7 processes position and outline information through a machine vision perception identification module under the shooting of a guide camera 9-4 of the sewing device, two pieces of sweater woven pieces to be sewn in the sewing preparation device are sequentially transferred to a workbench 15 of the sewing device, the outline of the outer edges of the two pieces of sweater woven pieces to be sewn is placed below a sewing machine 11 on the right side of the sewing device, and the transfer robot 7 aligns the two pieces of sweater woven pieces to be sewn;

then, the sewing machine 11 on the right side of the sewing device starts to work, the carrying robot 7 processes position and outline information of the position through the machine vision perception identification module under the shooting of the sewing camera 9-3, and pushes the sweater woven piece in an auxiliary mode to realize automatic sewing of the sweater woven piece; after the sewing machine 11 on the right side of the sewing device finishes sewing, the sewing machine 11 on the left side of the sewing device automatically sews the left side of the sweater woven piece according to the flow; the sewing frequency of the sewing machine 11 is proportional to the advancing speed of the carrier robot 7; due to the laying of the balls 16 on the workbench 15, the motion friction resistance of the frozen sweater knitting piece is reduced;

a sorting stage; the carrying robot 7 carries out sewing quality detection on the sewn sweater fabric piece through a machine vision perception identification module under the shooting of a guide camera 9-4 of the sewing device to obtain sewing defect information; the qualified products of the sewing quality are moved to the output end at the rear side of the sewing device by the carrying robot 7, and the unqualified products are put into a waste frame by the carrying robot 7. The transfer robot 7 transfers the sweater woven piece by using a manipulator suction cup.

The machine vision perception identification module is used for detecting the sewing quality and completing a deep convolution network sub-module, and the specific working process is as follows;

the sweater piece stitched image after the image segmentation submodule is converted into 3 RGB monochromatic images, the images are sent to an A1 convolutional layer in a depth convolution network submodule to generate 18 126 x 126 pixel images, the images are convolved by adopting 5 x 5 windows, then 1/2 pooling operation is carried out by an A2 pooling layer to generate 18 63 x 63 pixel images, then second convolution operation is carried out, the images are sent to an A3 convolutional layer in the depth convolution network submodule to generate 72 61 x 61 pixel images, the images are convolved by adopting 3 x 3 windows, then 1/2 pooling operation is carried out by an A4 pooling layer to generate 72 30 x 30 pixel images, further, the images are processed by an A5 full-connection layer in the depth convolution network submodule to output 1024-dimensional vectors, and further, the images are processed by an A6 full-connection layer in the depth convolution network submodule, outputting a 256-dimensional vector;

further, the sweater piece stitched image after the image segmentation submodule is converted into 3 RGB monochromatic images, and is sent to a B1 convolution layer in a depth convolution network submodule to generate 18 127 × 127 pixel images, the images are convoluted by adopting a3 × 3 window, then 1/2 pooling operation is carried out by a B2 pooling layer to generate 18 63 × 63 pixel images, then, second convolution operation is carried out, the images are sent to a B3 convolution layer in the depth convolution network submodule to generate 72 63 × 63 pixel images, the images are convoluted by adopting a1 × 1 window, then 1/2 pooling operation is carried out by a B4 pooling layer to generate 72 31 × 31 pixel images, further, the images are processed by a B5 full-link layer in the depth convolution network submodule to output 1024-dimensional vectors, and further, the images are processed by a B6 full-link layer in the depth convolution network submodule, outputting a 256-dimensional vector;

finally, performing collective addition operation on the 256-dimensional vector output by the A6 full-connection layer and the 256-dimensional vector output by the B6 full-connection layer by using an S7 collection layer in a deep convolutional network submodule to generate characteristic information for representing sewing defects, wherein the characteristic information is 512-dimensional vector, and then outputting 2-dimensional vector by using an S8 soft regression layer in the deep convolutional network submodule to represent qualified and unqualified probability distribution of the image to be identified, so as to identify whether the sewing of the sweater piece to be identified has defects;

the number of samples can be increased by an offline training sample library of the deep convolutional network sub-module; therefore, the images to be identified belong to qualified and unqualified categories, and can be further subdivided along with the increase of the number of samples; the deep convolutional network sub-module can be trained and updated by a user in the using process, and can also be selected to be updated regularly by an equipment manufacturer; the device supports multiple versions of deep convolutional network sub-modules, the multiple versions are different convolution and pooling parameters, and the multiple versions can be selected by an end user according to an actual application scene.

By adopting the technical scheme, the invention has the following technical effects: the invention firstly stretches the sweater woven piece, then sprays water mist and freezes the sweater woven piece, and then converts the flexible sweater woven piece into the rigid frozen sweater woven piece, thereby being convenient for realizing the operations of turning, carrying and sewing, and realizing the automatic sewing of the sweater. On one hand, the invention can replace the traditional manual wool weaving sewing to improve the processing efficiency, and on the other hand, the sewing quality is improved, the product consistency under the batch production is more stable, thereby improving the added value of the product. Therefore, the invention can be used for automatically sewing sweater knitted pieces in the wool weaving industry, has certain significance for improving the development of equipment in the wool weaving industry in China, and has better market application prospect.

Drawings

FIG. 1 is a schematic perspective view of an automatic sweater sewing machine according to the present invention from one perspective;

FIG. 2 is a schematic perspective view of an automatic sweater sewing machine according to the present invention, from another perspective view;

FIG. 3 is a schematic view of the tension freezing apparatus of FIG. 1 with the upper housing removed;

FIG. 4 is a schematic bottom view of the upper housing of FIG. 1;

FIG. 5 is a schematic view of the arrangement of the refrigeration unit and the water mist spray pipe of FIG. 3;

FIG. 6 is a schematic view of a sweater panel stretching mechanism of FIG. 3;

FIG. 7 is a schematic view of a sewing machine automatically sewing an upper sweater panel and a lower sweater panel;

FIG. 8 is a schematic diagram of the control module logical connections;

FIG. 9 is a schematic diagram of a deep convolutional network submodule.

Detailed Description

As shown in fig. 1 to 9, the automatic sweater sewing method based on machine vision perception of the present invention is operated by using an automatic sweater sewing device; the sweater automatic sewing equipment comprises an input flattening device, a stretching and freezing device, a sewing preparation device, a sewing device, a carrying robot 7 and a control box 14;

the automatic sweater sewing method comprises the following steps;

(1) operating a button on the control box 14 to start the sweater automatic sewing device;

(2) the operator places the sweater woven sheet on the input flattening device for unfolding, and the sweater woven sheet is conveyed into the stretching and freezing device under the guide of the operator and the forward and backward conveying of the input flattening device;

(3) the sweater woven piece is stretched in a stretching and freezing device and then is frozen into a rigid body convenient for sewing;

(4) the sweater fabric piece which is stretched and frozen into a rigid body is conveyed to a sewing preparation device;

(5) the transfer robot 7 moves back and forth on the track 6 to complete turning, carrying and sewing auxiliary pushing and sorting of the frozen sweater woven pieces, meanwhile, the sewing device sews the upper sweater woven piece and the lower sweater woven piece which are correspondingly placed up and down, the qualified sweater woven pieces are conveyed to the rear output end of the sewing device by the transfer robot 7, and unqualified products are placed in a waste product frame.

The specific process of the step (2) is as follows: an operator stands on the left side or the right side of the input flattening device to place a sweater woven sheet to be sewn on a flattening roller 51 for flattening, a flattening conveying motor drives the flattening roller 51 to rotate, the flattening roller 51 drives the sweater woven sheet to move backwards, and when the sweater woven sheet moves to a gap between a lower roller 2 and an upper elastic roller 18 on the foremost side, the sweater woven sheet is slightly extruded under the guidance of the operator and is sent into the gap between the lower roller 2 and the upper elastic roller 18, namely, the sweater woven sheet enters the stretching and freezing device;

the specific process of the step (3) is as follows: one piece of sweater woven piece can be contained in the stretching and freezing device at one time, and when the infrared sensor in the stretching and freezing device detects the sweater woven piece, whether the sweater woven piece can move forwards subsequently is judged; if the stretching and freezing device can move, the control module controls the start and stop of a conveying stepping motor 30 of the stretching and freezing device to move the conveying stepping motor to the proper position for stretching and freezing, and if the conveying stepping motor cannot move, the conveying stepping motor is stretched and frozen at the current position; the execution of the stretching action is completed by the sweater piece stretching mechanism, and the stretching stepping motor 22 of the sweater piece stretching mechanism stretches the sweater piece in place according to the high-definition camera 9-1 under the control of the control module and then transmits the stretching stepping motor to the recognition result of the machine vision perception recognition module so as to start and stop one or more stretching stepping motors 22; the sweater panel is then frozen.

the step (4) is specifically as follows: outputting the stretched and frozen sweater knitted piece to a sewing preparation device, and identifying the position and the front and back surfaces of the current sweater knitted piece by a machine vision perception identification module; when the subsequent sewing device requires the current piece of sweater fabric to be turned over, it is handled by the transfer robot 7 under the direction of the pre-sewing camera 9-2 of the sewing preparation device.

The specific process of the step (5) is as follows: the transfer robot 7 processes position and outline information through a machine vision perception identification module under the shooting of a guide camera 9-4 of the sewing device, two pieces of sweater woven pieces to be sewn in the sewing preparation device are sequentially transferred to a workbench 15 of the sewing device, the outline of the outer edges of the two pieces of sweater woven pieces to be sewn is placed below a sewing machine 11 on the right side of the sewing device, and the transfer robot 7 aligns the two pieces of sweater woven pieces to be sewn;

a sorting stage; the carrying robot 7 carries out sewing quality detection on the sewn sweater fabric piece through a machine vision perception identification module under the shooting of a guide camera 9-4 of the sewing device to obtain sewing defect information; the qualified products of the sewing quality are moved to the output end at the rear side of the sewing device by the carrying robot 7, and the unqualified products are put into a waste frame by the carrying robot 7. The transfer robot 7 operates a robot chuck (i.e., a working part of the transfer robot 7) for transferring the sweater woven fabric.

the sweater piece stitched image after the image segmentation submodule is converted into 3 RGB single-color images, the 3 RGB single-color images are sent to an A1 convolutional layer in a deep convolutional network submodule to generate 18 images of 126 x 126 pixels (5 x 5 windows are adopted for convolution), then 1/2 pooling operation is carried out by an A2 pooling layer to generate 18 images of 63 x 63 pixels, and then, performing a second convolution operation, sending the second convolution operation to an A3 convolution layer in a deep convolution network submodule to generate 72 images with 61 × 61 pixels (performing convolution by adopting A3 × 3 window), then performing 1/2 pooling operation by an A4 pooling layer to generate 72 images with 30 × 30 pixels, further performing A5 full-connection layer processing in the deep convolution network submodule to output vectors with 1024 dimensions, and further performing A6 full-connection layer processing in the deep convolution network submodule to output vectors with 256 dimensions;

further, the sweater piece stitched image after the image segmentation submodule is converted into 3 RGB monochromatic images, and is sent to a B1 convolution layer in a depth convolution network submodule to generate 18 127 × 127 pixel images (convolution is carried out by adopting a3 × 3 window), then 1/2 pooling operation is carried out by a B2 pooling layer to generate 18 63 × 63 pixel images, then, second convolution operation is carried out, the sweater piece stitched image is sent to a B3 convolution layer in the depth convolution network submodule to generate 72 63 × 63 pixel images (convolution is carried out by adopting a1 × 1 window), then 1/2 pooling operation is carried out by a B4 pooling layer to generate 72 31 × 31 pixel images, further, the sweater piece stitched image is processed by a B5 full connection layer in the depth convolution network submodule to output 1024-dimension vectors, and further processed by a B6 full connection layer in the depth convolution network submodule, outputting a 256-dimensional vector;

the number of samples can be increased by an offline training sample library of the deep convolutional network sub-module; therefore, the images to be identified belong to qualified and unqualified categories, and can be further subdivided along with the increase of the number of samples; the deep convolutional network sub-module can be trained and updated by a user in the using process, and can also be selected to be updated regularly by an equipment manufacturer; the device supports multi-version (convolution and pooling parameters) deep convolution network sub-modules, and can be selected by an end user according to actual application scenarios.

The present embodiment is not intended to limit the shape, material, structure, etc. of the present invention in any way, and any simple modification, equivalent change and modification made to the above embodiments according to the technical spirit of the present invention are within the scope of the technical solution of the present invention.

Claims

1. The sweater automatic sewing method based on machine vision perception is characterized by comprising the following steps: the automatic sweater sewing method is operated by adopting automatic sweater sewing equipment; the sweater automatic sewing equipment comprises an input flattening device, a stretching and freezing device, a sewing preparation device, a sewing device, a carrying robot (7) and a control box (14);

the input flattening device, the stretching and freezing device, the sewing preparation device and the sewing device are sequentially connected from front to back, the left sides of the lower parts of the sewing preparation device and the sewing device are provided with rails (6) along the front-back direction, the bottom of the transfer robot is connected to the rails (6) in a rolling manner, and the working part of the transfer robot (7) extends to the upper parts of the sewing preparation device and the sewing device;

a control module and a machine vision perception identification module are arranged in the control box (14), the control module is respectively connected with the input flattening device, the stretching and freezing device, the sewing preparation device, the sewing device and the transfer robot (7) through an industrial Ethernet, and the machine vision perception identification module is connected with the control module through a control cable;

the automatic sweater sewing method comprises the following steps;

(1) operating a button on the control box (14) to start the sweater automatic sewing equipment;

(5) the carrying robot (7) moves back and forth on the track (6) to complete turning, carrying, sewing, assisting, pushing and sorting of the frozen sweater woven pieces, meanwhile, the sewing device sews the upper sweater woven piece and the lower sweater woven piece which are correspondingly placed up and down, the qualified sweater woven pieces are carried to the rear output end of the sewing device by the carrying robot (7), and unqualified products are placed in a waste product frame;

the input flattening device comprises a flattening conveying motor and two supporting plates (1) which are vertically arranged, a plurality of flattening rollers (51) are rotatably arranged between the two supporting plates (1), and the flattening conveying motor is connected with all the flattening rollers (51) through a belt transmission mechanism; the flattening roller (51) at the rear side is equal in height and connected with the lower roller (2) at the front side;

the stretching freezing device comprises an upper box body (3) and a lower box body (5), the cooling device comprises a conveying stepping motor (30) and lower rollers (2), wherein the top of a lower box body (5) is open, the left side and the right side of the bottom of the lower box body (5) are respectively provided with supporting legs (4), the left side and the right side of the top of the lower box body (5) are respectively provided with a mounting plate (24), the bottom in the lower box body (5) is uniformly provided with a plurality of refrigerating devices (20) which spray cold air upwards and a plurality of water spray pipelines (19), the upper part of each water spray pipeline (19) is provided with a row of nozzles which spray water fog upwards, the lower rollers (2) are provided with a plurality of lower rollers, the lower rollers (2) are rotatably arranged in the lower box body (5) along the front-back direction, the lower rollers (2) are positioned above the refrigerating devices (20), the water spray pipelines (19) and the lower rollers (2) are horizontally arranged along the left-right direction; the conveying stepping motor (30) is in transmission connection with all the lower rollers (2) through a synchronous belt (23);

the bottom of the upper box body (3) is open, the upper box body (3) is buckled on the lower box body (5), a front-back through conveying channel is arranged in the middle of the bottom of the upper box body (3), a plurality of sweater woven piece stretching mechanisms positioned in the upper box body (3) are arranged between the mounting plates (24) on the left side and the right side, the number of sweater woven piece stretching mechanisms is the same as that of the lower rollers (2), each sweater woven piece stretching mechanism is positioned right above one lower roller (2), a plurality of high-definition cameras (9-1) for shooting downwards are uniformly arranged at the top of the inner side of the upper box body (3), an annular LED lamp is arranged on the periphery of each high-definition camera (9-1), and the high-definition cameras (9-1) are used for detecting the shapes of the sweater woven pieces before and after stretching;

each sweater woven piece stretching mechanism comprises an upper elastic roller (18) which is parallel to the lower roller (2) and is positioned above the lower roller (2), a gap which is smaller than the thickness of a sweater woven piece is formed between the lower roller (2) and the upper elastic roller (18), the upper elastic roller (18) comprises two rigid guide rods (18-3) which are coaxially arranged, a plurality of sections of elastic rubber columns (18-1) and rigid cylinders (18-2) are arranged between the two rigid guide rods (18-3), and the elastic rubber columns (18-1) and the rigid cylinders (18-2) are arranged at intervals and are fixedly connected with each other in the same axial direction; the two rigid guide rods (18-3) are respectively connected with synchronous stretching driving mechanisms which have the same structure and are symmetrically arranged left and right; an infrared sensor (25) is arranged on a mounting plate (24) between two synchronous stretching driving mechanisms;

the left synchronous stretching driving mechanism comprises a supporting block (32), a stretching stepping motor (22), a supporting piece (26), a rack (21), a rack guide rail (31) and a spring (18-4), wherein the rack guide rail (31) and the supporting piece (26) are fixedly arranged on a left mounting plate (24), the rack guide rail (31) is parallel to an upper elastic roller (18), the rack (21) is connected onto the rack guide rail (31) in a sliding manner, the stretching stepping motor (22) is fixedly arranged on the supporting piece (26), a gear meshed with the rack (21) for transmission is arranged on a main shaft of the stretching stepping motor (22), the supporting block (32) is fixedly arranged at the right end part of the rack (21), a left through guide hole and a right through guide hole are formed in the supporting block (32), a rigid guide rod (18-3) at the left end is inserted into the guide hole in a sliding manner, a limiting cap with the diameter larger than that of the rigid guide rod (18-3), the limiting cap is positioned above the rack (21) and on the left side of the supporting block (32), the spring (18-4) is sleeved on the rigid guide rod (18-3), and the left end and the right end of the spring (18-4) are respectively in pressing fit with the limiting cap and the supporting block (32).

2. The automatic sweater sewing method based on machine vision perception according to claim 1, characterized in that: the sewing preparation device comprises a bracket (13), wherein a horizontal transmission belt mechanism (17) is arranged on the bracket (13) along the front-back direction, and the front side of the horizontal transmission belt mechanism (17) is connected with a lower roller (2) at the rearmost side; a first L-shaped bracket (8) is arranged on the right side of the bracket (13), a pre-sewing camera (9-2) positioned above the horizontal conveying belt mechanism (17) is arranged on the first L-shaped bracket (8), the horizontal conveying belt mechanism (17) conveys the sweater woven piece which is stretched and frozen by the stretching and freezing device to an input port of the sewing device, and the movement, pause and stop actions are executed under the visual perception of the pre-sewing camera (9-2);

the sewing device comprises a sewing workbench (15) horizontally arranged on a support (13), a plurality of balls (16) are arranged on the sewing workbench (15), the balls (16) are uniformly embedded on the sewing workbench (15) according to the distance of 5-10 mm, a second L-shaped support (50) is arranged on the right side of the sewing workbench (15), a guide camera (9-4) located above the sewing workbench (15) is arranged on the second L-shaped support (50), a sewing machine support (10) is respectively arranged on the left side and the right side of the sewing workbench (15), a sewing machine (11) located above the left side and the right side of the sewing workbench (15) is arranged at the upper end of each sewing machine support (10), and a sewing camera (9-3) used for guiding the sewing process of the sewing machine (11) is arranged on each sewing machine (11).

3. The automatic sweater sewing method based on machine vision perception according to claim 2, characterized in that: the control module comprises an industrial control computer, an embedded ARM single chip microcomputer, an input/output sub-module, an industrial Ethernet communication sub-module, a display sub-module and a light warning sub-module; the industrial control computer is connected with the embedded ARM single chip microcomputer through an industrial Ethernet communication sub-module, and the embedded ARM single chip microcomputer is connected through an input/output sub-module; the input/output sub-module is respectively connected with a conveying stepping motor control signal of the input flattening module, a conveying stepping motor and a stretching stepping motor control signal of the stretching freezing device and a conveying stepping motor control signal of the suture preparation device, and sequentially controls the corresponding stepping motors to orderly act according to a control flow, and is also connected with a water spray pipeline (19) of the stretching freezing device and an electric control valve of a refrigerating device (20), and the electric control valve is opened or closed according to the action flow; the input/output sub-module is also connected with and controls the opening and closing of the annular LED lamp and the brightness of the annular LED lamp; the input/output sub-module is also connected with an infrared sensor of the stretching and freezing device to sense the sweater knitted piece; the high-definition camera (9-1) of the stretching freezing device, the pre-sewing camera (9-2) of the sewing preparation device, the sewing camera (9-3) of the sewing device and the guiding camera (9-4) of the sewing device are all connected with an industrial control computer in a USB bus mode, the industrial control computer starts the shooting of all the high-definition cameras (9-1) in the stretching freezing device, the position information of the sweater woven piece in the stretching freezing device is calculated by a machine vision perception identification module after the photo is obtained, the outline shape of the sweater woven piece is calculated, furthermore, the industrial control computer also starts the cameras in the sewing preparation device and the guiding camera (9-4) in the sewing device, the front and back sides and the position information of the sweater woven piece are identified by the machine vision perception identification module after the photo is obtained, and machine vision perception parameters are provided for a carrying robot (7), further, the industrial control computer also starts a sewing camera (9-3) in the sewing device, and after the picture is obtained, the machine vision perception identification module identifies the current sewing position information of the sweater woven piece and provides information for the carrying robot (7) to sew and assist in pushing the sweater woven piece; the carrying robot (7) is connected with an industrial control computer through an industrial Ethernet;

the machine vision perception identification module comprises an image enhancement sub-module, a monocular vision positioning sub-module, an image segmentation sub-module and a depth convolution network sub-module; the image enhancement submodule performs enhancement processing on the picture acquired by the camera to meet the requirements of sweater woven pieces with different colors and shapes, and the processed image is sent to the monocular vision positioning submodule and the image segmentation submodule to be processed respectively; the monocular vision positioning sub-module obtains the position information and the outline edge of the sweater woven piece through a characteristic point matching algorithm, wherein the characteristic points are mark points on a horizontal transmission belt mechanism (17) of the sweater woven piece stretching mechanism (18-1), a rigid cylinder (18-2) and a sewing preparation device and balls (16) of the sewing device; the image segmentation submodule performs segmentation processing on the edge image of the sweater piece, performs segmentation and scaling to a fixed size of 128 x 128 pixels, and then sends the edge image to the deep convolution network submodule; the deep convolution network submodule detects the front and back information of the sweater knitted piece and the sewing quality of the sweater knitted piece and feeds the detection result back to the control module.

4. The automatic sweater sewing method based on machine vision perception according to claim 3, characterized in that: the specific process of the step (2) is as follows: an operator stands on the left side or the right side of the input flattening device to place a sweater woven sheet to be sewn on a flattening roller (51) for flattening, a flattening conveying motor drives the flattening roller (51) to rotate, the flattening roller (51) drives the sweater woven sheet to move backwards, and when the sweater woven sheet moves to a gap between a lower roller (2) and an upper elastic roller (18) on the foremost side, the sweater woven sheet is slightly extruded under the guidance of the operator and is sent into the gap between the lower roller (2) and the upper elastic roller (18), namely, the sweater woven sheet enters the stretching and freezing device;

the specific process of the step (3) is as follows: one piece of sweater woven piece can be contained in the stretching and freezing device at one time, and when the infrared sensor in the stretching and freezing device detects the sweater woven piece, whether the sweater woven piece can move forwards subsequently is judged; if the mobile stretching and freezing device can move, the control module controls the start and stop of a conveying stepping motor (30) of the stretching and freezing device to move the stretching and freezing device to the proper position for stretching and freezing, and if the stretching and freezing device cannot move, the stretching and freezing device stretches and freezes at the current position; the execution of the stretching action is finished by a sweater piece stretching mechanism, a stretching stepping motor (22) of the sweater piece stretching mechanism stretches the sweater piece in place according to a high-definition camera (9-1) under the control of a control module and then transmits the stretching stepping motor to an identification result of a machine vision perception identification module so as to start and stop one or more stretching stepping motors (22); the sweater panel is then frozen.

5. The automatic sweater sewing method based on machine vision perception according to claim 4, characterized in that: the sweater woven piece is stretched by the following steps: the lower roller (2) is driven by the conveying stepping motor (30) to rotate, the lower roller (2) drives the sweater woven pieces to move backwards until all the sweater woven pieces enter the upper box body (3), the conveying stepping motor (30) stops, all the stretching stepping motors (22) are started, the stretching stepping motor (22) on the left side drives the rack (21) to move leftwards along the rack guide rail (31) through the gear, the stretching stepping motor (22) on the right side drives the rack (21) to move rightwards along the rack guide rail (31) through the gear, a supporting block (32) fixedly connected with the end part of the rack (21) compresses a spring (18-4) and drives a limiting cap to move outwards and axially through the compression spring (18-4), the limiting cap axially pulls the rigid guide rod (18-3), the elastic rubber column (18-1) and the rigid cylinder (18-2), and the elastic rubber column (18-1) is stretched, the rigid cylinder (18-2) also moves axially, the lower part of the rigid cylinder (18-2) pulls the sweater woven piece to move left and right, the sweater woven piece is stretched and unfolded left and right until the contour of the sweater woven piece at the current position is consistent with the preset stretching state image information in the industrial control computer or the error is less than 0.1-0.2 cm, and then the stretching stepping motor (22) stops.

6. The automatic sweater sewing method based on machine vision perception according to claim 5, characterized in that: the process of freezing the sweater woven piece comprises the following steps: the freezing action is completed by the cooperation of the water spray pipeline (19) and the refrigerating device (20), when the sweater woven piece is stretched in place, the control module starts an electromagnetic valve on the water spray pipeline (19) through the input/output module, and independently controls each nozzle in the water spray pipeline (19) according to the identification result of the sweater woven piece profile of the machine vision perception identification module, so that unnecessary water spray is reduced, and water mist is prevented from being sprayed onto the high-definition camera (9-1); after the water mist is sprayed, the control module starts the refrigerating device (20) through the input/output module, the refrigerating device (20) sends cold air upwards towards the sweater woven piece, the refrigerating device (20) adopts low-temperature nitrogen or dry ice for refrigeration, the wet sweater woven piece is quickly frozen, the soft body is converted into the rigid body, and automatic sewing is facilitated.

7. The automatic sweater sewing method based on machine vision perception according to claim 6, characterized in that: after the sweater woven piece is frozen, starting a conveying stepping motor (30), driving a lower roller (2) to rotate by the conveying stepping motor (30), driving the sweater woven piece to move backwards to a horizontal conveying belt mechanism (17) of a sewing preparation device by the lower roller (2), then stretching the stepping motor (22) to rotate reversely, driving a rack (21) to reset, and contracting and resetting an upper elastic roller (18); stretching the next sweater woven piece conveyed by the input flattening device;

the step (4) is specifically as follows: outputting the stretched and frozen sweater knitted piece to a sewing preparation device, and identifying the position and the front and back surfaces of the current sweater knitted piece by a machine vision perception identification module; when the subsequent sewing device needs the current sweater piece to turn over, the handling robot (7) processes the sweater piece under the guidance of a pre-sewing camera (9-2) of the sewing preparation device.

8. The automatic sweater stitching method based on machine vision perception according to claim 7, characterized in that: the specific process of the step (5) is as follows: the transfer robot (7) processes position and profile information through a machine vision perception identification module under the shooting of a guide camera (9-4) of the sewing device, two pieces of sweater woven pieces to be sewn in the sewing preparation device are sequentially transferred to a workbench (15) of the sewing device, the outer edge profiles of the two pieces of sweater woven pieces to be sewn are placed below a sewing machine (11) on the right side of the sewing device, and the transfer robot (7) aligns the two pieces of sweater woven pieces to be sewn;

then, the sewing machine (11) on the right side of the sewing device starts to work, and the carrying robot (7) processes position and outline information thereof through the machine vision perception identification module under the shooting of the sewing camera (9-3) to assist in pushing the sweater woven piece, so that the sweater woven piece is automatically sewn; after the sewing machine (11) on the right side of the sewing device finishes sewing, the sewing machine (11) on the left side of the sewing device automatically sews the left side of the sweater woven piece according to the flow; the sewing frequency of the sewing machine (11) is in direct proportion to the advancing speed of the carrying robot (7); due to the laying of the balls (16) on the workbench (15), the motion friction resistance of the frozen sweater knitting piece is reduced;

a sorting stage; the carrying robot (7) performs sewing quality detection on the sewn sweater woven piece through the machine vision perception identification module under the photographing of a guide camera (9-4) of the sewing device to acquire sewing defect information; the qualified products of the sewing quality are moved to the output end at the rear side of the sewing device by the carrying robot (7), and the unqualified products are put into a waste product frame by the carrying robot (7); the carrying robot (7) carries the sweater knitted piece by adopting a mechanical arm sucker.

9. The automatic sweater stitching method based on machine vision perception according to claim 8, characterized in that: the machine vision perception identification module is used for detecting the sewing quality and completing a deep convolution network sub-module, and the specific working process is as follows;