CN113562465B - Visual guiding method and system for sheet placement - Google Patents
Visual guiding method and system for sheet placement Download PDFInfo
- Publication number
- CN113562465B CN113562465B CN202111127301.7A CN202111127301A CN113562465B CN 113562465 B CN113562465 B CN 113562465B CN 202111127301 A CN202111127301 A CN 202111127301A CN 113562465 B CN113562465 B CN 113562465B
- Authority
- CN
- China
- Prior art keywords
- channel
- data
- template
- position information
- sheet
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
- B65G—TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS
- B65G49/00—Conveying systems characterised by their application for specified purposes not otherwise provided for
- B65G49/05—Conveying systems characterised by their application for specified purposes not otherwise provided for for fragile or damageable materials or articles
- B65G49/06—Conveying systems characterised by their application for specified purposes not otherwise provided for for fragile or damageable materials or articles for fragile sheets, e.g. glass
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
- B65G—TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS
- B65G43/00—Control devices, e.g. for safety, warning or fault-correcting
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
- B65G—TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS
- B65G49/00—Conveying systems characterised by their application for specified purposes not otherwise provided for
- B65G49/05—Conveying systems characterised by their application for specified purposes not otherwise provided for for fragile or damageable materials or articles
- B65G49/06—Conveying systems characterised by their application for specified purposes not otherwise provided for for fragile or damageable materials or articles for fragile sheets, e.g. glass
- B65G49/061—Lifting, gripping, or carrying means, for one or more sheets forming independent means of transport, e.g. suction cups, transport frames
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
- B65G—TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS
- B65G2203/00—Indexing code relating to control or detection of the articles or the load carriers during conveying
- B65G2203/04—Detection means
- B65G2203/041—Camera
Abstract
The invention discloses a visual guiding method and a system for sheet placement, which comprises the following steps: positioning all channels under the camera to generate first channel position information; extracting second channel position data from the first channel position information; controlling the grabbing equipment to place the sheet object into the current channel according to the second channel position data; acquiring position data of a current channel as third channel position data; acquiring channel position change data according to the change of the third channel position data relative to the second channel position data, and correcting the position information of the next channel in the first channel position information according to the channel position change data; and performing next sheet channel placement according to the corrected first channel position information. According to the visual guiding method and system for sheet placement, disclosed by the invention, by the technology, the influence of lens imaging deformation is reduced, the efficiency of machine visual identification when the sheet is placed is improved, and the balance of placement precision and efficiency is realized.
Description
Technical Field
The invention relates to a machine vision technology, in particular to a visual guiding method and a visual guiding system for sheet placement.
Background
The problem of carrying the sheet materials in the loading and unloading process is encountered in many fields, and the typical case is the problem of loading and unloading various glass sheets, such as liquid crystal screens and mobile phone screens in the field of consumer electronics, and also such as lenses with various functions, and the basic parts of the glass sheets are various glass-contained sheet materials. Handling of sheets is often accompanied by problems such as brittleness, surface scratches, etc. In an automatic production process, in order to reduce the process complexity, the foam box is often freely placed on a conveyor belt or a transfer table, the foam box cannot be correspondingly fixed in a certain area, and a machine vision system is introduced to position the foam box and a related installation slot, so that a guiding manipulator guides the placement of the sheet-shaped object.
For efficiency, the foam channels are typically formed in a foam box with a series of channel arrays (e.g., 3 x 5, 1 x 4, or 1 x 6 arrays) of different specifications depending on the size of the sheet, and in industrial automation, the sheet is typically handled by sucking the sheet with a suction cup and then placing the sheet into the foam box with vertical channels, and in this process, the foam channels are typically positioned with a machine vision system, and the positioning data is then transmitted to a robot, which then performs the sucking and handling of the sheet, see fig. 7.
The problem of doing so is three-fold as follows: (1) when searching for a matching area, a template is generally used for searching, but the corresponding channel is searched for the whole image, so that the searching time is long, and the efficiency is not high enough; (2) the concrete forms of different foam channels are often slightly different due to production factors such as processing, and other different industrial lenses have certain deformation during imaging, so that when a template extracted from the area A is placed in a search groove image in the area B, the matching degree is often insufficient, please refer to fig. 8; (3) after the placement is completed, the foam box storing the glass sheet may move to some extent, and for the placement movements with accuracy requirements of 0.5-1mm, the movement of the foam box may affect the positioning accuracy of the channels, and the first two problems may occur again if the slot images are re-matched. The three aspects need to be adjusted correspondingly in both the mounting precision and the working time.
Disclosure of Invention
The invention aims to solve the technical problem that the machine vision scheme for carrying the sheet-shaped objects in the prior art cannot realize the balance of precision and efficiency, and aims to provide a sheet-shaped object placing vision guiding method and system to solve the problems.
The invention is realized by the following technical scheme:
a sheet placement visual guidance method comprising:
positioning all channels under the camera to generate first channel position information;
when sheet channel placement is executed, extracting the position information of the current channel from the first channel position information as second channel position data;
controlling a gripping device to place the sheet into a current channel according to the second channel position data;
after the sheet-shaped object is placed in the current channel, acquiring position data of the current channel as third channel position data;
acquiring channel position change data according to the change of the third channel position data relative to the second channel position data, and correcting the position information of the next channel in the first channel position information according to the channel position change data;
and performing next sheet channel placement according to the corrected first channel position information.
In the prior art, some technologies for feeding and discharging through machine vision are disclosed, automatic feeding and discharging are realized mainly through the design of a limiting structure and the design of a simple visual algorithm, but the control precision of loading the sheet-shaped objects is difficult to guarantee in the process of 0.5-1mm, and the precision and the efficiency are difficult to guarantee.
When the method is implemented, in order to save computing resources and improve the recognition efficiency, a channel matching template mode is adopted as a reference of machine vision recognition, and the method has the advantages of being obvious. In practice, the inventor finds that in order to improve the loading efficiency of the sheet materials, the foam box for storing the sheet materials is not fixed generally, which causes the foam box to move when each sheet material is loaded, and needs to be repositioned after each movement of the foam box, and if the full channel is positioned by one template in the prior art, the loading efficiency is reduced. In the embodiment, after the second channel position data is extracted and the sheet is loaded, the current channel is identified again, the identified result can represent the displacement change of the current channel, namely the channel position change data, and then the first channel position information is corrected through the channel position change data, so that the accuracy of the next channel loading is realized. It should be understood that, by correcting the first slot position information through the slot position change data, all the first slot position information may be corrected in order to reduce code redundancy, or unloaded first slot position information may be corrected in order to reduce consumption of computing resources, and a person skilled in the art may select the first slot position information as needed, so as to achieve the technical effects of the embodiment. The correction of each loading is equivalent to the iterative correction of the identification of each channel, the error caused by the movement of the foam box is effectively reduced through the correction for a plurality of times, and the loading precision is improved. Similarly, since the embodiment adopts a plurality of different channel matching templates for identification, the computing resource requirement for each identification is reduced, and the precision of each sheet loading can be ensured by combining the iterative correction technology.
Further, the step of locating all channels under the camera to generate the first channel position information comprises the following steps:
acquiring channel matching templates under different imaging deformation conditions; the channel matching template comprises part or all of at least one channel;
acquiring an image under a camera view as first image data;
selecting a plurality of channel matching templates corresponding to the imaging deformation condition of each channel in the first image data as second image data;
and searching the channels in the first image data according to the channels in the second image data, and generating channel position information corresponding to all the channels according to the search result to serve as first channel position information.
Further, searching the channels in the first image data according to the channels in the second image data, and generating channel position information corresponding to all the channels according to the search result as first channel position information includes:
establishing a search area at a channel position corresponding to the channel matching template in the first image data according to the template area of the channel matching template;
searching channels for each search area through a corresponding channel matching template;
carrying out proportional conversion on the searched channel according to the pixel position and the size to generate first channel position information; the first channel position information includes an actual physical position and size of the channel.
Further, establishing a search region in the first image data corresponding to the channel of the channel matching template according to the template region of the channel matching template includes:
importing four corner coordinates of a template area of the channel matching template as first coordinate data; the template area is a quadrilateral area;
acquiring the coordinates of the center point of the template area as second coordinate data according to the first coordinate data;
and establishing the search area according to the preset size of the search area by taking the second coordinate data as the coordinate of the central point.
Further, acquiring the channel matching templates under different imaging deformation conditions comprises:
acquiring an image under a lens view as a sample image;
selecting template regions from the sample image, the template regions each comprising a portion or all of at least one channel;
extracting channel data within the template region and using the channel data and the template region as the channel matching template.
A substrate placement visual guidance system, comprising:
a full channel positioning module configured to position all channels under the camera to generate first channel position information;
a control module configured to extract position information of a current slot from the first slot position information as second slot position data when sheet slot placement is performed; controlling a gripping device to place the sheet into a current channel according to the second channel position data;
the channel position correction module is configured to acquire position data of the current channel as third channel position data after the sheet is placed in the current channel; acquiring channel position change data according to the change of the third channel position data relative to the second channel position data, and correcting the position information of the next channel in the first channel position information according to the channel position change data;
and the control module executes the next sheet material channel placement according to the corrected first channel position information.
Further, the all-channel positioning module is also configured to acquire channel matching templates under different imaging deformation conditions; the channel matching template comprises part or all of at least one channel; acquiring an image under a camera view as first image data; selecting a plurality of channel matching templates corresponding to the imaging deformation condition of each channel in the first image data as second image data; and searching the channels in the first image data according to the channels in the second image data, and generating channel position information corresponding to all the channels according to the search result to serve as first channel position information.
Further, the all-channel locating module is further configured to establish a search region at a channel corresponding to the channel matching template in the first image data according to the template region of the channel matching template; searching channels for each search area through a corresponding channel matching template; carrying out proportional conversion on the searched channel according to the pixel position and the size to generate first channel position information; the first channel position information includes an actual physical position and size of the channel.
Further, the all-channel locating module is further configured to import four corner coordinates of a template region of the channel matching template as first coordinate data; the template area is a quadrilateral area; acquiring the coordinates of the center point of the template area as second coordinate data according to the first coordinate data; and establishing the search area according to the preset size of the search area by taking the second coordinate data as the coordinate of the central point.
Further, the all channel positioning module comprises:
a template image extraction module configured to acquire an image under a lens field of view as a sample image; selecting template regions from the sample image, the template regions each comprising a portion or all of at least one channel; extracting channel data within the template region and using the channel data and the template region as the channel matching template.
Compared with the prior art, the invention has the following advantages and beneficial effects:
according to the visual guiding method and system for sheet placement, disclosed by the invention, by the technology, the influence of lens imaging deformation is reduced, the efficiency of machine visual identification when the sheet is placed is improved, and the balance of placement precision and efficiency is realized.
Drawings
The accompanying drawings, which are included to provide a further understanding of the embodiments of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the principles of the invention. In the drawings:
FIG. 1 is a schematic diagram of the method steps of an embodiment of the present invention;
FIG. 2 is a system architecture diagram according to an embodiment of the present invention;
FIG. 3 is a diagram illustrating the determination of a search area based on a template according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating a step of determining a search area according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of a template region corresponding to a single channel in accordance with an embodiment of the present invention;
FIG. 6 is a schematic diagram of a template region in a sample image according to an embodiment of the present invention;
FIG. 7 is a schematic view of a sheet load according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of a sample image graying according to an embodiment of the present invention;
fig. 9 is a schematic diagram of a sample image graying inversion according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail below with reference to examples and accompanying drawings, and the exemplary embodiments and descriptions thereof are only used for explaining the present invention and are not meant to limit the present invention.
Examples
Referring to fig. 1, a flow chart of a visual sheet placement guiding method according to an embodiment of the present invention is shown, where the visual sheet placement guiding method can be applied to the visual sheet placement guiding system in fig. 2, and further, the visual sheet placement guiding method specifically includes the following steps S1-S6:
s1: positioning all channels under the camera to generate first channel position information;
s2: when sheet channel placement is executed, extracting the position information of the current channel from the first channel position information as second channel position data;
s3: controlling a gripping device to place the sheet into a current channel according to the second channel position data;
s4: after the sheet-shaped object is placed in the current channel, acquiring position data of the current channel as third channel position data;
s5: acquiring channel position change data according to the change of the third channel position data relative to the second channel position data, and correcting the position information of the next channel in the first channel position information according to the channel position change data;
s6: and performing next sheet channel placement according to the corrected first channel position information.
When the method is implemented, in order to save computing resources and improve the recognition efficiency, a channel matching template mode is adopted as a reference of machine vision recognition, and the method has the advantages of being obvious. In practice, the inventor finds that in order to improve the loading efficiency of the sheet materials, the foam box for storing the sheet materials is not fixed generally, which causes the foam box to move when each sheet material is loaded, and needs to be repositioned after each movement of the foam box, and if the full channel is positioned by one template in the prior art, the loading efficiency is reduced. In the embodiment, after the second channel position data is extracted and the sheet is loaded, the current channel is identified again, the identified result can represent the displacement change of the current channel, namely the channel position change data, and then the first channel position information is corrected through the channel position change data, so that the accuracy of the next channel loading is realized. It should be understood that, by correcting the first slot position information through the slot position change data, all the first slot position information may be corrected in order to reduce code redundancy, or unloaded first slot position information may be corrected in order to reduce consumption of computing resources, and a person skilled in the art may select the first slot position information as needed, so as to achieve the technical effects of the embodiment. The correction of each loading is equivalent to the iterative correction of the identification of each channel, the error caused by the movement of the foam box is effectively reduced through the correction for a plurality of times, and the loading precision is improved. Similarly, since the embodiment adopts a plurality of different channel matching templates for identification, the computing resource requirement for each identification is reduced, and the precision of each sheet loading can be ensured by combining the iterative correction technology.
In one embodiment, step S1 includes the following sub-steps:
s11: acquiring channel matching templates under different imaging deformation conditions; the channel matching template comprises part or all of at least one channel;
s12: acquiring an image under a camera view as first image data;
s13: selecting a plurality of channel matching templates corresponding to the imaging deformation condition of each channel in the first image data as second image data;
s14: and searching the channels in the first image data according to the channels in the second image data, and generating channel position information corresponding to all the channels according to the search result to serve as first channel position information.
In this embodiment, the calculation process of searching for the slot in the first image data according to the slot in the second image data may use the existing machine vision identification technologies such as edge identification and corresponding end point identification, which are not repeated herein; similarly, the control of the grasping device through the identified channel position information to realize the placement of the sheet-shaped object can also be realized in the prior art, and the realization means can include the modes existing in the prior art such as coordinate positioning, respective freedom amount and the like, which are not repeated herein. It should be understood that, the imaging deformation condition corresponds to the same or similar imaging deformation condition in the present embodiment, and the channel search in the first image data is performed by selecting the corresponding channel matching template, so that the imaging deformation influence can be greatly reduced. Through the technology, the influence of lens imaging deformation is reduced, the efficiency of machine vision identification when the sheet object is placed is improved, and the balance of placing precision and efficiency is realized.
In a more specific embodiment, the channel data includes first imaging deformation condition data;
step S11 includes the following substeps:
extracting the coordinates of the central point of the template area, and acquiring the distance from the coordinates of the central point to the central point of the sample image as a first reference distance;
and calculating the ratio of the first reference distance to the radius of the sample image as first imaging deformation condition data.
In practice, the inventor finds that the deformation of the lens imaging generally occurs mainly at the edge of the lens, and the change of the deformation does not occur linearly along the edge, so the inventor adopts the ratio as the first imaging deformation data to perform the targeted positioning of the lens imaging deformation for the subsequent deformation matching. Although machine vision detection, such as deformation curvature and the like, can be performed on deformation conditions of the channel under the lens, considering the problem of efficiency of loading the sheet, detection of imaging deformation conditions again through machine vision is not beneficial to rapid loading, so indirect detection of imaging deformation conditions through positions is adopted in the embodiment, and recognition efficiency is improved.
In a more specific embodiment, step S13 includes:
acquiring center point coordinates of channels in the first image data, and acquiring a distance from the center point coordinates to a center point of the first image data as a second reference distance;
calculating the ratio of the second reference distance to the radius of the first image data as second imaging deformation condition data;
and acquiring a channel matching template corresponding to the first imaging deformation condition data and the second imaging deformation condition data of the channel as second image data.
In one embodiment, step S14 includes:
establishing a search area at a channel position corresponding to the channel matching template in the first image data according to the template area of the channel matching template;
searching channels for each search area through a corresponding channel matching template;
carrying out proportional conversion on the searched channel according to the pixel position and the size to generate first channel position information; the first channel position information includes an actual physical position and size of the channel.
In the embodiment, in order to improve the accuracy of machine vision through template identification, a search area is established at the corresponding channel of the first image data based on the template area of the channel matching template, so that in subsequent channel identification, elements needing to be searched can be reduced, and the search efficiency is improved. The actual physical position and size can likewise be obtained by scaling the channels, it being understood that the actual physical position and size can be applied directly to the control gripping device. The grabbing device mentioned in this embodiment may be a device for grabbing materials, such as a manipulator, a sucker gripper, and the like, which are mature in the prior art, and this embodiment is not limited herein.
In one embodiment, establishing a search region at a channel in the first image data corresponding to the channel matching template based on the template region of the channel matching template comprises:
importing four corner coordinates of a template area of the channel matching template as first coordinate data; the template area is a quadrilateral area;
acquiring the coordinates of the center point of the template area as second coordinate data according to the first coordinate data;
and establishing the search area according to the preset size of the search area by taking the second coordinate data as the coordinate of the central point.
The true bookIn example embodiments, referring to fig. 3 and 4, fig. 3 illustrates a specific example of establishing a search area at a channel based on the channel matching template, and fig. 4 illustrates specific steps of establishing a search area at a channel based on the channel matching template; wherein R isiMatching the template region of the template, PS, for the channel corresponding to channel iiFor the search area corresponding to the channel i, when the second coordinate data is acquired from the first coordinate data, R is determined using the determination shown in fig. 4iCenter of area OiThe manner of location; when the search area is established according to the preset size of the search area, at least one of parameters which can represent the size of the search area, such as the width, the height, the diagonal length and the like of the search area, can be preset; for example, a search area width and a height of twice may be used, and the preset search area width is wiHeight of hiThe preset search area width is 2wiHeight of 2hi(ii) a Under the condition that the size of a preset search area and the coordinates of a center point are determined, the search area can be established, and elements required by image machine vision recognition are reduced.
In one embodiment, acquiring channel matching templates for different imaging deformations includes:
acquiring an image under a lens view as a sample image;
selecting template regions from the sample image, the template regions each comprising a portion or all of at least one channel;
extracting channel data within the template region and using the channel data and the template region as the channel matching template.
In the implementation of this embodiment, the channel matching template is obtained through the template region, wherein the template region needs to include a portion that can be used for channel identification, and a complete channel matching template can be formed by extracting channel data from the template region. It should be understood that the channel data includes data relating to channel keypoint coordinates, current template region imaging deformation, and the like.
By way of example, referring to FIG. 6, four channels are shownThe template region of the image under the lens view is R1、R2、R3And R4Wherein the template region R1Four corners of the corresponding template area are A1、B1、C1And D1Similarly, the template region R2、R3And R4The angle of (c) is also shown. Referring to fig. 5, coordinates of four corners of the template region are also extracted for coordinate matching at the time of recognition.
For example, referring to fig. 8 for the sample image, the inventor finds in practice that in such a grayed picture, since the channels are not clear enough to facilitate the positioning of the channels, and the positioning accuracy of the channels in the sample image directly affects the loading accuracy in the subsequent recognition loading, the image processing is performed in the manner of inverting the grayscale in this example, and the processed image refers to fig. 9.
Based on the same inventive concept, a sheet placement visual guidance system is also provided, and for convenience of explanation of the above sheet placement visual guidance system, please refer to fig. 2, a schematic diagram of a communication architecture of the sheet placement visual guidance system disclosed in the embodiment of the present invention is provided, including:
a full channel positioning module configured to position all channels under the camera to generate first channel position information;
a control module configured to extract position information of a current slot from the first slot position information as second slot position data when sheet slot placement is performed; controlling a gripping device to place the sheet into a current channel according to the second channel position data;
the channel position correction module is configured to acquire position data of the current channel as third channel position data after the sheet is placed in the current channel; acquiring channel position change data according to the change of the third channel position data relative to the second channel position data, and correcting the position information of the next channel in the first channel position information according to the channel position change data;
and the control module executes the next sheet material channel placement according to the corrected first channel position information.
In one embodiment, the all-channel localization module is further configured to obtain channel matching templates under different imaging deformations; the channel matching template comprises part or all of at least one channel; acquiring an image under a camera view as first image data; selecting a plurality of channel matching templates corresponding to the imaging deformation condition of each channel in the first image data as second image data; and searching the channels in the first image data according to the channels in the second image data, and generating channel position information corresponding to all the channels according to the search result to serve as first channel position information.
In one embodiment, the all-channels locating module is further configured to establish a search region at a channel in the first image data corresponding to the channel matching template according to a template region of the channel matching template; searching channels for each search area through a corresponding channel matching template; carrying out proportional conversion on the searched channel according to the pixel position and the size to generate first channel position information; the first channel position information includes an actual physical position and size of the channel.
In one embodiment, the all-channel localization module is further configured to import four corner coordinates of a template region of the channel matching template as first coordinate data; the template area is a quadrilateral area; acquiring the coordinates of the center point of the template area as second coordinate data according to the first coordinate data; and establishing the search area according to the preset size of the search area by taking the second coordinate data as the coordinate of the central point.
In one embodiment, the all channel positioning module comprises:
a template image extraction module configured to acquire an image under a lens field of view as a sample image; selecting template regions from the sample image, the template regions each comprising a portion or all of at least one channel; extracting channel data within the template region and using the channel data and the template region as the channel matching template.
It should be understood that the manner in which objects are placed into channels achieved by the above-described embodiments of the present application should be considered equivalent to the present application, and the flaps themselves may be equivalent to other objects that need to be placed into channels; and the shape of the channel, whatever the way it opens, should be considered equivalent to the channel of the present application.
By way of example, a glass sheet or a liquid crystal panel may be placed into the foam box channel by the embodiments described above in this application; for example, the electronic component may be assembled into a preset position through the above embodiments of the present application; for example, the battery may be placed or mounted in a predetermined area through the above-described embodiments of the present application; for example, operations such as installation and replacement of the new energy automobile battery can be performed through the above-described embodiments of the present application. The above examples should all be considered within the scope of protection of the present application.
Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or combinations of both, and that the components and steps of the examples have been described in a functional general in the foregoing description for the purpose of illustrating clearly the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may also be an electric, mechanical or other form of connection.
The elements described as separate parts may or may not be physically separate, as one of ordinary skill in the art would appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or combinations of both, and that the components and steps of the examples have been described in a functional general sense in the foregoing description for clarity of explanation of the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention essentially or partially contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a grid device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are merely exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (10)
1. A sheet placement visual guidance method characterized by comprising:
positioning all channels under the camera to generate first channel position information;
when sheet channel placement is executed, extracting the position information of the current channel from the first channel position information as second channel position data;
controlling a gripping device to place the sheet into a current channel according to the second channel position data;
after the sheet-shaped object is placed in the current channel, acquiring position data of the current channel as third channel position data;
acquiring channel position change data according to the change of the third channel position data relative to the second channel position data, and correcting the position information of the next channel in the first channel position information according to the channel position change data;
and performing next sheet channel placement according to the corrected first channel position information.
2. The visual guidance method for sheet placement according to claim 1, wherein the step of locating all of the channels under the camera to generate first channel position information comprises the steps of:
acquiring channel matching templates under different imaging deformation conditions; the channel matching template comprises part or all of at least one channel;
acquiring an image under a camera view as first image data;
selecting a plurality of channel matching templates corresponding to the imaging deformation condition of each channel in the first image data as second image data;
and searching the channels in the first image data according to the channels in the second image data, and generating channel position information corresponding to all the channels according to the search result to serve as first channel position information.
3. The visual guidance method for sheet placement according to claim 2, wherein searching for a channel in the first image data based on a channel in the second image data, and generating channel position information corresponding to all channels as first channel position information based on the search result includes:
establishing a search area at a channel position corresponding to the channel matching template in the first image data according to the template area of the channel matching template;
searching channels for each search area through a corresponding channel matching template;
carrying out proportional conversion on the searched channel according to the pixel position and the size to generate first channel position information; the first channel position information includes an actual physical position and size of the channel.
4. The visual guidance method for sheet placement according to claim 3, wherein establishing a search area in the first image data at a channel corresponding to the channel matching template from the template area of the channel matching template comprises:
importing four corner coordinates of a template area of the channel matching template as first coordinate data; the template area is a quadrilateral area;
acquiring the coordinates of the center point of the template area as second coordinate data according to the first coordinate data;
and establishing the search area according to the preset size of the search area by taking the second coordinate data as the coordinate of the central point.
5. The visual guidance method for sheet placement according to claim 2, wherein obtaining a channel matching template for different imaging deformations includes:
acquiring a sample image under a lens view;
selecting template regions from the sample image, the template regions each comprising a portion or all of at least one channel;
extracting channel data within the template region and using the channel data and the template region as the channel matching template.
6. A substrate placement visual guidance system, comprising:
a full channel positioning module configured to position all channels under the camera to generate first channel position information;
a control module configured to extract position information of a current slot from the first slot position information as second slot position data when sheet slot placement is performed; controlling a gripping device to place the sheet into a current channel according to the second channel position data;
the channel position correction module is configured to acquire position data of the current channel as third channel position data after the sheet is placed in the current channel; acquiring channel position change data according to the change of the third channel position data relative to the second channel position data, and correcting the position information of the next channel in the first channel position information according to the channel position change data;
and the control module executes the next sheet material channel placement according to the corrected first channel position information.
7. The substrate placement visual guidance system of claim 6, wherein the all channel positioning module is further configured to obtain channel matching templates for different imaging deformations; the channel matching template comprises part or all of at least one channel; acquiring an image under a camera view as first image data; selecting a plurality of channel matching templates corresponding to the imaging deformation condition of each channel in the first image data as second image data; and searching the channels in the first image data according to the channels in the second image data, and generating channel position information corresponding to all the channels according to the search result to serve as first channel position information.
8. The substrate placement visual guidance system of claim 7, wherein the all-channel localization module is further configured to establish a search area in the first image data at a channel corresponding to the channel matching template according to a template area of the channel matching template; searching channels for each search area through a corresponding channel matching template; carrying out proportional conversion on the searched channel according to the pixel position and the size to generate first channel position information; the first channel position information includes an actual physical position and size of the channel.
9. The substrate placement visual guidance system of claim 8, wherein the all-channel localization module is further configured to import four corner coordinates of a template region of the channel matching template as first coordinate data; the template area is a quadrilateral area; acquiring the coordinates of the center point of the template area as second coordinate data according to the first coordinate data; and establishing the search area according to the preset size of the search area by taking the second coordinate data as the coordinate of the central point.
10. The substrate placement visual guide system of claim 7, wherein the all channel positioning module comprises:
a template image extraction module configured to acquire a sample image under a lens field of view; selecting template regions from the sample image, the template regions each comprising a portion or all of at least one channel; extracting channel data within the template region and using the channel data and the template region as the channel matching template.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111127301.7A CN113562465B (en) | 2021-09-26 | 2021-09-26 | Visual guiding method and system for sheet placement |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111127301.7A CN113562465B (en) | 2021-09-26 | 2021-09-26 | Visual guiding method and system for sheet placement |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113562465A CN113562465A (en) | 2021-10-29 |
| CN113562465B true CN113562465B (en) | 2021-12-03 |
Family
ID=78174563
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111127301.7A Active CN113562465B (en) | 2021-09-26 | 2021-09-26 | Visual guiding method and system for sheet placement |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113562465B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109146865A (en) * | 2018-08-22 | 2019-01-04 | 成都新西旺自动化科技有限公司 | A kind of visual-alignment detection figure source generates system |
| CN110120366A (en) * | 2018-02-06 | 2019-08-13 | 中芯国际集成电路制造(上海)有限公司 | Semiconductor devices and forming method thereof |
| CN113269723A (en) * | 2021-04-25 | 2021-08-17 | 浙江省机电设计研究院有限公司 | Unordered grasping system for three-dimensional visual positioning and mechanical arm cooperative work parts |
Family Cites Families (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5370216A (en) * | 1993-03-05 | 1994-12-06 | Shibuya Kogyo Co., Ltd. | Apparatus for aligning vessels |
| US6173750B1 (en) * | 1998-02-18 | 2001-01-16 | Hover-Davis, Inc. | Method and apparatus for removing die from a wafer and conveying die to a pickup location |
| JP4976756B2 (en) * | 2006-06-23 | 2012-07-18 | キヤノン株式会社 | Information processing method and apparatus |
| JP6517666B2 (en) * | 2015-11-02 | 2019-05-22 | 株式会社東芝 | Article management device, method thereof, and program thereof |
| US10289930B2 (en) * | 2017-02-09 | 2019-05-14 | Glasstech, Inc. | System and associated for online measurement of the optical characteristics of a glass sheet |
| CN107798677B (en) * | 2017-10-31 | 2021-03-05 | 南京林业大学 | Bracket soldering paste visual detection method based on MATLAB image processing |
| US10867275B1 (en) * | 2018-09-17 | 2020-12-15 | Amazon Technologies, Inc. | Optimized package loading |
| US10987744B2 (en) * | 2019-02-22 | 2021-04-27 | Kendall Keith Gill | Apparatus and methods for automatically training saw blades on a saw mandrel |
| CN110443093B (en) * | 2019-07-31 | 2022-07-29 | 安徽大学 | Intelligent-oriented infrared digital panoramic system and warehouse management method thereof |
| WO2021070188A1 (en) * | 2019-10-11 | 2021-04-15 | Beyeonics Surgical Ltd. | System and method for improved electronic assisted medical procedures |
| CN111307828B (en) * | 2020-03-13 | 2022-12-13 | 安徽工程大学 | Equipment for detecting defects of intelligent household control panel and detection method thereof |
| CN111815634A (en) * | 2020-09-09 | 2020-10-23 | 苏州浪潮智能科技有限公司 | Machine vision-based memory alignment plug-in method, system, equipment and storage medium |
| CN112132109A (en) * | 2020-10-10 | 2020-12-25 | 北京百度网讯科技有限公司 | Lane line processing and lane positioning method, device, equipment and storage medium |
| CN113160143B (en) * | 2021-03-23 | 2022-05-24 | 中南大学 | Method and system for measuring material liquid level in material stirring tank |
| CN112785648B (en) * | 2021-04-12 | 2021-07-06 | 成都新西旺自动化科技有限公司 | Visual alignment method, device and equipment based on to-be-imaged area and storage medium |
-
2021
- 2021-09-26 CN CN202111127301.7A patent/CN113562465B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110120366A (en) * | 2018-02-06 | 2019-08-13 | 中芯国际集成电路制造(上海)有限公司 | Semiconductor devices and forming method thereof |
| CN109146865A (en) * | 2018-08-22 | 2019-01-04 | 成都新西旺自动化科技有限公司 | A kind of visual-alignment detection figure source generates system |
| CN113269723A (en) * | 2021-04-25 | 2021-08-17 | 浙江省机电设计研究院有限公司 | Unordered grasping system for three-dimensional visual positioning and mechanical arm cooperative work parts |
Non-Patent Citations (1)
| Title |
|---|
| 高韬等.背景渐变的视频对象分割算法研究及实现.《电视技术》.2006,(第07期), * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113562465A (en) | 2021-10-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9987747B2 (en) | Stocker for receiving cassettes and method of teaching a stocker robot disposed therein | |
| CN110076029B (en) | Glue spraying control method, system, computer equipment and computer storage medium | |
| US20140088765A1 (en) | Method for invalidating sensor measurements after a picking action in a robot system | |
| JP2019509559A (en) | Box location, separation, and picking using a sensor-guided robot | |
| CN109052180B (en) | Automatic container alignment method and system based on machine vision | |
| EP4155234A1 (en) | Control method and apparatus for warehouse robot, and robot and warehouse system | |
| CN112200058B (en) | System and method for intelligently correcting auxiliary data | |
| CN113526125B (en) | Cell specimen sample carrying system and method based on multi-label positioning | |
| CN114474056B (en) | Monocular vision high-precision target positioning method for grabbing operation | |
| CN114751153B (en) | Full-angle multi-template stacking system | |
| CN110519508B (en) | Method for identifying characteristic object in automatic aligning process of camera module | |
| CN113562465B (en) | Visual guiding method and system for sheet placement | |
| CN112257607B (en) | Correction method for processing mobile phone image distortion acquired on production line | |
| CN113172636B (en) | Automatic hand-eye calibration method and device and storage medium | |
| CN112927190A (en) | Type-C multi-station multi-defect unified measurement system based on deep learning | |
| JPH04244391A (en) | Step disordering device using robot | |
| JPH11175150A (en) | Stop position deviation amount detecting device for moving body | |
| US11845191B1 (en) | Robotic picking of cuboidal items from a pallet | |
| CN110969661A (en) | Image processing device and method, position calibration system and method | |
| CN110060330B (en) | Three-dimensional modeling method and device based on point cloud image and robot | |
| CN113837950A (en) | Crystal fillet degree correction method, system and storage medium | |
| CN111447366B (en) | Transportation method, transportation device, electronic device, and computer-readable storage medium | |
| CN111410045A (en) | Container handling method and device | |
| CN117711994B (en) | Wafer monolithic transportation control method and system based on link logic diagram | |
| CN114897997B (en) | Camera calibration method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |