KR20240025248A - A method of teaching a screw assembly location based on a deep learning automatically, an apparatus of teaching a screw assembly location based on a deep learning automatically, and medium of storitng a program teaching a screw assembly location based on a deep learning automatically - Google Patents

Abstract

The deep learning-based screw fastening position automation teaching method according to embodiments includes the steps of acquiring an image; performing preprocessing on the image; Browsing images; Extracting the screw fastening area from the image; Performing preprocessing on the extracted image; Extracting the hole area from the screw fastening area; and storing coordinates for the hole area; may include.

Description

Deep learning-based screw tightening position automation teaching method, deep learning-based screw tightening position automation teaching device, deep learning-based screw tightening position automation teaching program {A METHOD OF TEACHING A SCREW ASSEMBLY LOCATION BASED ON A DEEP LEARNING AUTOMATICALLY, AN APPARATUS OF TEACHING A SCREW ASSEMBLY LOCATION BASED ON A DEEP LEARNING AUTOMATICALLY, AND MEDIUM OF STORITNG A PROGRAM TEACHING A SCREW ASSEMBLY LOCATION BASED ON A DEEP LEARNING AUTOMATICALLY}

These embodiments relate to a deep learning-based screw fastening position automation teaching method, a deep learning-based screw fastening position automation teaching device, and a storage medium that stores a deep learning-based screw fastening position automation teaching program.

In the case of the existing screw fastening position adjustment technology, the person in charge visually checks the fastening area of the product and numerically adjusts the position of the screw fastening equipment to move it. Due to the nature of the product, repeated position adjustments are required for each fastening position point on multiple surfaces. Theoretical and technical considerations are required to construct a deep learning-based automatic screw fastening position teaching system that minimizes the repetitive work of adjusting the position of each fastening area performed by field personnel and provides guidance on the fastening area position for new products. do.

In the case of the current fastening position calculation method, the person in charge visually determines the fastening surface of the product, calculates the fastening points existing on the fastening surface, and manually enters the coordinates with the fastener and moves them. It is necessary for the person in charge to calculate the fastening surface and individually distinguish the fastening points for one surface. Since the fastening location is different for each product, field managers face difficulties in the process of changing the location for each product. In addition, since there is a fastener on each fastening surface of the product, the position of the fastener needs to be adjusted as it moves for each process, which is a problem and limitation in that additional man-hours are incurred.

Hereinafter, the purpose of the disclosed embodiments is to provide a deep learning-based screw fastening position automation teaching method, a deep learning-based screw fastening position automation teaching device, and a deep learning-based screw fastening position automation teaching program to solve the above-mentioned problems and limitations. It provides a storage medium for storing data.

Furthermore, the purpose of the embodiments is to store a deep learning-based screw fastening position automation teaching method, a deep learning-based screw fastening position automation teaching device, and a deep learning-based screw fastening position automation teaching program including a method for classifying the fastening area of the product and image acquisition. It provides a storage medium that

Furthermore, the purpose of the embodiments is a deep learning-based screw fastening position automation teaching method, a deep learning-based screw fastening position automation teaching device, and a deep learning-based screw fastening position automation including a prediction algorithm for confirming the screw fastening point of the fastening surface. It provides a storage medium for storing teaching programs.

Furthermore, the purpose of the embodiments is a deep learning-based screw fastening position automation teaching method, a deep learning-based screw fastening position automation teaching device, and a deep learning-based screw fastening position automation including a method of configuring a fastening system operated with predicted fastening coordinates. It provides a storage medium for storing teaching programs.

Hereinafter, in order to solve the above-described problem, a deep learning-based screw fastening position automation teaching method according to embodiments includes the steps of acquiring an image; performing preprocessing on the image; Browsing images; Extracting the screw fastening area from the image; Performing preprocessing on the extracted image; Extracting the hole area from the screw fastening area; and storing coordinates for the hole area; may include.

According to embodiments, fastening points for each fastening area of a product can be predicted, and work man-hours can be minimized by moving fastening equipment to the predicted fastening point. Additionally, it is possible to configure an automated system that can respond to new products with changed fastening positions.

Embodiments (1) classify the fastening area of the product and acquire an image, (2) predict the screw fastening point of the fastening surface using an algorithm based on the image, and (3) convert it to the predicted fastening coordinates to determine how each fastening equipment operates. The system can be configured.

Embodiments can achieve the effect of reducing man-hours by minimizing the repetitive work of manually setting fastening points by field personnel. It is possible to respond to changing fastening areas by acquiring images of the fastening areas of classified products and predicting the fastening point locations using a deep learning algorithm. By transmitting predicted fastening coordinates from one image acquisition process to multiple processes for fastening each side, the positioning work of each fastening equipment is simplified. The fastening point is predicted through additional learning of the algorithm using the new fastening area and point image, enabling expansion in the fastening process of similar products.

The drawings are included to further understand the embodiments, and the drawings represent the embodiments along with descriptions related to the embodiments. For a better understanding of the various embodiments described below, reference should be made to the following description of the embodiments in conjunction with the following drawings, in which like reference numerals refer to corresponding parts throughout the drawings.
Figure 1 shows a method for acquiring images and predicting screw fastening positions according to embodiments.
Figure 2 shows a method for calculating screw fastening hole coordinates according to embodiments.
Figure 3 shows a deep learning algorithm learning method according to embodiments.
Figure 4 shows a screw hole prediction method according to embodiments.
Figure 5 shows a method for automatically adjusting the position of fastening equipment according to embodiments.
Figure 6 shows a method of transmitting coordinates for an image acquisition process and a fastening process according to embodiments.
Figure 7 shows a screw fastening position automated teaching device according to embodiments.

Preferred embodiments of the embodiments will be described in detail, examples of which are shown in the attached drawings. The detailed description below with reference to the accompanying drawings is intended to explain preferred embodiments of the embodiments rather than showing only embodiments that can be implemented according to the embodiments. The following detailed description includes details to provide a thorough understanding of the embodiments. However, it will be apparent to those skilled in the art that embodiments may be practiced without these details.

Most of the terms used in the embodiments are selected from common ones widely used in the field, but some terms are arbitrarily selected by the applicant and their meaning is detailed in the following description as necessary. Accordingly, the embodiments should be understood based on the intended meaning of the terms rather than their mere names or meanings.

In this document, a deep learning-based screw fastening position automation teaching method, a deep learning-based screw fastening position automation teaching device, and a storage medium for storing a deep learning-based screw fastening position automation teaching program will be abbreviated as a method/device according to embodiments. You can.

The deep learning-based screw fastening position automatic teaching method and device according to embodiments is a deep learning-based screw fastening position automatic teaching method/system for predicting fastening points existing on the fastening surface and minimizing the user's man-hours for moving fastening equipment. Method may be included.

For example, (1) it includes a deep learning-based algorithm that acquires images by dividing the fastening area of the product and predicts the screw fastening point, and (2) predicts multiple fastening processes from one image acquisition process. It may include an operation of transmitting fastening coordinates to adjust the positions of each fastening equipment.

Figure 1 shows a method for acquiring images and predicting screw fastening positions according to embodiments.

Figure 1 shows a deep learning-based automated teaching method for screw fastening positions according to embodiments.

Figure 1 shows a process for predicting the location of a screw fastening point by acquiring an image.

The method/device according to embodiments divides the fastening area of the product, optimizes the camera and lighting environment for each area, and stores the image of the fastening area. The saved image is passed to the algorithm to recognize the screw hole that requires fastening and convert the area into coordinates.

The deep learning-based screw fastening position automation teaching method according to embodiments may include each step as shown in FIG. 1.

Step 1000, aligning the camera and product positions may align the positions of the cameras to obtain information about the area where the screw fastening point is located. Additionally, the positions of products can be aligned. The position of the product and/or the camera may be adjusted so that the position between the area of the product where the fastening point is located and the area captured by the camera of the device according to the embodiments are aligned. The position alignment step can be performed automatically. Additionally, the position alignment step can be controlled by the user. The screw fastening position automation teaching device according to embodiments may include a camera.

1001, the step of controlling lighting (brightness) may emit lighting in order to obtain information about the area where the screw fastening point is located. The screw fastening position automation teaching device according to embodiments may include a lighting emitting unit. By adjusting the brightness of the lighting, it is possible to obtain a shaded area created in the area relative to the location of the screw fastening point. The screw fastening position can be automatically taught based on the shape of the shadow and/or shadow caused by lighting and brightness. There is a hole in the screw fastening part, and when light hits the hole and the area around the hole, a shaded area may be created. The location of the hole can be estimated from shadow information.

1002, the step of photographing a product image (all areas) may capture (photograph) an area related to the location of the screw fastening point of the product. Image data about the point area can be generated. Video data may be called variously, such as image data. Video data or product images can cover any area.

Step 1003, performing the algorithm, may perform an operation for automated teaching of the screw fastening position. Specific algorithms according to embodiments will be described with reference to the drawings below.

In step 1004, storing and displaying coordinates, the generated coordinates can be stored in memory and coordinate data can be displayed.

Figure 2 shows a method for calculating screw fastening hole coordinates according to embodiments.

Figure 2 shows in more detail the process of calculating screw fastening hole coordinates in the step of storing and marking the coordinates shown in Figure 1.

An image photographed (captured) as shown in FIG. 1 may be input as input data to an automatic teaching device for screw fastening position according to embodiments. After input, image preprocessing such as resize and/or gray scale can be performed to increase the performance of the algorithm.

When taking an image, when a strong light is shined on the product in a dark room environment, the parts excluding the screw hole are displayed relatively brightly, and the screw hole has the characteristic of being darker than the surrounding area. After lighting, you can capture images (videos) with a camera using these characteristics. In order to detect screw holes (objects), a large amount of images with various products and shooting conditions can be acquired, and the scroll hole location can be calculated using an algorithm (e.g., CNN). As shown in Figure 2, the method/device according to embodiments can calculate screw fastening hole coordinates.

2000, a method/device according to embodiments may receive a product image as input data. Receive product images captured by a camera.

2001, a method/apparatus according to embodiments may perform a pre-processing process for a product image. For example, preprocessing can adjust the size of the image or apply gray scale to the image.

2002, a method/apparatus according to embodiments may scan an image to detect a screw fastening area. To determine whether a product image includes a screw fastening area, bright and dark features can be detected depending on the object's structure, location, shooting environment, etc.

2003, a method/apparatus according to embodiments can extract a screw fastening area. By comparing and learning multiple images, the characteristics or rules of the screw fastening area can be identified, and the screw fastening area can be extracted based on an algorithm such as deep learning.

2004, a method/device according to embodiments may perform preprocessing on an extracted image. For example, based on a threshold or threshold, extracted images that satisfy a certain value or range can be selected. The accuracy of screw position extraction can be improved.

2005, a method/apparatus according to embodiments may re-extract the screw hole area from the extracted image.

2006, a method/device according to embodiments can store and display pixel coordinates for the extracted hole area.

Figure 3 shows a deep learning algorithm learning method according to embodiments.

Figure 3 shows the process of calculating the screw hole area position through the calculated screw fastening hole coordinates in Figure 2.

The location of the screw hole is converted from pixel coordinates into a box format to learn labeled coordinates and images. Figure 3 shows this learning process. The conditions for acquiring various images are 1) variable conditions for various product groups with different screw fastening positions, 2) conditions for variable brightness of the image taken by changing the lighting value, and 3) conditions for variable deviation through changing the jig that fixes the product. Various types of images can be acquired from the product. The acquired image is changed into an image suitable for learning through resize and grayscale, and in order for the algorithm to distinguish between the area where the screw hole exists and the area where the screw hole does not exist, the area where the screw hole exists is divided into a square box. Label with pixel coordinates. In the case of the CNN algorithm, learning is performed while maintaining the spatial information of the image, and a feature map can be calculated through a convolution filter. For example, the screw hole area may be characterized as being a darker area than the surrounding area in the image. A features map can be calculated based on the characteristics of these screw hole areas. Based on the features map, the screw hole area is converted to coordinates, and accurate location prediction is possible through learning a large amount of images.

After inputting the entire image of the product, the location of the screw hole area is displayed in the form of a square box through an algorithm, but preprocessing can be re-processed by cropping the image in the form of a square box to increase the accuracy of the center position of the screw hole. there is.

For example, the cropped image can be processed through threshold to distinguish the screw hole and the surrounding area, and a box-shaped cropped image can be re-extracted based on the outermost black pixel information of the screw hole. . At this time, the coordinates of the pixel are stored and displayed. This is to ensure the exact location of the screw hole.

The deep learning algorithm learning process is essential to increase the prediction rate of the algorithm and respond to similar products. Through additional learning from image data, meaningful decisions can be made, and based on this, the optimal algorithm can be updated to accurately predict.

3000, a method/device according to embodiments may receive a product image and screw hole area coordinates. Product image and screw hole area coordinates can be obtained through the flow chart as shown in Figure 2.

3001, a method/apparatus according to embodiments may perform image pre-processing. May include preprocessing such as resizing and/or gray scale of the image.

3002, a method/apparatus according to embodiments may perform a CNN algorithm. Images can be learned through the CNN algorithm.

3003, a method/apparatus according to embodiments may extract a feature map. When a CNN algorithm is applied to an image, the features of the image can be extracted and map data for these features can be generated. For example, an image may have bright areas due to lighting and dark areas due to the depth or shape of the hole. These regular features can be extracted.

3004, a method/apparatus according to embodiments may calculate a screw hole area location. The location of the screw hole area can be estimated from the image through the feature map. The accuracy of location estimation can be increased through CNN algorithms and images.

3005, the method/apparatus according to embodiments may store the coordinates of the screw hole area location.

Figure 4 shows a screw hole prediction method according to embodiments.

Figure 4 shows the coordinate prediction process of the screw hole on the product image described in Figures 2 and 3. Figure 4 illustrates the product image acquisition step, image search step, region extraction step, image pre-processing step, region re-extraction step, and coordinate storage step included in Figures 2 and 3 as examples.

Product image 4000 may be a product image including the entire area captured by a camera. Product image 4000 may include one or more screw hole areas 4001.

You can search the box area 4002 for the screw hole area through image search. Box area 4002 may be a darker area than surrounding areas. Box area 4002 can be explored through shading characteristics.

Through region extraction, region 4003 including the screw hole can be extracted from product image 4000. Area 4003 includes a screw hole, but may also include an area surrounding the screw hole rather than the exact location of the screw hole.

Through image preprocessing, area 4003 can be converted to area 4004. As described above, area image 4004 can be generated through resizing and/or grayscale so that the screw hole area can be more accurately distinguished.

Through region re-extraction, region 4005, which is more precise than region 4004, can be extracted. Area 4005 may include more elaborate screw holes than areas 4003 to 4004. Unnecessary areas may be cropped.

Through the coordinate storage step, coordinate information on the X and Y axes in the form of a rectangular box for area 4005 can be calculated. For area 4005, location information for the area including the screw hole can be generated based on the minimum and maximum coordinates on the X-axis and the minimum and maximum coordinates on the Y-axis.

Figure 5 shows a method for automatically adjusting the position of fastening equipment according to embodiments.

Figure 5 shows an automated system that adjusts the position of each fastening equipment by transmitting coordinates from one process to multiple fastening processes, which is performed by a method/device according to embodiments. For example, the method/device according to the embodiments changes the pixel coordinates of the stored rectangular box into length coordinates by comparing them with the origin of the image acquisition process based on the center coordinates after the algorithm operation as shown in FIG. 3. The standard for converting length coordinates is to use the upper left corner of the camera that captures each fastening area as the origin, convert the actual length per pixel for each camera area, and convert and store how far the product is located relative to the origin.

The length coordinates transmitted through PC-to-PC communication are corrected based on the left origin of each fastening process and transmitted to the PLC and stored as fastener coordinates. Each fastening equipment moves in three axes. The field manager minimizes movement work on the X and Y axes based on the changed fastener coordinates, and saves them as the final coordinates after checking the Z axis. Additionally, coordinate storage and equipment movement control can be performed automatically.

5000, a method/device according to embodiments may receive pixel coordinates. Pixel coordinates can be obtained by the same method as shown in Figures 1 to 3.

5001, the method/apparatus according to the embodiments may confirm whether the device (equipment) according to the embodiments is at the origin of the image acquisition process. If the position is not the origin, the position can be adjusted. Position adjustment may be performed by personnel or the system.

5002, a method/device according to embodiments may convert length coordinates. Coordinate values can be converted based on the origin.

5003, a method/device according to embodiments may transmit each fastening process coordinate. Fastening process coordinate transfer can be performed by PC communication.

5004, the method/apparatus according to the embodiments can input the fastening process product.

5005, a method/device according to embodiments can confirm whether the product is at the fastener origin. If it is not the origin, the position can be adjusted. Position adjustment may be performed by personnel or the system.

5006, a method/device according to embodiments can transform fastener coordinates. Coordinate values can be converted based on the origin.

5007, a method/apparatus according to embodiments can move fastening equipment. The movement of the equipment can be along the X-axis and/or Y-axis.

5008, the method/device according to embodiments can inspect the Z-axis after moving the X-axis and Y-axis.

5009, the method/device according to embodiments can check whether the fastening position is by checking the Z axis. If the position is not suitable, it can be readjusted.

5010, a method/device according to embodiments may store final coordinates.

Figure 6 shows a method of transmitting coordinates for an image acquisition process and a fastening process according to embodiments.

Figure 6 shows a method of transferring from one process to multiple fastening processes.

Figure 6 shows a process in which a method/device according to embodiments efficiently processes multiple fastening processes based on the operations shown in Figures 1 to 5.

The image acquisition process refers to the process of acquiring an image of the fastening position of the product. At least one fastening surface of the product may include at least one screw fastening position. At least one camera of the device according to embodiments may capture images of all fastening surfaces. For example, if there are three fastening surfaces, methods according to embodiments may include fastening process A, fastening B process, and fastening C process.

The fastening A process can obtain the fastening position area from the image of the screw fastening position A as described above, and generate and transform coordinates from the fastening position area. And, the fastening position can be adjusted with respect to the coordinates.

Likewise, the fastening B process can obtain the fastening position area from the image for the screw fastening position B as described above, and generate and transform coordinates from the fastening position area. And, the fastening position can be adjusted with respect to the coordinates.

Likewise, the fastening C process can obtain the fastening position area from the image for the screw fastening position C as described above, and generate and transform coordinates from the fastening position area. And, the fastening position can be adjusted with respect to the coordinates.

Figure 7 shows a screw fastening position automated teaching device according to embodiments.

Figure 7 shows a deep learning-based screw fastening position teaching system (or device, 7000) according to embodiments that performs the method, operation, and/or flowchart of Figures 1 to 6.

The interface unit 7001 allows the device to transmit and receive signals, data, and/or information. The device may further include a camera and/or lighting unit. Image data acquired by the camera may be received by the interface unit and transmitted to a processor, memory, etc.

Processor 7002 may perform the operations, flowcharts, and methods of FIGS. 1 to 6 and control related operations.

The memory 7003 can store and provide information, data, etc. generated and transmitted and received in the operations, flowcharts, and methods of FIGS. 1 to 6.

Referring to Figures 1 and 2, the deep learning-based screw fastening position automation teaching method according to embodiments includes the steps of acquiring an image; performing preprocessing on the image; Browsing images; Extracting the screw fastening area from the image; Performing preprocessing on the extracted image; Extracting the hole area from the screw fastening area; and storing coordinates for the hole area; may include.

Referring to Figure 3, with respect to deep learning algorithms according to embodiments, methods according to embodiments include performing preprocessing on an image based on coordinates for a hole area; extracting a feature map for the image; Calculating a screw hole area location based on the feature map; and storing coordinates for the screw hole area location; may further include.

Referring to Figure 4, the step of searching images according to embodiments includes searching for the screw fastening hole area included in the image, extracting the screw fastening hole area, and determining the size or color of the screw fastening hole area (grayscale: (the color for the dark area can be adjusted to make it more visible, such as by inverting the image color), extract the screw fastening hole area in which at least one of the size or color has been adjusted, and Coordinates can be created.

Referring to Figure 5, the method according to the embodiments further includes performing a screw fastening process based on coordinates, and the step of performing the screw fastening process includes controlling a fastener for screw fastening based on the coordinates. It may include steps.

Referring to Figure 6, methods according to embodiments may calculate a plurality of coordinate information from at least one fastening surface where the screw hole area of the product is located.

Referring to Figure 7, the method according to embodiments may be performed by a deep learning-based screw fastening position automation teaching device. The device has memory; and a processor coupled to the memory; It includes, the processor acquires an image, performs preprocessing on the image, searches the image, extracts the screw fastening area from the image, performs preprocessing on the extracted image, and extracts the hole area from the screw fastening area. extract; and coordinates for the hole area can be stored.

Additionally, the processor performs preprocessing on the image based on the coordinates for the hole region, extracts a feature map for the image, calculates a screw hole region location based on the feature map, and calculates a screw hole region location. You can store the coordinates for .

Additionally, in order to search the image, the processor searches for a screw fastening hole area included in the image, extracts the screw fastening hole area, adjusts at least one of the size or color of the screw fastening hole area, and selects the size or color of the screw fastening hole area. At least one adjusted screw fastening hole area may be extracted, and coordinates for the screw fastening area may be generated.

Additionally, the processor may perform a screw fastening process based on coordinates and control a fastener for screw fastening based on the coordinates.

Additionally, the processor may calculate a plurality of coordinate information from at least one fastening surface where the screw hole area of the product is located.

A storage medium storing a program for a deep learning-based screw fastening position automation teaching method according to embodiments may be a non-transitory computer-readable storage medium in which computer instructions are stored. The storage medium includes computer instructions that cause the computer to acquire an image; performing preprocessing on the image; Browsing images; Extracting the screw fastening area from the image; Performing preprocessing on the extracted image; Extracting the hole area from the screw fastening area; and storing coordinates for the hole area; It may be a storage medium for performing a deep learning-based screw fastening position automation method, including.

As a result, the repetitive work of manually setting the fastening point can be minimized, thereby reducing man-hours. It is possible to respond to changing fastening areas by acquiring images of the fastening areas of classified products and predicting the fastening point locations using a deep learning algorithm. By transmitting predicted fastening coordinates from one image acquisition process to multiple processes for fastening each side, the positioning work of each fastening equipment is simplified. The fastening point is predicted through additional learning of the algorithm using the new fastening area and point image, enabling expansion in the fastening process of similar products.

The embodiments have been described in terms of a method and/or an apparatus, and the description of the method and the description of the apparatus may be applied to complement each other.

For convenience of explanation, each drawing has been described separately, but it is also possible to design a new embodiment by merging the embodiments described in each drawing. In addition, according to the needs of those skilled in the art, designing a computer-readable recording medium on which programs for executing the previously described embodiments are recorded also falls within the scope of the rights of the embodiments. The apparatus and method according to the embodiments are not limited to the configuration and method of the embodiments described above, but all or part of the embodiments can be selectively combined so that various modifications can be made. It may be composed. Although preferred embodiments of the embodiments have been shown and described, the embodiments are not limited to the specific embodiments described above, and are within the scope of common knowledge in the technical field to which the invention pertains without departing from the gist of the embodiments claimed in the claims. Of course, various modifications are possible by those who have, and these modifications should not be understood individually from the technical ideas or perspectives of the embodiments.

The various components of the devices of the embodiments may be implemented by hardware, software, firmware, or a combination thereof. Various components of the embodiments may be implemented with one chip, for example, one hardware circuit. Depending on the embodiments, the components according to the embodiments may be implemented with separate chips. Depending on the embodiments, at least one or more of the components of the device according to the embodiments may be composed of one or more processors capable of executing one or more programs, and the one or more programs may be executed. It may perform one or more of the operations/methods according to the examples, or may include instructions for performing them. Executable instructions for performing methods/operations of a device according to embodiments may be stored in a non-transitory CRM or other computer program product configured for execution by one or more processors, or may be stored in one or more processors. It may be stored in temporary CRM or other computer program products configured for execution by processors. Additionally, memory according to embodiments may be used as a concept that includes not only volatile memory (eg, RAM, etc.) but also non-volatile memory, flash memory, and PROM. Additionally, it may also be implemented in the form of a carrier wave, such as transmission over the Internet. Additionally, the processor-readable recording medium is distributed in a computer system connected to a network, so that the processor-readable code can be stored and executed in a distributed manner.

In this document, “/” and “,” are interpreted as “and/or.” For example, “A/B” is interpreted as “A and/or B”, and “A, B” is interpreted as “A and/or B”. Additionally, “A/B/C” means “at least one of A, B and/or C.” Additionally, “A, B, C” also means “at least one of A, B and/or C.” Additionally, in this document, “or” is interpreted as “and/or.” For example, “A or B” may mean 1) only “A”, 2) only “B”, or 3) “A and B”. In other words, “or” in this document may mean “additionally or alternatively.”

Terms such as first, second, etc. may be used to describe various components of the embodiments. However, the interpretation of various components according to the embodiments should not be limited by the above terms. These terms are used to distinguish one component from another. It's just a thing. It's just a thing. For example, a first user input signal may be referred to as a second user input signal. Similarly, the second user input signal may be referred to as the first user input signal. Use of these terms should be interpreted without departing from the scope of the various embodiments. The first user input signal and the second user input signal are both user input signals, but do not mean the same user input signals unless clearly indicated in the context.

The terminology used to describe the embodiments is for the purpose of describing specific embodiments and is not intended to limit the embodiments. As used in the description of the embodiments and the claims, the singular is intended to include the plural unless the context clearly dictates otherwise. The expressions and/or are used in a sense that includes all possible combinations between the terms. The expression includes describes the presence of features, numbers, steps, elements, and/or components and does not imply the absence of additional features, numbers, steps, elements, and/or components. . Conditional expressions such as when, when, etc. used to describe the embodiments are not limited to optional cases. It is intended that when a specific condition is satisfied, the relevant action is performed or the relevant definition is interpreted in response to the specific condition.

Additionally, operations according to embodiments described in this document may be performed by a transmitting and receiving device including a memory and/or a processor depending on the embodiments. The memory may store programs for processing/controlling operations according to embodiments, and the processor may control various operations described in this document. A processor may be referred to as a controller, etc. In embodiments, operations may be performed by firmware, software, and/or a combination thereof, and the firmware, software, and/or combination thereof may be stored in a processor or stored in memory.

Meanwhile, operations according to the above-described embodiments may be performed by a transmitting device and/or a receiving device according to the embodiments. The transmitting and receiving device may include a transmitting and receiving unit that transmits and receives media data, a memory that stores instructions (program code, algorithm, flowchart and/or data) for the process according to embodiments, and a processor that controls the operations of the transmitting and receiving device. You can.

A processor may be referred to as a controller, etc., and may correspond to, for example, hardware, software, and/or a combination thereof. Operations according to the above-described embodiments may be performed by a processor. Additionally, the processor may be implemented as an encoder/decoder, etc. for the operations of the above-described embodiments.

Claims (15)

acquiring an image;
performing preprocessing on the image;
searching the image;
Extracting a screw fastening area from the image;
performing preprocessing on the extracted image;
extracting a hole area from the screw fastening area; and
storing coordinates for the hole area; Including,
Deep learning-based automated teaching method for screw fastening positions.
The method of claim 1, wherein
performing preprocessing on the image based on the coordinates for the hole area;
extracting a feature map for the image;
calculating a screw hole area location based on the feature map; and
storing coordinates for the screw hole area location; Containing more,
Deep learning-based automated teaching method for screw fastening positions.
According to paragraph 1,
The step of searching the image is,
Explore the screw fastening hole area included in the image,
Extract the screw fastening hole area,
Adjusting at least one of the size or color of the screw fastening hole area,
Extracting a screw fastening hole area in which at least one of the size or color is adjusted,
Generating coordinates for the screw fastening area,
Deep learning-based automated teaching method for screw fastening positions.
The method of claim 3, wherein the method
Based on the coordinates, further comprising performing a screw fastening process,
The step of performing the screw fastening process includes controlling a fastener for screw fastening based on the coordinates,
Deep learning-based automated teaching method for screw fastening positions.
The method of claim 4, wherein
Calculating a plurality of coordinate information from at least one fastening surface where the screw hole area of the product is located,
Deep learning-based automated teaching method for screw fastening positions.
Memory; and
a processor connected to the memory; Includes, and the processor,
acquire the image,
Perform preprocessing on the image,
Explore the image above,
Extract the screw fastening area from the image,
Perform preprocessing on the extracted image,
extracting a hole area from the screw fastening area; and
storing coordinates for the hole area,
Deep learning-based screw tightening position automation teaching device.
The method of claim 6, wherein the processor:
Based on the coordinates for the hole area, perform preprocessing on the image,
Extract a feature map for the image,
Based on the feature map, calculate the screw hole area location,
Storing coordinates for the screw hole area location,
Deep learning-based screw tightening position automation teaching device.
The method of claim 6, wherein the processor:
To navigate the image,
Explore the screw fastening hole area included in the image,
Extract the screw fastening hole area,
Adjusting at least one of the size or color of the screw fastening hole area,
Extracting a screw fastening hole area in which at least one of the size or color is adjusted,
Generating coordinates for the screw fastening area,
Deep learning-based screw tightening position automation teaching device.
The method of claim 8, wherein the processor:
Based on the coordinates, perform a screw fastening process,
Here, the fastener for screw fastening is controlled based on the coordinates,
Deep learning-based screw tightening position automation teaching device.
The method of claim 8, wherein the processor:
Calculating a plurality of coordinate information from at least one fastening surface where the screw hole area of the product is located,
Deep learning-based screw tightening position automation teaching device.
In a non-transitory computer-readable storage medium storing computer instructions,
The computer command is
acquiring an image; performing preprocessing on the image; searching the image; Extracting a screw fastening area from the image; performing preprocessing on the extracted image; extracting a hole area from the screw fastening area; and storing coordinates for the hole area; A storage medium for performing a deep learning-based screw fastening position automation method, including.
The method of claim 11, wherein
performing preprocessing on the image based on the coordinates for the hole area;
extracting a feature map for the image;
calculating a screw hole area location based on the feature map; and
storing coordinates for the screw hole area location; Containing more,
A storage medium intended to perform a deep learning-based screw fastening position automation method.
According to clause 11,
The step of searching the image is,
Explore the screw fastening hole area included in the image,
Extract the screw fastening hole area,
Adjusting at least one of the size or color of the screw fastening hole area,
Extracting a screw fastening hole area in which at least one of the size or color is adjusted,
Generating coordinates for the screw fastening area,
A storage medium intended to perform a deep learning-based screw fastening position automation method.
The method of claim 13, wherein the method
Based on the coordinates, further comprising performing a screw fastening process,
The step of performing the screw fastening process includes controlling a fastener for screw fastening based on the coordinates,
A storage medium intended to perform a deep learning-based screw fastening position automation method.
15. The method of claim 14, wherein
Calculating a plurality of coordinate information from at least one fastening surface where the screw hole area of the product is located,
A storage medium intended to perform a deep learning-based screw fastening position automation method.