KR20240077645A - 3D Printer Safty System Using Image Analysis - Google Patents

Abstract

The method of securing 3D printer safety using an image analysis process according to the present invention is a method of securing 3D printer safety that is installed in a 3D printer and executed in a control unit linked to a camera that photographs the inner bottom surface including the bed of the 3D printer, A first step of determining the printing boundary of the output based on the g-code information of the output to be printed with a 3D printer; A second step of detecting abnormal printing by comparing the printing boundary with a low-brightness cell area calculated through analysis of real-time images from the camera.

Description

3D Printer Safety System Using Image Analysis}

The present invention relates to a method and system for securing the safety of a 3D printer using an image analysis process.

Abnormal output from a 3D printer is undesirable because it leads to wasted material and time due to reprinting. Several technologies are known for immediately detecting abnormal output from such 3D printers and stopping the output.

For example, when using expensive equipment such as a Depth Camera, it is not desirable in terms of cost-effectiveness to use it to detect abnormal output that mainly occurs in low-priced 3D printers.

In addition, in Patent Document 1, the nozzle position is estimated based on the image, the single layer of the 3D manufactured product output from the nozzle of the 3D printer is extracted, the center of each extracted fault is estimated, and the position of the nozzle is at the preset position indicated on the design drawing. If it deviates from or the estimated center position of the fault does not match the center position of the fault preset in the design drawing, it is judged to be an abnormal output.

However, the process of estimating the nozzle position based on the image and extracting the single layer of the 3D manufactured product printed from the nozzle of the 3D printer has the problem that it is not easy to implement or is currently a very high-specification and high-cost process.

Republic of Korea Public Patent No. 10-2019-0000182, filed on June 22, 2017

The present invention is intended to provide a simpler and more efficient 3D printer safety system and method.

A method of securing 3D printer safety using an image analysis process according to an aspect of the present invention is a method of securing 3D printer safety that is installed in a 3D printer and executed in a control unit linked to a camera that photographs the inner bottom surface including the bed of the 3D printer. As, a first step of determining a printing boundary of the output based on the G-code information of the output to be printed with the 3D printer; A second step of detecting abnormal printing by comparing the printing boundary with a low-brightness cell area calculated through analysis of real-time images from the camera.

In the method of securing 3D printer safety using the image analysis process described above, the cell area is created by dividing the entire area of the image horizontally and vertically, and is an area containing a predetermined plurality of pixels, and the low-brightness cell area is The cell area may be a cell area in which the average luminance of pixels belonging to the cell area is below a reference value.

In the method of securing 3D printer safety using the above-mentioned image analysis process, in the second step, after starting printing of the output according to the g-code information, the low-brightness cell area is newly generated, and the low-brightness cell area is If it is outside the printing boundary, it can be judged as abnormal printing.

In the method of securing 3D printer safety using the above-mentioned image analysis process, in order to determine whether the low-brightness cell area has been newly generated, i) before starting printing of the output according to the g-code information, by the camera In the captured image, a cell area with an average luminance value smaller than the average luminance value of the entire image is determined as an initial set of low-brightness cell areas, ii) After starting printing of the output according to the g-code information, the In an image captured by a camera, a cell area with an average luminance value smaller than the average luminance value of the entire image is determined as a real-time low-brightness cell area, iii) the real-time low-brightness cell area that does not belong to the initial set of low-brightness cell areas. If there is a low brightness cell area, it can be determined that the low brightness cell area has been newly created.

In the method of securing 3D printer safety using the image analysis process described above, in the second step, after starting printing of the output according to the G-code information, in the image captured by the camera, the entire area of the bed A cell area with an average luminance value smaller than the average luminance value is determined as a real-time low-brightness cell area, and if the real-time low-brightness cell area occurs within the area of the bed but is outside the printing boundary, it is determined to be abnormal printing. can do.

In the method of securing 3D printer safety using the image analysis process described above, the area of the bed can be set by the user by designating the vertex of the bed in the image captured by the camera.

In the method of securing 3D printer safety using the above image analysis process, in the second step, if it is determined that the abnormal printing is above, printing by the 3D printer can be stopped and a warning message can be sent to a pre-registered email address. there is.

A 3D printer safety system using an image analysis process according to an aspect of the present invention includes a camera installed in a 3D printer to photograph the inner bottom surface including the bed of the 3D printer, and a control unit linked to the camera. As a safety system, the control unit determines a printing boundary of the output based on the G-code information of the output to be printed with the 3D printer, and the low-brightness cell area calculated through analysis of the real-time image from the camera and the It is characterized by detecting abnormal printing by comparing printing boundaries.

In the 3D printer safety system using the image analysis process described above, the cell area is created by dividing the entire area of the image horizontally and vertically, and is an area containing a predetermined plurality of pixels, and the low-brightness cell area is the It may be a cell area where the average luminance of pixels belonging to the cell area is below a reference value.

In the 3D printer safety system using the above-described image analysis process, the control unit starts printing the output according to the g-code information, and then the low-brightness cell area is newly generated, and the low-brightness cell area is used in the printing. If it is outside the boundary, it can be judged as abnormal printing.

In the 3D printer safety system using the image analysis process described above, in the second step, after starting printing of the output according to the G-code information, in the image captured by the camera, the entire area of the bed A cell area with an average luminance value smaller than the average luminance value is determined as a real-time low-luminance cell area.

The real-time low-brightness cell area is generated within the area of the bed, but if it is outside the printing boundary, it can be determined to be abnormal printing.

According to the present invention, there is an advantage that the camera normally equipped with a 3D printer can be used as is, and abnormal output can be efficiently detected even with simple software and low-cost hardware resources.

In addition, according to the present invention, the color value of the image acquired through streaming (and the luminance value processed thereof) is analyzed and abnormal output is detected using the relative brightness difference, so it has the advantage of being resistant to disturbance or noise.

Figure 1 is a block diagram showing a situation in which a 3D printer safety system using an image analysis process according to an embodiment of the present invention is applied.
Figure 2 is a flow chart showing a method of securing 3D printer safety using an image analysis process according to the first embodiment of the present invention.
FIG. 3(a) shows an example of a g-code, and FIG. 3(b) shows an example of a printout P on a bed.
Figure 4(a) is an image captured by a camera, and Figure 4(b) shows a low-brightness map overlaid on the image.
Figure 5(a) is an image when an abnormal output is discovered, and Figure 5(b) is an example of an image and a low brightness map overlaid to further display a new low brightness cell area.
Figure 6 is an example of an email containing a warning message sent to a user through a mailing system.
Figure 7 is a flow chart showing a method of securing 3D printer safety using an image analysis process according to the second embodiment of the present invention.

Figure 1 is a block diagram showing a situation in which a 3D printer safety system using an image analysis process according to an embodiment of the present invention is applied.

A 3D printer safety system using an image analysis process according to an embodiment of the present invention may be configured to include a control unit 10 and a camera 20.

The camera 20 is installed in a 3D printer to photograph the inner bottom surface including the bed of the 3D printer. It may be a camera that is usually installed in a commercial 3D printer and is used to display the printing situation to the user. , It is usually installed at the upper corner of the chamber 2, and accordingly, the area of the bed photographed by the camera 20 is generally photographed obliquely. In the present invention, it can be applied even when the area of the bed is photographed at an angle by the camera 20.

The control unit 10 is linked with the 3D printer 1 and the camera 20, and can receive messages indicating the operating status from the 3D printer 1 or the computer that controls the 3D printer 1. G-code information can be received from the user or in conjunction with the 3D printer 1 or the computer that controls it using the input port. Additionally, the control unit 10 may stop printing of the 3D printer 1 when abnormal printing is detected. The control unit 10 may be implemented as a controller, a microprocessor, a one-board computer, or a general-purpose computer, for example, a Raspberry Pi-based one-board computer.

Figure 2 is a flow chart showing a method of securing 3D printer safety using an image analysis process according to the first embodiment of the present invention.

The control unit 10 can receive g-code information of the output to be printed with a 3D printer from a user, etc., or receive g-code information from a 3D printer, etc., and determines the printing boundary of the output based on such g-code information. Decide (S10).

For example, when a user uploads a g-code as shown in Figure 3(a) to the control unit (Raspberry Pi), only the values for x-y coordinates are extracted from the coordinates in the g-code and the printing boundary in the x-y plane is calculated. This printing boundary is considered the location where the output is placed on the bed of the 3D printer in printing, and the area outside this printing boundary is considered an area where the output can never be placed. Figure 3(b) shows an example of a printout (P) printed on a bed, and the printing boundary (B) can be determined by analyzing g-code information.

Then, before starting printing of the output according to the G-code information, the control unit 10 selects a cell area with an average luminance value smaller than the average luminance value of the entire image as a low-luminance cell area in the image captured by the camera. It is decided as the initial set of (S12).

Here, the cell area is an area created by dividing the entire area of the image horizontally and vertically, and includes a certain number of pixels. For example, if an image consists of 640X480 pixels, the image can be divided into an array of cell areas as if it were composed of 64X48 pixels.

A low brightness cell area refers to a cell area where the average brightness of pixels belonging to the cell area is below a reference value, and the reference value may be the average brightness of the entire image. Ultimately, the low brightness cell area is compared to the average brightness of the entire current image and becomes a cell area with relatively low brightness.

The control unit 20 can obtain an image as shown in Figure 4(a) from an arbitrary image included in the real-time video received from the camera 20. The image of the streaming color video is first converted to gray scale (i.e., luminance value As described above, this image is divided into a plurality of cell areas arranged in a tiled manner in consideration of the camera resolution, and the average of the cell areas (i.e., the average luminance value) is used to judge based on these cell areas. Additionally, the average luminance value (reference value), which is the average of the luminance values of the entire image, is also calculated.

A cell area where the average luminance value of the cell area is smaller than the average luminance value of the entire image is called a low-brightness cell area (relatively low-brightness cell area), and the set of these low-brightness cell areas is determined as the initial set of low-brightness cell areas. . Therefore, the average luminance value of the entire image becomes a reference value considering the overall brightness of the image, and the initial set of low-brightness cell areas becomes a set of cell areas with luminance less than the reference value considering the overall brightness of the image.

FIG. 4(b) shows a low-brightness map overlaid on an image. As shown in FIG. 4(b), the control unit 10 sets the low-brightness cell areas to 255, which is the maximum value, on the low-brightness map to display white. It can also be expressed as . Each white area (dot) represents a low-brightness cell area and is displayed as a representative of the included pixels. This low brightness map is referenced to determine whether subsequent abnormal printing occurs.

In FIG. 2, step S10 is shown as if performed before step S12, but the execution order does not matter. The control unit 10 can use a message from the 3D printer 1 to know before starting printing of the output. For example, the 3D printer (1) sends a message (or a code indicating this) such as “printer selected” before starting printing, and sends a message (or a code indicating this) such as “printing” when it starts printing. , you can receive this and know before printing of the output begins.

After starting printing, the control unit 10 detects abnormal printing by comparing the low-brightness cell area calculated through analysis of the real-time image from the camera 20 and the printing boundary (B) (S20) ). After the control unit 10 starts printing the output according to the g-code information, a new low-brightness cell area (relatively low-brightness cell area) is generated, and if the low-brightness cell area is outside the printing boundary, it determines that printing is abnormal. .

First, after starting the printing of the output according to the g-code information, in the real-time image captured by the camera, the cell area with the average luminance value smaller than the average luminance value of the entire image is determined as the real-time low-brightness cell area (S22 ). Since the method of calculating the average luminance value of the entire image and the average luminance value of the cell area is the same as the method used to determine the low-brightness cell area before starting printing of the output as described above, the description thereof is omitted.

The real-time low-brightness cell area is a cell area with an average luminance value that is smaller than the average luminance value of the image in each image (video frame) streamed in the current situation. Since the average luminance value of each frame (image) of a streaming video changes for various reasons such as shadows of moving objects or changes in lighting conditions, the real-time low-brightness cell area reflects these changes. Therefore, compared to the average luminance value of the image (over the entire image) calculated when determining the initial set of low-luminance cell areas, the average luminance value of the image (over the entire image) calculated when determining the real-time low-luminance cell area is different. This can be done by taking into account the current disturbance or noise.

The control unit 20 determines whether a new low-brightness cell area is generated in real time and whether the low-brightness cell area is outside the above-mentioned printing boundary (S24). If the result of this judgment is true, it determines abnormal printing (S26).

Specifically, if the real-time low-brightness cell area of the image does not belong to the initial set of low-brightness cell areas, this real-time low-brightness cell area is judged to be a newly created low-brightness cell area, which naturally means that the real-time low-brightness cell area is not included in the initial set of low-brightness cell areas. This is done through comparison. In this way, if a real-time low-brightness cell area that does not belong to the initial set is outside the printing boundary, it becomes an area with abnormal printing, and it is considered that abnormal printing was performed in that cell area.

In order to determine whether a real-time low-brightness cell area is outside the printing boundary, the coordinate system representing the position of the cell area in the image and the coordinate system representing the printing boundary are converted to be unified. To this end, one coordinate system can be converted to the other coordinate system. there is. For example, since images taken from an oblique position of the camera 20 are distorted, these images can be transformed to compensate for the distortion, and the coordinates of the G-code can be transformed to the corresponding positions of the camera image.

The present invention has the aspect of comparing the printing boundary obtained through G-code information with the streaming image coming through the camera 20. Additionally, the bed typically appears bright in the image and the output appears dark compared to it.

This difference is used to find the cell area where the output is found outside the printing boundary. In other words, the bed area corresponding to the outer part of the printing boundary must always display an appropriate color, and if a print is found in this area through the process considered above, an abnormality is detected.

Figure 5(a) is an image when abnormal output is discovered, and Figure 5(b) is an example of overlapping the image and the low brightness map, further displaying new low brightness cell areas (L).

The areas expressed in green in Figure 5(b) are the cell areas where abnormal output occurred. In the drawing, they are marked in green to distinguish them from the initial low-brightness cell areas, but in the actual process, the luminance value of 255, that is, is considered white. can be processed. In addition, in the above process, if abnormal output is detected three or more times in the same area, stability can be promoted through methods such as making a final judgment as output defect.

And, if the control unit 10 determines that printing is abnormal, it stops printing by the 3D printer and sends a warning message to a pre-registered email address. When the system of the present invention determines that the output is defective, it sends an email including a warning message and streaming address as shown in Figure 6 to the user through a mailing system and stops the printer.

Figure 7 is a flow chart showing a method of securing 3D printer safety using an image analysis process according to the second embodiment of the present invention.

Since the method for securing 3D printer safety according to the second embodiment of the present invention is different from the method for securing 3D printer safety according to the first embodiment of the present invention only in some processes, the description will focus on this.

Before starting printing of the output according to the G-code information, in step 8, the user sets the bed area by designating the vertices of the bed area in the image captured by the camera 20. These settings can be set once when introducing a 3D printer and have nothing to do with the output. Therefore, it can be executed prior to step S10. Thereafter, the control unit 10 may identify the bed area in the image of the input streaming video.

In step 23, the control unit 10 determines a real-time low-brightness cell area within the bed area. After starting printing of the output according to the g-code information, in the image captured by the camera, a cell area with an average luminance value smaller than the average luminance value of the entire bed area is determined as a real-time low-brightness cell area, step In steps 23 and 26, if a real-time low-brightness cell area occurs within the bed area but is outside the printing boundary, it is determined to be abnormal printing.

According to the present invention, there is an advantage that the camera normally equipped with a 3D printer can be used as is, and abnormal output can be efficiently detected even with simple software and low-cost hardware resources.

In addition, according to the present invention, the color value of the image acquired through streaming (and the luminance value processed thereof) is analyzed and abnormal output is detected using the relative brightness difference, so it has the advantage of being resistant to disturbance or noise. Even if there are shadows created when people pass by, changes in illuminance due to changes in external incident light and the ON/OFF of indoor lighting, such noise is reflected as a common factor in the algorithm of the present invention and is excluded in the process of detecting abnormal output ( This has the advantage of increasing detection accuracy because it can be rejected.

1: 3D printer
2: Chamber
10: control unit
20: Camera

Claims (11)

In a method of securing 3D printer safety that is installed in a 3D printer and executed in a control unit linked to a camera that photographs the inner bottom surface including the bed of the 3D printer,
A first step of determining a printing boundary of the output based on the G-code information of the output to be printed with the 3D printer;
A second step of detecting abnormal printing by comparing the printing boundary with a low-brightness cell area calculated through analysis of real-time images from the camera.
How to secure 3D printer safety using the image analysis process. In claim 1,
The cell area is created by dividing the entire area of the image horizontally and vertically, and is an area containing a certain number of pixels,
The low-brightness cell area is a cell area in which the average brightness of pixels belonging to the cell area is less than a reference value,
How to secure 3D printer safety using the image analysis process. In claim 2,
In the second step,
After starting printing of the output according to the g-code information, a new low-brightness cell area is generated, and if the low-brightness cell area is outside the printing boundary, it is determined that printing is abnormal.
How to secure 3D printer safety using the image analysis process. In claim 3,
In order to determine whether the low-brightness cell area has been newly created,
i) Before starting printing of the output according to the g-code information, in the image captured by the camera, a cell area with an average luminance value smaller than the average luminance value of the entire image is an initial set of low-brightness cell areas. It is decided that
ii) After starting printing of the output according to the g-code information, in the image captured by the camera, a cell area with an average luminance value smaller than the average luminance value of the entire image is determined as a real-time low-brightness cell area; ,
iii) If there is a real-time low-brightness cell area that does not belong to the initial set of low-brightness cell areas, determining that the low-brightness cell area is newly generated,
How to secure 3D printer safety using the image analysis process. In claim 2,
In the second step,
After starting printing of output according to the G-code information,
In the image captured by the camera, a cell area having an average luminance value smaller than the average luminance value of the entire area of the bed is determined as a real-time low-brightness cell area,
If the real-time low-brightness cell area is generated within the area of the bed, but is outside the printing boundary, it is determined to be abnormal printing.
How to secure 3D printer safety using the image analysis process. In claim 5,
The area of the bed is set by the user by specifying the vertex of the bed in the image captured by the camera,
How to secure 3D printer safety using the image analysis process. In claim 1,
In the second step,
If it is determined that the above-described abnormal printing is occurring, printing by the 3D printer is stopped and a warning message is sent to a pre-registered email address.
How to secure 3D printer safety using the image analysis process. In the 3D printer safety system including a camera installed in a 3D printer and photographing the inner bottom surface including the bed of the 3D printer, and a control unit linked to the camera,
The control unit,
The printing boundary of the output is determined based on the g-code information of the output to be printed with the 3D printer, and the printing boundary is compared with the low-brightness cell area calculated through analysis of real-time images from the camera. Characterized by detecting abnormal printing,
3D printer safety system using image analysis process. In claim 8,
The cell area is created by dividing the entire area of the image horizontally and vertically, and is an area containing a certain number of pixels,
The low-brightness cell area is a cell area in which the average brightness of pixels belonging to the cell area is less than a reference value,
3D printer safety system using image analysis process. In claim 9,
The control unit,
After starting printing of the output according to the g-code information, a new low-brightness cell area is generated, and if the low-brightness cell area is outside the printing boundary, it is determined that printing is abnormal.
3D printer safety system using image analysis process. In claim 9,
In the second step,
After starting printing of output according to the G-code information,
In the image captured by the camera, a cell area having an average luminance value smaller than the average luminance value of the entire area of the bed is determined as a real-time low-brightness cell area,
If the real-time low-brightness cell area is generated within the area of the bed, but is outside the printing boundary, it is determined to be abnormal printing.
3D printer safety system using image analysis process.