KR20230134895A - LED chip Fluidic self-arranging device, system and method using machine learning - Google Patents

Abstract

Provided are an LED chip fluidic self-arranging device, system, and method using machine learning. According to an embodiment of the present invention, the LED chip fluidic self-arranging device using machine learning comprises: an image photographing module photographing an image including a substrate and chips, and acquiring image information including information on an arrangement location which is a location where the chips should be arranged on the substrate; a chip arrangement module located under the image photographing module, formed in a polyhedral shape with an open upper surface and an empty inside to store the substrate and the chips, and filled with a preset fluid for a certain volume for moving the chips; a wave energy generation module formed to come in contact with a lower surface of the chip arrangement module, generating wave energy, and transmitting the wave energy to the fluid; and a chip arrangement control module determining the moving direction of the chips by using the image information and the wave energy, and determining the properties of the wave energy for moving an unarranged chip which is not arranged among the chips to an unarranged part which is a part with no chip arranged among arrangement locations. Therefore, an unarranged chip can be arranged on an unarranged part of a substrate.

Description

LED chip fluid self-arranging device, system and method using machine learning {LED chip Fluidic self-arranging device, system and method using machine learning}

The present invention relates to a device, system, and method for self-aligning an LED chip fluid using machine learning. In particular, the present invention relates to a device, system, and method for self-aligning an LED chip using machine learning. In particular, wave energy having characteristics acquired through machine learning is applied to a substrate and a chip inserted inside the fluid to transform the unaligned chip into the substrate. This relates to a device, system, and method for self-arranging LED chip fluids using machine learning that can be arrayed in non-aligned areas.

In general, a microLED display is a display that uses LED chips as pixels, and the chips must be 100% completely transferred to the substrate on which the circuit is formed. If there is no chip in the pixel or a defective chip is transferred to the substrate, a defect may occur in the display module, so a very careful process is required. Accordingly, the currently used transfer technology can very easily achieve a transfer rate of up to 90%, but in the actual commercial stage, a transfer rate of 5N (99.999%) or more is required.

The most problematic aspect of any transcription technique is completeness. Most technologies easily reach 2N or 3N, but at the commercial stage, perfection of 5N or more is required as described above. For example, the number of subpixels with 4K resolution (4096x2160) is 8.8M x 3, which is 26,542,080, and in the case of 4N, there are 2,654 bad pixels. In practice, correcting this requires a high period of time and cost.

Korean Patent No. 10-1791380

In order to solve the problems of the prior art as described above, an embodiment of the present invention applies wave energy having characteristics acquired through machine learning to the substrate and chip inserted inside the fluid to separate the unaligned chip from the unaligned portion of the substrate. The goal is to provide a device, system, and method for self-arranging LED chip fluids using machine learning that can array them.

According to one aspect of the present invention for solving the above problems, an LED chip fluid self-arranging device using machine learning is provided. The LED chip fluid self-arranging device using the machine learning includes an image capture module that captures an image including a substrate and a chip and obtains image information including information about an array position, which is a position where the chip should be arranged on the substrate; A chip located at the bottom of the image capture module, formed in the form of a polyhedron with an open upper surface to contain the substrate and the chip, and an empty interior, and filled with a predetermined volume of fluid for movement of the chip. array module; a wave energy generation module formed to contact the lower surface of the chip array module and generating wave energy and transmitting the wave energy to the fluid; and determining the direction of movement of the chip using the image information and the wave energy, and moving the unaligned chip, which is the non-arranged chip among the chips, to the unarranged portion, which is the portion where the chip is not aligned, among the array positions. It includes a chip array control module for determining the characteristics of the wave energy.

The characteristics of the wave energy include frequency information corresponding to the moving direction of the chip as a result of machine learning using previously acquired learning data, and the chip array control module is configured to move the unarranged chip to the unarranged portion. The frequency information can be determined as a characteristic of the wave energy so that the non-arranged chip can move to the non-arranged area.

The unaligned chip moving to the non-aligned portion may be the unaligned chip located closest to the non-aligned portion obtained by analyzing the image information.

According to one aspect of the present invention, an LED chip fluid self-arranging system using machine learning is provided. The LED chip fluid self-arrangement system using the machine learning acquires an image including the chip and the upper surface of the substrate into which the chip is to be inserted, and processes the obtained image to determine which of the upper surfaces the chip is not arranged in. an image information processing unit that acquires image analysis information, which is information about the unarranged portion and the unarranged chips among the chips that are not arranged on the substrate; A machine learning result acquisition unit that learns the movement characteristics of the chip within the fluid according to wave energy applied to the chip inserted into the preset fluid using a preset machine learning algorithm and obtains a learning result; and a self-arrangement performing unit that generates the wave energy using the learning result and performs self-arrangement of the unaligned chip so that the unaligned chip can be located in the non-arranged region.

The image information processing unit may include an image information acquisition module that acquires the image including the upper surface and the chip; and an image information analysis module configured to analyze the image to acquire the image analysis information, which is information about the non-arranged portion and the non-arranged chip.

The image information processing unit matches a preset number of unaligned chips close to each unaligned portion, selects defective chips among the unaligned chips using pre-learned results of image analysis results for the chips, and , an array information processing module that performs exclusion processing from the matching; may further include.

The machine learning result acquisition unit acquires basic information for acquiring fluid information, which is information about the preset fluid, which is basic information for performing the machine learning, and wave information, which is a specific characteristic of the wave energy emitted to the preset fluid. module; A state change result acquisition module that acquires an initial state image and a subsequent state image, which is an image after the wave energy is emitted, among the images, and obtains a path along which the chip moves through the emission of the wave energy as a state change result; and a machine learning result acquisition module that generates a vector path, which is movement information of the chip, using the state change result, and obtains the learning result by performing repeated learning a preset number of times.

The specific characteristic of the wave energy is frequency, and the machine learning result acquisition module obtains a final learning result by applying a correction variable obtained externally to the simulation learning result obtained through simulation, and applies the final learning result to the above. It can be obtained as a learning result.

The self-arranging unit may include a dictionary information processing module that acquires the image analysis information and the learning result, and divides the image based on a preset standard to obtain at least one divided area; a self-arrangement performing module that determines priorities of the divided areas using preset conditions and outputs the wave energy to perform self-arrangement according to the priorities; When the self-arrangement is completed, a self-arrangement verification module verifies whether the unaligned portion does not exist using the image analysis information.

According to one aspect of the present invention, a method for self-arranging LED chip fluids using machine learning is provided. The LED chip fluid self-arrangement method using the machine learning acquires an image including the chip and the upper surface of the substrate into which the chip is to be inserted using an image information processor, and processes the acquired image to determine the image of the upper surface of the upper surface. Obtaining image analysis information, which is information about an unarranged area where the chips are not aligned and unaligned chips among the chips that are not aligned on the substrate; Using a machine learning result acquisition unit, the movement characteristics of the chip within the fluid are learned using a preset machine learning algorithm according to the wave energy applied to the chip inserted inside the preset fluid, and the learning results are obtained. step; and generating the wave energy in a self-arrangement performing unit using the learning result to perform self-arrangement of the non-arranged chip so that the non-arranged chip can be located in the non-arranged region.

Obtaining the image analysis information may include acquiring the image including the upper surface and the chip; and analyzing the image to obtain the image analysis information, which is information about the non-arranged portion and the non-arranged chip.

The step of acquiring the image analysis information includes matching a preset number of unarranged chips close to each unarranged region, and using pre-learned results of the image analysis results for the chips to determine which of the unarranged chips are defective. It may further include selecting a chip and performing exclusion processing from the matching.

The step of learning the movement characteristics of the chip using a preset machine learning algorithm and obtaining a learning result includes fluid information, which is information about the preset fluid, which is basic information for performing the machine learning, and the preset fluid. Obtaining wave information that is a specific characteristic of the wave energy emitted to; Obtaining an initial state image and a subsequent state image, which is an image after the wave energy is emitted, among the images, and obtaining a path along which the chip moves through the emission of the wave energy as a result of a state change; and generating a vector path that is movement information of the chip using the state change result, and performing repeated learning a preset number of times to obtain the learning result.

The specific characteristic of the wave energy is frequency, and the step of obtaining the learning result includes obtaining a final learning result by applying a correction variable obtained externally to the simulation learning result obtained through simulation, and obtaining the final learning result. It can be obtained as a result of the above learning.

The step of performing self-arrangement of the unaligned chip includes obtaining the image analysis information and the learning result, dividing the image based on a preset standard to obtain at least one divided area; determining priorities of the divided areas using preset conditions and outputting the wave energy to perform self-arrangement according to the priorities; When the self-alignment is completed, the method may include verifying whether the unaligned portion does not exist using the image analysis information.

The LED chip fluid self-arranging device, system, and method using machine learning according to an embodiment of the present invention acquire movement information of unaligned chips according to the characteristics of wave energy through machine learning, and identify unaligned chips and By identifying the unaligned portion of the substrate and providing specific wave energy, there is an effect of allowing the unaligned chips to be aligned in the non-aligned portion of the substrate.

Figure 1 is (a) a perspective view and (b) a block diagram showing a portion of an LED chip fluid self-arranging device using machine learning according to an embodiment of the present invention.
Figure 2 is a block diagram showing an LED chip fluid self-arranging system using machine learning according to an embodiment of the present invention.
FIG. 3 is a block diagram showing the image information processing unit of FIG. 2.
Figure 4 is a block diagram showing the machine learning result acquisition unit of Figure 2.
FIG. 5 is a block diagram showing the self-arrangement performing unit of FIG. 2.
Figure 6 is a flowchart showing the LED chip fluid self-arrangement method using machine learning according to an embodiment of the present invention.
Figure 7 is a flowchart showing step S21 of Figure 6.
Figure 8 is a flowchart showing step S23 of Figure 6.
Figure 9 is a flowchart showing step S25 of Figure 6.

Hereinafter, with reference to the attached drawings, embodiments of the present invention will be described in detail so that those skilled in the art can easily implement the present invention. The present invention may be implemented in many different forms and is not limited to the embodiments described herein. In order to clearly explain the present invention in the drawings, parts not related to the description are omitted, and identical or similar components are given the same reference numerals throughout the specification.

Hereinafter, the LED chip fluid self-alignment policy, system, and method using machine learning according to an embodiment of the present invention will be described using FIGS. 1 to 9, respectively. Hereinafter, for convenience of explanation, the LED chip is expressed as a chip, and the LED chip of the present invention may be a micro LED, for example. However, the present invention is not necessarily limited to this and can be applied to various chips combined with a substrate.

1) LED chip fluid self-arranging device using machine learning

Figure 1 is (a) a perspective view and (b) a block diagram showing a portion of an LED chip fluid self-arranging device using machine learning according to an embodiment of the present invention. 1A and 1B, the LED chip fluid self-arranging device using machine learning according to an embodiment of the present invention includes an image capture module 11, a chip array module 13, a wave energy generation module 15, and a chip. It is formed to include an array control module (17). FIG. 1A is an example of a perspective view for explaining in detail the chip array module 13 and the wave energy generation module 15, and FIG. 1B is a block diagram of the modules constituting the device of the present invention.

The image capturing module 11 is configured to acquire image information including information about the arrangement position, which is the position where the chips should be arranged on the substrate, by capturing an image including the substrate and the chip. For example, the image capturing module 11 may be a camera commonly used in various industrial fields. The image capture module 11 can capture images at various moments, and a plurality of captured images can be used in the device of the present invention as needed. In addition, the image capturing module 11 is preferably installed in a direction perpendicular to the ground in order to capture images of the chip array module 13, which will be described later, and may be configured to capture images in the direction of the ground. Furthermore, the image capturing module 11 may also be provided with an absorption member at a fixed portion in order not to be affected by vibration generated from the wave energy generation module 15, which will be described later, and the wave energy generation module 15 and They can also be physically separated and combined to form the fluid self-arranging device 1 of the present invention.

The chip array module 13 is located below the image capture module and may have an open upper surface to contain a substrate and a chip. The chip array module 13 is formed in the shape of a polyhedron with an empty interior, and a substrate, a chip, and a preset fluid are inserted therein, allowing the chip to move and be inserted into the substrate. The chip array module 13 according to an embodiment of the present invention is formed, for example, in a hexahedral shape, as shown in FIG. 1A. The chip array module 13 is formed in the shape of a hexahedron with an open upper surface and an empty interior, and the interior is filled with fluid to a certain height. The fluid may preferably fill a volume greater than that in which the substrate and chip can be submerged. In addition, as shown in FIG. 1A, the chip array module 13 may be provided with a drain pipe and a permeable pipe on one side to facilitate the injection or drainage of fluid. Additionally, in another embodiment of the present invention, the chip array module 13 may be formed so that its lower surface has a certain angle of inclination to facilitate the placement and collection of chips, and may be set to a variable type to adjust the inclination as needed. It can be configured to adjust.

The wave energy generation module 15 is provided at the lower part of the chip array module 13. The wave energy generation module 15 is formed to be in contact with the lower surface of the chip array module 13, and is formed to generate wave energy and transfer the wave energy to the fluid introduced into the chip array module 13. The wave energy generation module 15 is formed to generate wave energy by vibrating forward and backward, left and right, and up and down, as shown in FIG. 1A. Furthermore, the wave energy generation module 15 is formed so that the vibrator can be replaced, and the contact position and distance between the vibrator and the vibration providing member (not shown) can be adjusted in three axes. Through this, the wave energy generation module 15 can apply desired energy to the chip array module 13 or fix the contact position and area.

The chip array control module 17 uses image information and wave energy to determine the direction of movement of the chips, and moves the unarranged chips, which are the unarranged chips among the chips, to the unarranged area, which is the part where the chips are not arranged. It is formed to determine the characteristics of wave energy. Here, the characteristics of wave energy include frequency information corresponding to the direction of movement of the chip as a result of machine learning using previously acquired learning data.

To explain in more detail, the characteristics of wave energy are frequency information of wave energy, which is obtained through machine learning. Machine learning is obtained using simulation and correction variables, and simulation refers to the simulation of the direction in which the chips move when wave energy with a specific frequency is delivered to chips inserted into a specific fluid. That is, the chip array control module 17 acquires the moving direction of the chip for a specific frequency through simulation, learns the moving direction of the chip through machine learning, and learns realistic movement by applying correction variables to the learning results. Obtain results.

The chip arrangement control module 17 obtains the positions of the unarranged area and the unarranged chip through analysis of image information. Once the positions of the unarranged area and the unarranged chip are acquired, the chip array control module 17 determines the unarranged chip to be moved to the non-arranged area according to a preset standard, and the direction in which the non-arranged chip will move to the non-arranged area. can be obtained. Once the movement direction is obtained, the learning result can be used to move the unaligned chip in the obtained movement direction.

In more detail, the chip array control module 17 generates a vector path through which the unarranged chip can move to the non-arranged area, and uses the learning result to enable the non-arranged chip to move to the non-arranged area. can be determined, and the determined wave energy frequency can be transmitted to the above-described wave energy generation module 15.

Meanwhile, a preset standard for determining unarranged chips in the chip array control module 17 may be determined according to the unarranged portion obtained by analyzing image information. The unaligned chip moving to the non-aligned region may be an unaligned chip located closest to a specific unaligned region obtained by analyzing image information. In addition, the chip array control module 17 analyzes image information, divides the image into a plurality of zones, and assigns a priority to each zone so that unaligned chips preferentially move to the unarranged portions in a specific zone. You may.

2) LED chip fluid self-arranging system using machine learning

2 to 5 show an LED chip fluid self-arranging system using machine learning according to an embodiment of the present invention. Figure 2 is a block diagram showing an LED chip fluid self-arrangement system using machine learning according to an embodiment of the present invention, Figure 3 is a block diagram showing the image information processing unit of Figure 2, and Figure 4 is a machine learning system of Figure 2. This is a block diagram showing the result acquisition unit, and FIG. 5 is a block diagram showing the self-arrangement performing unit of FIG. 2. Hereinafter, the LED chip fluid self-arranging system using machine learning according to an embodiment of the present invention will be described in detail using FIGS. 2 to 5.

Referring to FIG. 2, the LED chip fluid self-arranging system 2 using machine learning according to an embodiment of the present invention includes an image information processing unit 21, a machine learning result acquisition unit 23, and a self-arranging unit 25. ) is formed to include.

The image information processing unit 21 acquires an image including the chip and the upper surface of the substrate where the chip should be inserted, and processes the acquired image to determine the unarranged portion of the upper surface where the chip is not arranged and the chip not arranged on the substrate. It is configured to obtain image analysis information, which is information about unarranged chips. The image information processing unit 21 confirms the position of the substrate on which the chip is not aligned, and obtains the position of the unarranged chip, thereby providing basic information for arranging the unaligned chip in the non-arranged area in the self-arrangement performing unit 25, which will be described later. can be obtained. To this end, the image information processing unit 21 according to an embodiment of the present invention may be formed to include an image information acquisition module 211 and an image information analysis module 213, as shown in FIG. 3.

The image information acquisition module 211 is configured to acquire an image including the upper surface and the chip, and the image information analysis module 213 analyzes the acquired image to obtain image analysis information, which is information about the unarranged area and the unarranged chip. can be obtained. The image information analysis module 213 uses DNN to obtain image analysis information, and more specifically, may use the YOLO algorithm. In one embodiment of the present invention, YOLOv5 is used, but it is not necessarily limited thereto and various machine learning algorithms that can be used for image analysis can be used.

Meanwhile, the image information processing unit 21 according to an embodiment of the present invention may be formed to further include an array information processing module 215. The arrangement information processing module 215 matches a preset number of nearby unarranged chips for each unarranged area, and can select defective chips among the unarranged chips using pre-learned results of the image analysis results for the chips. . Additionally, the arrangement information processing module 215 may exclude the selected defective chips from matching to prevent the defective chips from being placed in non-arranged areas.

Meanwhile, when image analysis information is acquired, the LED chip fluid self-arranging system 2 using machine learning according to an embodiment of the present invention is formed to obtain machine learning results using the machine learning result acquisition unit 23. . The machine learning result acquisition unit 23 according to an embodiment of the present invention learns the movement characteristics of the chip inside the fluid according to the wave energy applied to the chip inserted inside the preset fluid using a preset machine learning algorithm. It is formed to obtain learning results. To this end, the machine learning result acquisition unit 23 of the present invention is formed to include a basic information acquisition module 231, a state change result acquisition module 233, and a machine learning result acquisition module 235, as shown in FIG. 4. It can be.

The basic information acquisition module 231 is configured to acquire basic information for performing machine learning. Basic information for performing machine learning may be fluid information and wave information. In the present invention, the substrate and the unaligned chip are formed to be inserted into a specific fluid in order to place the unaligned chip on the non-aligned portion of the substrate. Therefore, the basic information acquisition module 231 of the present invention can acquire basic information for obtaining learning results in the machine learning result acquisition module 235, which will be described later, by acquiring fluid information, which is information about the fluid into which the substrate is inserted. there is.

In addition, in the present invention, wave energy is used to control the movement of the chip, and more specifically, the movement of the chip is controlled by changing the frequency of the wave energy. Accordingly, the basic information acquisition module 231 is configured to acquire frequency information, which is a specific characteristic of wave energy provided to the fluid into which the substrate is inserted, as wave information.

The state change result acquisition module 233 acquires an initial state image and a subsequent state image, which is an image after wave energy is emitted, and obtains the path along which the chip moves through the emission of wave energy as a state change result. The state change result acquisition module 233 obtains and compares the initial state image, which is the image before the wave energy is released, and the subsequent state image, which is the image after the wave energy is released, for machine learning, thereby generating the wave energy with a specific frequency. When is released, the movement path in which direction the chip moves can be obtained and this can be obtained as a result of the state change.

Here, the state change result acquisition module 233 can obtain state change results for various variables by applying various basic variables such as the emission time and emission location of wave energy. The state change result acquisition module 233 acquires the state change result as learning data for performing machine learning in the machine learning result acquisition module 235, which will be described later. As a result, the state change result acquisition module 233 performs the preset number of times. The above state change results can be obtained.

The machine learning result acquisition module 235 is configured to generate a vector path, which is the movement information of the chip, using the state change result, and obtain the learning result by performing repeated learning a preset number of times. Here, the preset number of times may be the number of state change results obtained by the state change result acquisition module 233 described above.

The machine learning result acquisition module 235 may obtain the final learning result by applying a correction variable to the simulation learning result to obtain the learning result. The machine learning result acquisition module 235 acquires simulation learning results, which are learning results about the movement of the chip using wave energy of a specific frequency obtained through simulation, and applies correction variables obtained externally to the obtained simulation learning results. can do. Correction variables are variables involved in the movement of the chip when the corresponding operation is actually performed in reality, and can be obtained through actual testing in reality, for example. When the machine learning result acquisition module 235 obtains the final learning result by applying the correction variable to the simulation learning result, it can set the obtained final learning result as the learning result. In addition, the machine learning result acquisition module 235 of the present invention can perform machine learning using reinforcement learning, for example.

Meanwhile, in one embodiment of the present invention, when the learning result is obtained, the self-arrangement performing unit 25 is configured to perform self-arrangement of the unaligned chip using the learning result. The self-arrangement performing unit 25 uses the learning results to generate wave energy so that the unaligned chip can be located in the non-arranged area, and through this, it participates in the movement of the non-arranged chip, so that the non-arranged chip can self-register in the non-arranged area. It is formed so that it can be arranged. Figure 5 shows a specific block diagram of this self-arrangement performing unit 25, and the self-arranging performing unit 25 includes a dictionary information processing module 251, a self-arranging performing module 253, and a self-arranging verification module ( 255).

The dictionary information processing module 251 is configured to obtain image analysis information and learning results, which are dictionary information for performing self-arrangement, and divide the image based on a preset standard to obtain at least one divided area. When the preliminary information processing module 251 obtains image analysis information, it classifies the substrates in the image according to preset standards. As an example, the dictionary information processing module 251 may divide the substrate into a total of four quadrant areas by using vertical and horizontal axes, respectively.

The self-arrangement module 253 is configured to determine priorities for divided areas using preset conditions and output wave energy to perform self-arrangement for each priority. The self-arrangement module 253 determines the priorities of the areas classified in the dictionary information processing module 251. Priorities use preset conditions and, for example, may be set in order from left to right or top to bottom. When the priority for each region is determined, the self-arrangement module 253 can output wave energy of a specific frequency using the learning results to arrange the unarranged chips into the unarranged portions present in the high-priority region. . Once the arrangement for the high-priority area is completed, the self-arrangement module 253 can proceed with the arrangement for the next high-priority area and repeat this to perform self-arrangement of the unarranged chips for the entire substrate. there is.

The self-arrangement verification module 255 is configured to verify that there are no unaligned portions using image analysis information when self-arrangement is completed. When self-arrangement for all areas is completed in the self-arrangement performing module 253, the self-array verification module 255 according to an embodiment of the present invention obtains the alignment completed image analysis information through the image information processing unit 21. , it is possible to verify that there are no unaligned areas on the substrate through the array completed image analysis information. As a result of the verification, if there is an unaligned portion, the self-arrangement verification module 255 determines that the self-arrangement has not been completed and can perform the self-arrangement again.

3) LED chip fluid self-arrangement method using machine learning

Meanwhile, Figures 6 to 9 show an LED chip fluid self-arrangement method using machine learning according to an embodiment of the present invention. Figure 6 is a flowchart showing a method of self-arranging LED chip fluid using machine learning according to an embodiment of the present invention, Figure 7 is a flowchart showing step S21 of Figure 6, and Figure 8 is a flowchart showing step S23 of Figure 6. , and FIG. 9 is a flowchart showing step S25 of FIG. 6. Hereinafter, the LED chip fluid self-arrangement method using machine learning according to an embodiment of the present invention will be described in detail using FIGS. 6 to 9. In addition, for convenience of description of the present invention, the present invention will be described below using FIGS. 2 to 5, but this does not limit the present invention.

Referring to FIG. 6, the LED chip fluid self-arranging method 20 using machine learning according to an embodiment of the present invention includes processing image information (S21), obtaining machine learning results (S23), and self-arranging. It is formed to include a step of performing arrangement (S25).

In the image information processing step (S21), an image including the upper surface of the substrate where the chip is to be inserted and the chip is acquired using the image information processing unit, and the acquired image is processed to determine the image of the upper surface where the chip is not arranged. It is formed to obtain image analysis information, which is information about the array area and the unarranged chips among the chips that are not arranged on the substrate. The step of processing image information (S21) confirms the position of the substrate on which the chip is not aligned, and obtains the position of the non-arranged chip. In the step of performing self-arrangement (S25), which will be described later, the unaligned chip is placed in the non-arranged area. Basic information for arrangement can be obtained. To this end, the step of processing image information (S21) according to an embodiment of the present invention is formed to include a step of acquiring image information (S211) and a step of analyzing the image information (S213), as shown in FIG. It can be.

The step of acquiring image information (S211) is to obtain an image including the upper surface and the chip, and the step of analyzing the image information (S213) analyzes the acquired image to obtain information about the unarranged region and the unarranged chip. In-video analysis information can be obtained. The step of analyzing image information (S213) uses DNN to obtain image analysis information, and more specifically, the YOLO algorithm can be used. In one embodiment of the present invention, YOLOv5 is used, but it is not necessarily limited thereto and various machine learning algorithms that can be used for image analysis can be used.

Meanwhile, the step S21 of processing image information according to an embodiment of the present invention may further include a step S251 of processing arrangement information. In the step of processing the array information (S251), a preset number of nearby unaligned chips are matched for each unaligned region, and defective chips are selected among the unaligned chips using pre-learned results of the image analysis results for the chips. You can. In addition, the step of processing the arrangement information (S251) may exclude the selected defective chip from matching to prevent the defective chip from being placed in an unarranged area.

On the other hand, when image analysis information is acquired, the LED chip fluid self-arrangement method 20 using machine learning according to an embodiment of the present invention uses the step of obtaining machine learning results (S23) to obtain machine learning results. is formed The step (S23) of obtaining machine learning results according to an embodiment of the present invention uses a preset machine learning algorithm to determine the movement characteristics of the chip inside the fluid according to the wave energy applied to the chip inserted inside the preset fluid. It is formed to obtain learning results by learning from the machine learning result acquisition unit. To this end, the step of acquiring the machine learning results of the present invention (S23) includes acquiring basic information (S231), obtaining the state change result (S233), and obtaining the machine learning result, as shown in FIG. It may be formed to include step S235.

The step of acquiring basic information (S231) is designed to acquire basic information for performing machine learning. Basic information for performing machine learning may be fluid information and wave information. In the present invention, the substrate and the unaligned chip are formed to be inserted into a specific fluid in order to place the unaligned chip on the non-aligned portion of the substrate. Therefore, the step of acquiring the basic information of the present invention (S231) is the basic information for obtaining the learning result in the step of acquiring the machine learning results described later (S235) by acquiring fluid information, which is information about the fluid into which the substrate is inserted. can be obtained.

In addition, in the present invention, wave energy is used to control the movement of the chip, and more specifically, the movement of the chip is controlled by changing the frequency of the wave energy. Therefore, the step of acquiring basic information (S231) is designed to acquire frequency information, which is a specific characteristic of wave energy provided to the fluid into which the substrate is inserted, as wave information.

In the step of acquiring the state change result (S233), the initial state image and the subsequent state image, which are the images after the wave energy is emitted, can be acquired, and the path along which the chip moves through the emission of the wave energy can be obtained as a state change result. there is. The step of acquiring the state change result (S233) is to obtain and compare the initial state image, which is the image before the wave energy is emitted, and the later state image, which is the image after the wave energy is emitted, for machine learning, respectively, so that the state has a specific frequency. When wave energy is emitted, the movement path in which direction the chip moves can be obtained and this can be obtained as a result of the state change.

Here, in the step of obtaining the state change result (S233), the state change result for various variables can be obtained by applying various basic variables such as the emission time and emission location of the wave energy. The step of obtaining the state change result (S233) is to obtain the state change result as learning data for performing machine learning in the step of acquiring the machine learning result (S235), which will be described later, and thereby obtaining the state change result ( S233) can obtain the result of a state change more than a preset number of times.

The step of acquiring the machine learning result (S235) is to generate a vector path, which is the movement information of the chip, using the state change result, and obtain the learning result by performing repeated learning a preset number of times. Here, the preset number of times may be the number of state change results obtained in the step of obtaining the state change result described above (S233).

In the step of acquiring the machine learning result (S235), the final learning result may be obtained by applying a correction variable to the simulation learning result to obtain the learning result. In the step of acquiring machine learning results (S235), simulation learning results, which are learning results about the movement of the chip using wave energy of a specific frequency obtained through simulation, are acquired, and correction variables acquired externally are added to the obtained simulation learning results. can be applied. Correction variables are variables involved in the movement of the chip when the corresponding operation is actually performed in reality, and can be obtained through actual testing in reality, for example. In the step of acquiring the machine learning result (S235), when the final learning result is obtained by applying the correction variable to the simulation learning result, the obtained final learning result can be set as the learning result. In addition, the step of obtaining the machine learning results of the present invention (S235) can be performed by using reinforcement learning, for example.

Meanwhile, in one embodiment of the present invention, when the learning result is obtained, the self-arrangement step (S25) is configured to perform self-arrangement of the unaligned chip using the learning result. The step of performing self-arrangement (S25) uses a self-arrangement unit to generate wave energy using the learning results so that the unarranged chip can be located in the non-arranged area, and through this, it participates in the movement of the unarranged chip and An array chip is formed so that it can be self-aligned in an unaligned area. Figure 9 shows a specific block diagram of the self-arrangement step (S25), which includes the self-arrangement step (S25), the dictionary information processing step (S251), and the self-arrangement step (S251). It is formed to include a step of verifying the self-arrangement (S253) and a step of verifying the self-arrangement (S255).

The dictionary information processing step (S251) is formed to obtain image analysis information and learning results, which are dictionary information for performing self-arrangement, respectively, and to obtain at least one divided area by dividing the image based on a preset standard. In the preliminary information processing step (S251), when image analysis information is obtained, the substrates in the image are classified according to preset standards. For example, in the step of processing prior information (S251), the substrate can be divided into a total of four quadrant areas by dividing the substrate using vertical and horizontal axes, respectively.

The step of performing self-arrangement (S253) is configured to determine priorities for divided areas using preset conditions and output wave energy for performing self-arrangement for each priority. The self-arranging step (S253) determines the priorities of the areas divided in the dictionary information processing step (S251). Priorities use preset conditions and, for example, may be set in order from left to right or top to bottom. In the step of performing self-arrangement (S253), once the priority for each region is determined, wave energy of a specific frequency is output using the learning result to arrange the unarranged chips into the unarranged portions present in the high-priority region. You can. When the arrangement of the high-priority area is completed, the self-arrangement step (S253) proceeds with the arrangement of the next high-priority area and repeats this to perform self-arrangement of the unarranged chips for the entire substrate. can do.

The step of verifying self-arrangement (S255) is performed to verify that there are no unaligned portions using image analysis information once self-arrangement is completed. When the self-arrangement for all areas is completed in the self-arrangement step (S253), the self-arrangement verification step (S255) according to an embodiment of the present invention is performed through the image information processing step (S21). Completed image analysis information can be obtained, and it can be verified that there are no unaligned areas on the substrate through the arrayed image analysis information. As a result of the verification, if there is an unaligned portion, the self-arrangement verification step (S255) determines that the self-arrangement is not completed, and self-arrangement can be performed again.

4) Wave energy frequency

A plurality of frequencies of wave energy are provided according to preset conditions. At this time, the result of vibration may be formed to take a specific form depending on the frequency provided by the wave energy. Figure 10 shows a specific form formed through the wave energy of a specific frequency band when wave energy of a specific frequency band is provided. Figures 10a to 10d show the form in which particles placed on the upper plate gather when wave energy with wave numbers of 13.7, 18.3, 22.8, and 27.4 is provided, respectively. When identical particles are randomly placed on the top plate, when wave energy of a specific frequency is supplied, each particle moves from its initial position to create a specific shape.

Although one embodiment of the present invention has been described above, the spirit of the present invention is not limited to the embodiment presented in the present specification, and those skilled in the art who understand the spirit of the present invention can add components within the scope of the same spirit. , other embodiments can be easily proposed by change, deletion, addition, etc., but this will also be said to be within the scope of the present invention.

1: LED chip fluid self-arranging device using machine learning
11: Video recording module
13: Chip array module
15: Wave energy generation module
17: Chip array control module
2: LED chip fluid self-arranging system using machine learning
21: Image information processing unit
23: Machine learning result acquisition unit
25: Self-arrangement execution unit
211: Image information acquisition module
213: Video information analysis module
231: Basic information acquisition module
233: State change result acquisition module
235: Machine learning result acquisition module
251: Dictionary information processing module
253: self-arrangement module
255: Self-alignment verification module

Claims (15)

An image capturing module that captures an image including a substrate and a chip to obtain image information including information about an array position, which is a position where the chip should be arranged on the substrate;
A chip located at the bottom of the image capture module, formed in the form of a polyhedron with an open upper surface to contain the substrate and the chip, and an empty interior, and filled with a predetermined volume of fluid for movement of the chip. array module;
a wave energy generation module formed to contact the lower surface of the chip array module and generating wave energy and transmitting the wave energy to the fluid;
The image information and the wave energy are used to determine the direction of movement of the chip, and to move the unarranged chip, which is the unarranged chip among the chips, to the unarranged portion, which is the portion where the chip is not aligned, among the array positions. LED chip fluid self-arranging device using machine learning, including a chip array control module for determining the characteristics of wave energy. According to clause 1,
The characteristics of the wave energy include frequency information corresponding to the moving direction of the chip as a result of machine learning using previously acquired learning data,
The chip array control module,
Generating a vector path through which the non-aligned chip can move to the non-aligned portion,
An LED chip fluid self-arranging device using machine learning to determine the frequency information as a characteristic of the wave energy so that the non-aligned chip moves to the non-aligned area. According to clause 1,
The unaligned chip moving to the non-aligned area,
An LED chip fluid self-arranging device using machine learning, wherein the unarranged chip is located closest to the unarranged portion obtained by analyzing the image information. Obtain an image including the chip and the upper surface of the substrate where the chip should be inserted, and process the acquired image to determine the unaligned portion of the upper surface where the chip is not aligned and the portion of the chip that is not aligned on the substrate. an image information processing unit that acquires image analysis information, which is information about unarranged chips;
A machine learning result acquisition unit that learns the movement characteristics of the chip within the fluid according to wave energy applied to the chip inserted into the preset fluid using a preset machine learning algorithm and obtains a learning result; and
An LED chip using machine learning that includes a self-arranging unit that generates the wave energy using the learning result to self-arrange the unarranged chip so that the unarranged chip can be located in the non-arranged area. Fluid self-arranging system. According to clause 4,
The image information processing unit,
an image information acquisition module that acquires the image including the upper surface and the chip; and
An image information analysis module that analyzes the image and acquires the image analysis information, which is information about the non-arranged portion and the non-arranged chip. LED chip fluid self-arrangement system using machine learning, including a. According to clause 5,
The image information processing unit,
Matches a preset number of nearby unaligned chips for each unaligned region, selects defective chips among the unaligned chips using pre-learned results of image analysis results for the chips, and excludes them from the one-to-one matching. An LED chip fluid self-array system using machine learning, further comprising an array information processing module that performs processing. According to clause 4,
The machine learning result acquisition unit,
A basic information acquisition module that acquires fluid information, which is information about the preset fluid, which is basic information for performing the machine learning, and wave information, which is a specific characteristic of the wave energy emitted to the preset fluid;
A state change result acquisition module that acquires an initial state image and a subsequent state image, which is an image after the wave energy is emitted, among the images, and obtains a path along which the chip moves through the emission of the wave energy as a state change result; and
A machine learning result acquisition module that generates a vector path, which is the movement information of the chip, using the state change result, and obtains the learning result by performing repeated learning a preset number of times; LED chip fluid using machine learning including a Self-arranging system. According to clause 7,
The specific characteristic of the wave energy is frequency,
The machine learning result acquisition module is,
An LED chip fluid self-arranging system using machine learning in which a final learning result is obtained by applying a correction variable obtained externally to the simulation learning result obtained through simulation, and the final learning result is obtained as the learning result. According to clause 4,
The self-arranging unit,
a dictionary information processing module that acquires the image analysis information and the learning results, and divides the image according to a preset standard to obtain at least one divided area;
a self-arrangement performing module that determines priorities of the divided areas using preset conditions and outputs the wave energy to perform self-arrangement according to the priorities;
When the self-arrangement is completed, a self-arrangement verification module that uses the image analysis information to verify whether the unaligned portion does not exist. An LED chip fluid self-arrangement system using machine learning including a. An image including the chip and the upper surface of the substrate where the chip should be inserted is obtained using an image information processing unit, and the acquired image is processed to determine the unarranged portion of the upper surface where the chip is not arranged and the image including the chip. Obtaining image analysis information, which is information about unarranged chips not arranged on the substrate;
Using a machine learning result acquisition unit, the movement characteristics of the chip within the fluid are learned using a preset machine learning algorithm according to the wave energy applied to the chip inserted inside the preset fluid, and the learning results are obtained. step; and
Using machine learning including; generating the wave energy in a self-arrangement performing unit using the learning result so that the non-arranged chip can be located in the non-arranged area; LED chip fluid self-arrangement method. According to clause 10,
The step of acquiring the image analysis information is,
acquiring the image including the upper surface and the chip; and
Analyzing the image to obtain the image analysis information, which is information about the non-arranged portion and the non-arranged chip. LED chip fluid self-arrangement method using machine learning comprising a. According to clause 11,
The step of acquiring the image analysis information is,
Matches a preset number of nearby unaligned chips for each unaligned region, selects defective chips among the unaligned chips using pre-learned results of image analysis results for the chips, and excludes them from the one-to-one matching. An LED chip fluid self-arranging method using machine learning, further comprising: performing processing. According to clause 10,
The step of learning the movement characteristics of the chip using a preset machine learning algorithm and obtaining the learning result is,
Obtaining fluid information, which is information about the preset fluid, which is basic information for performing the machine learning, and wave information, which is a specific characteristic of the wave energy emitted to the preset fluid;
Obtaining an initial state image and a subsequent state image, which is an image after the wave energy is emitted, among the images, and obtaining a path along which the chip moves through the emission of the wave energy as a result of a state change; and
An LED chip fluid self-arranging method using machine learning comprising: generating a vector path, which is movement information of the chip, using the state change result, and performing repeated learning a preset number of times to obtain the learning result. According to clause 13,
The specific characteristic of the wave energy is frequency,
The step of obtaining the learning result is,
An LED chip fluid self-arrangement method using machine learning in which a final learning result is obtained by applying a correction variable obtained externally to the simulation learning result obtained through simulation, and the final learning result is obtained as the learning result. According to clause 10,
The step of performing self-arrangement of the unaligned chip,
Obtaining the image analysis information and the learning result, dividing the image based on a preset standard to obtain at least one divided area;
determining priorities of the divided areas using preset conditions and outputting the wave energy to perform self-arrangement according to the priorities;
When the self-arrangement is completed, verifying whether the unaligned portion does not exist using the image analysis information. LED chip fluid self-arrangement method using machine learning comprising a.