CN112693904B

CN112693904B - Imaging system of chip tray stacking detection device and chip tray stacking detection device

Info

Publication number: CN112693904B
Application number: CN202011428712.5A
Authority: CN
Inventors: 储飞; 宋存霄
Original assignee: Intel Products Chengdu Co Ltd; Intel Corp
Current assignee: Intel Products Chengdu Co Ltd; Intel Corp
Priority date: 2020-12-07
Filing date: 2020-12-07
Publication date: 2022-11-08
Anticipated expiration: 2040-12-07
Also published as: CN112693904A

Abstract

The invention provides an imaging system of a chip tray stacking detection device and the chip tray stacking detection device. The chip tray stack includes a plurality of stacked chip trays, each chip tray containing a plurality of chips located in respective tray pockets. The imaging system includes: at least one conveying mechanism including two endless conveyor belts arranged in parallel, on which the chip trays are stacked to be conveyed; at least one rotation mechanism; and at least one imaging device for imaging four sides of the stack of chip trays; wherein the orientation of the four sides of the stack of chip trays can be changed by rotation of at least one rotation mechanism so that each of the four sides is imaged by the imaging device. According to the invention, the chip tray can be quickly, efficiently, accurately and reliably stacked for imaging.

Description

Imaging system of chip tray stacking detection device and chip tray stacking detection device

Technical Field

The present invention relates to an imaging system of a chip tray stack inspection apparatus and a chip tray stack inspection apparatus including such an imaging system.

Background

In a back-end process of a semiconductor factory, a plurality of chip trays each containing a plurality of chips are stacked together, the stacked plurality of chip trays are supported at the bottom by a base and covered at the top by a top cover, and they are tightly bundled together by a plastic strapping tape in a packaging machine to form a chip tray stack and then shipped to a user. During the packaging and bundling process, the chips in the trays sometimes come out of the designed pocket locations and are thus compressed by the adjacent trays, resulting in damage to the chips. At the operating site, visual inspection is usually performed by an operator, but not only is the visual inspection inefficient, but also it is often difficult to find damaged chips, thus resulting in damaged chips being shipped directly to the user.

The present inventors propose a method of imaging a stack of chip trays and analyzing the acquired images to determine if there are damaged chips, but how to image a stack of chip trays quickly, efficiently, accurately, and reliably is a problem that must be faced.

For this reason, it is necessary to design a system capable of automatically imaging a chip tray stack.

Disclosure of Invention

The present invention is directed to overcoming at least one of the above-mentioned disadvantages of the prior art, and to providing an imaging system of a chip tray stacking detection apparatus.

According to an aspect of the present invention, there is provided an imaging system of a chip tray stack inspection apparatus, a chip tray stack including a plurality of chip trays stacked together, each chip tray accommodating a plurality of chips located in corresponding tray pockets, the imaging system comprising:

at least one conveying mechanism including two endless conveyor belts arranged in parallel, on which the chip trays are stacked to be conveyed;

at least one rotation mechanism; and

at least one imaging device for imaging four sides of the stack of chip trays;

wherein the orientation of the four sides of the stack of chip trays can be changed by rotation of at least one rotation mechanism so that each of the four sides is imaged by the imaging device.

Preferably, the imaging device is a camera.

Preferably, the imaging system comprises:

the first conveying mechanism comprises two first annular conveying belts arranged in parallel;

a first lifting mechanism disposed between the first endless conveyor belts;

a second conveying mechanism mounted on the first lifting mechanism, the second conveying mechanism including two second endless conveyor belts arranged in parallel, the second endless conveyor belts being arranged in a direction substantially perpendicular to the first endless conveyor belts;

a third conveyor mechanism disposed outside one of the first endless conveyor belts, the third conveyor mechanism including two third endless conveyor belts disposed in parallel, the third endless conveyor belts being disposed in a direction substantially perpendicular to the first endless conveyor belts;

a rotary lifting mechanism disposed at an end of the third conveying mechanism remote from the second conveying mechanism, the rotary lifting mechanism including a second lifting mechanism rotatably supported, and a tray mounted to a top of the second lifting mechanism and for receiving and supporting the stack of chip trays; and

and a first imaging device disposed in an extending direction of the third endless belt and having a lens facing the second conveying mechanism and/or the third conveying mechanism.

Preferably, the imaging system further comprises:

and a second imaging device disposed outside the extending direction of the third endless belt and having a lens facing the third conveying mechanism.

Preferably, the imaging system further comprises:

a fourth conveyor mechanism disposed downstream of the rotary lift mechanism, the fourth conveyor mechanism including two fourth endless conveyor belts disposed in parallel, the fourth endless conveyor belts being generally aligned with the third endless conveyor belts.

Preferably, a baffle is arranged on the first lifting mechanism and is close to the downstream side of the second conveying mechanism, the baffle is slightly higher than the level of the second annular conveyor belt in the vertical direction, and the first lifting mechanism is lifted to move the second annular conveyor belt and the baffle to three positions: in the first position, the second endless conveyor is below the plane of the first endless conveyor, but the baffle extends above the plane of the first endless conveyor; in the second position, the second endless conveyor and the baffle are both elevated above the plane of the first endless conveyor; in a third position, both the second endless conveyor and the baffle are lowered below the plane of the first endless conveyor.

Preferably, the imaging system further comprises:

a stop mechanism disposed adjacent the input side of the first conveyor mechanism and between the first endless conveyors, the stop mechanism being movable between a blocking position projecting above a plane of the first endless conveyors and a release position moving below the plane of the first endless conveyors.

Preferably, the imaging system further comprises:

a chip tray stacking orientation correcting plate disposed near an input side of the first conveying mechanism and outside one of the first endless belts, the orientation correcting plate being movable in a direction substantially perpendicular to an extending direction of the first endless belts.

Preferably, the outer sides of the first endless conveyor belt and the third endless conveyor belt are respectively provided with a guide strip which is slightly higher than the plane of the corresponding endless conveyor belt.

Preferably, the rotary lifting mechanism rotates 90 degrees or 180 degrees at a time.

Preferably, the brackets of the rotary lifting mechanism are cross-shaped, and when the brackets are lowered to a position below the plane of the third endless conveyor, one branch of the cross-shaped brackets is located in a space between the third endless conveyor.

Preferably, the second lifting mechanism of the rotary lifting mechanism is an electric cylinder, a base of the second lifting mechanism is rotatably mounted on the base plate through two thrust bearings which are concentrically arranged, lower rings of the two bearings are respectively communicated with a power supply, and upper rings of the two bearings are respectively communicated with a motor of the electric cylinder, so that the two bearings also play a role of a conductive conductor.

Preferably, the imaging system comprises:

the rotary platform rotates for 90 degrees each time;

the conveying mechanism is arranged on the rotating platform and comprises two annular conveying belts which are arranged in parallel; and

and the lens faces the imaging device arranged on the conveying mechanism.

Preferably, the imaging system further comprises:

chip tray stacking orientation adjusting push rods provided on the rotary platform and respectively located outside the endless belt, each of the chip tray stacking orientation adjusting push rods being capable of moving toward each other in a closed and raised manner or moving away from each other in a separated and lowered manner.

Preferably, the imaging system further comprises:

two chip tray stack gripper heads disposed on the rotary platform and spaced apart between the endless conveyor belts, each of the chip tray stack gripper heads being raisable to a position above the plane of the endless conveyor belts and movable towards each other to grip the chip tray stack and being lowerable to avoid interference with imaging of the chip tray stack.

Preferably, the imaging system further comprises:

and the other conveying mechanism is arranged at the upstream of the rotating platform, and the bearing surfaces of the two parallel annular conveying belts of the other conveying mechanism are approximately flush with the bearing surfaces of the annular conveying belts of the conveying mechanism.

Preferably, the imaging system further comprises:

a further conveyor mechanism disposed downstream of the rotary platform, the bearing surfaces of the two parallel endless belts of the further conveyor mechanism being substantially flush with the bearing surfaces of the endless belts of the conveyor mechanism.

Preferably, a black background is used in imaging the four sides of the chip tray stack, and the imaging system further comprises an area array light source emitting white light.

Preferably, the area array light source has a luminous flux of about 400 lumens.

Preferably, the imaging device is mounted such that the imaging device lens is in the range of 60 to 80 degrees from the side to be photographed.

According to another aspect of the present invention, there is provided a chip tray stack detecting apparatus including:

an imaging system as described above; and

and the image processing system is used for processing the images shot by the imaging system.

According to the imaging system of the chip tray stacking detection device, the chip tray stack can be imaged quickly, efficiently, accurately and reliably.

Drawings

Fig. 1 schematically shows a chip tray stack detecting apparatus according to a first embodiment of the present invention;

FIG. 2 schematically illustrates, in a top side perspective view, an imaging system of the chip tray stack inspection device of FIG. 1;

FIG. 3 schematically illustrates the imaging system of FIG. 2 in another top side perspective view;

FIG. 4 schematically illustrates the imaging system of FIG. 2 in a top view;

FIG. 5 schematically illustrates the imaging system of FIG. 2 in a side view;

FIG. 6 schematically illustrates, in partial perspective view, a first conveyor and a second conveyor of the imaging system of FIG. 2;

FIG. 7 schematically illustrates, in partial perspective view, a rotary lift mechanism of the imaging system of FIG. 2;

fig. 8 schematically shows a detail of the rotating part of the rotary lifting mechanism of fig. 7 in a partially cut-away perspective view;

FIG. 9 schematically illustrates, in side view, a second guide strip outside of the third endless conveyor of the imaging system of FIG. 2;

fig. 10 schematically shows a chip tray stack detecting apparatus according to a second embodiment of the present invention;

fig. 11 schematically shows the chip tray stack detecting apparatus shown in fig. 10 in a partial perspective view;

FIG. 12 schematically illustrates the chip tray stack detecting apparatus shown in FIG. 10 in a top view;

FIG. 13 schematically shows a chip tray stack;

fig. 14 schematically shows the positional relationship of the chip tray stack, the light source, and the imaging device;

FIG. 15 is a view schematically showing a flow of processing an image by an image processing system of the chip tray stack detecting apparatus according to the present invention;

fig. 16 schematically shows an image of one side of a chip tray stack taken by an imaging device;

FIG. 17 schematically shows an interface for pre-processing an image;

FIG. 18 schematically shows an image profile obtained by derivation of the pre-processed image in the X and Y directions;

FIG. 19 schematically shows a straight line fit when no chip is removed from a tray pocket;

FIG. 20 schematically shows a line fit with a chip removed from a tray pocket; and

fig. 21 schematically shows the difference between the Y values of the center coordinates of the adjacent sub-areas.

Detailed Description

Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, and it should be understood by those skilled in the art that these exemplary embodiments do not imply any limitation on the present invention.

Fig. 1 schematically shows a chip tray stack detecting apparatus 1 according to a first embodiment of the present invention. As shown in fig. 1, a chip tray stack TS from the packaging machine enters from an entrance 3 of the chip tray stack inspection apparatus 1, and four sides (i.e., surfaces reflecting the stacking state between adjacent chip trays) of the chip tray stack TS exposed front, back, left, and right are photographed and imaged by a chip tray stack imaging system located inside the chip tray stack inspection apparatus 1. After the imaging of the chip tray stack TS is completed, the chip tray stack TS is output from the outlet 5 opposite to the inlet 3.

As shown in fig. 1, the chip tray stack detection apparatus 1 includes a housing 7, and the housing 7, in addition to enclosing the imaging system and the image processing system of the chip tray stack detection apparatus located inside, also protects the chip tray stack TS from external ambient light during imaging, thereby improving imaging quality. A display screen 9 is provided on one side of the housing 7 for inputting a control instruction to a control unit of the chip tray stack detecting device 1 located inside, displaying an operation state of the chip tray stack detecting device 1, and displaying a detection result of the chip tray stack detecting device. An alarm device 11 is also provided on the housing 7. When the image processing system analyzes the image of the chip tray stack TS acquired by the imaging system and determines that there is a damaged chip, the chip tray stack detection apparatus 1 notifies the operator of processing, e.g., taking out, the chip tray stack TS including the damaged chip by, e.g., mail or short message, or the like, so as to avoid sending it directly to the customer. If the operator takes no action, the chip tray stack detecting device will stop the packaging machine, while the alarm device 11 gives an alarm in the form of a sound or flashing light to remind the operator to take out the chip tray stack TS having the damaged chip. After the operator has responded to the alarm, the packaging machine and chip tray stack detection device will resume normal operation by simply depressing the start button again.

Fig. 2 schematically shows an imaging system of the chip tray stack detection apparatus shown in fig. 1 in a top side perspective view, fig. 3 schematically shows the imaging system shown in fig. 2 in another top side perspective view, fig. 4 schematically shows the imaging system shown in fig. 2 in a top view, fig. 5 schematically shows the imaging system shown in fig. 2 in a side view, fig. 6 schematically shows a first conveyance mechanism and a second conveyance mechanism of the imaging system shown in fig. 2 in a partial perspective view, and fig. 7 schematically shows a rotary elevating mechanism of the imaging system shown in fig. 2 in a partial perspective view. As shown in fig. 2 to 7, the image forming system 20 according to the first embodiment of the present invention includes a frame 21 and a first conveyance mechanism 23 provided on the frame 21. The first conveying mechanism 23 includes two first

endless belts

23a,23b arranged in parallel, and the first

endless belts

23a,23b are driven by pulleys to run synchronously, which are in turn driven by a motor to rotate. Outside the two first

endless belts

23a,23b arranged in parallel,

first guide bars

24a, 24b are provided, respectively, which

first guide bars

24a, 24b are slightly higher than the plane (support surface) of the first

endless belts

23a,23b, so as to guide the chip tray stack TS as it is conveyed on the first

endless belts

23a, 23b.

A first elevating mechanism 25 is provided between the two first

endless belts

23a and 23b provided in parallel, and the first elevating mechanism 25 can be elevated by a driving device such as an air cylinder, a hydraulic cylinder, an electric cylinder, or a lead screw. A second conveyor mechanism 27 is mounted on the first elevating mechanism 25, and the second conveyor mechanism 27 also includes two second

endless belts

27a,27b arranged in parallel. Second

endless belts

27a,27b are positioned generally perpendicular to first

endless belts

23a,23b and are also driven by pulleys that are rotated by the motor. A baffle 29 is provided on the first elevating mechanism 25 immediately downstream of the second conveying mechanism 27, the baffle 29 being slightly higher in the vertical direction than the level of the planes of the second

endless belts

27a,27b. The first elevating mechanism 25 can be elevated to move the second

endless belts

27a,27b and the shutter 29 to three positions: in the first position, second

endless belts

27a,27b are below the plane of first

endless belts

23a,23b, but baffles 29 extend above the plane of first

endless belts

23a, 23b; in the second position, the second

endless belts

27a,27b and the shutter 29 are both raised above the first guide strip 24b adjacent to the first imaging device (described below); in the third position, second

endless conveyors

27a,27b and guards 29 are lowered below the plane of first

endless conveyors

23a, 23b.

A stop mechanism 31 is also arranged adjacent to the input side of the first conveyor 23 and between the two first

endless belts

23a,23b arranged in parallel, which stop mechanism 31 is connected to the drive means by a link so as to be movable between a blocking position protruding above the plane of the first

endless belts

23a,23b and a release position moved below the plane of the first

endless belts

23a, 23b. When the stopper mechanism 31 moves to the blocking position above the plane of the first

endless belts

23a,23b, the chip tray stack TS from the packing machine may be prevented from entering the chip tray stack detecting apparatus 1; when the stopper mechanism 31 moves to the release position below the plane of the first

endless belts

23a,23b, the chip tray stack TS from the packing machine can continue to enter the chip tray stack detecting apparatus 1 for detection.

Near the input side of the first transport mechanism 23 and outside one of the two first

endless belts

23a,23b arranged in parallel, a chip tray stack orientation correcting plate 33 may be provided, and the orientation correcting plate 33 may be moved by a driving device in a direction substantially perpendicular to the extending direction of the first

endless belts

23a,23b so as to adjust the chip tray stack TS to a proper position required for imaging. A surface of the orientation correcting plate 33 that contacts the chip tray stack TS may be provided with a material such as foam or rubber to avoid damage to the chip tray stack TS when the orientation correcting plate 33 abuts against the chip tray stack TS. Of course, it is also possible that the orientation calibration plate 33 itself is made of a material such as foam or rubber.

The image forming system 20 according to the first embodiment of the present invention further includes a third transport mechanism 35 outside one of the two first

endless belt conveyors

23a,23b arranged in parallel. The third conveying mechanism 35 also includes two third

endless conveyor belts

35a,35b arranged in parallel. Third

endless conveyors

35a,35b are also disposed in a generally perpendicular orientation to the first

endless conveyors

23a,23b and are generally aligned with second

endless conveyors

27a,27b. The third

endless belts

35a,35b are also driven by pulleys that are rotated by a motor.

Second guide bars

37a, 37b are provided on the outer sides of the two third

endless conveyor belts

35a,35b arranged in parallel, respectively, the

second guide bars

37a, 37b being slightly higher than the plane of the third

endless conveyor belts

35a,35b so as to guide the chip tray stack TS when the chip tray stack TS is conveyed on the third

endless conveyor belts

35a,35b. In order to facilitate the guiding of the chip tray stack TS into position between the second guide strips 37a, 37b, the free ends of the second guide strips 37a, 37b may be formed as rounded edges, so that the two second guide strips 37a, 37b define a flared opening at the free ends.

A rotary lifting mechanism 39 is provided at an end of the third conveyance mechanism 35 remote from the second conveyance mechanism 27. The rotary elevating mechanism 39 includes a second elevating mechanism 43 rotatably supported by a bearing on a base plate 41 mounted on the frame 21. The base 45 of the second lifting mechanism 43 is fixedly connected with a rotating shaft 47 of a motor (not shown), so that the second lifting mechanism 43 is driven to rotate by the operation of the motor. The rotary lift mechanism 39 also includes a carriage 49 mounted to the top of the second lift mechanism 43 and for receiving and supporting the chip tray stack TS. The second elevating mechanism 43 can be elevated to move the carriage 49 to a position above the

second guide bars

37a, 37b and to a position below the plane of the third

endless conveyor belts

35a,35b. The bracket 49 is preferably cross-shaped to securely support the chip tray stack TS. When the carriage 49 is lowered to a position below the plane of the third

endless conveyors

35a,35b, one branch of the cross-shaped carriage 49 is located in the space between the third

endless conveyors

35a,35b.

In the case where the second elevating mechanism 43 is an electric cylinder, in order to prevent the wire for supplying electricity to the electric cylinder from being twisted and broken or damaged, two thrust bearings may be concentrically disposed between the base plate 41 and the base 45 of the second elevating mechanism 43, wherein the outer bearing 51 having a larger diameter may function as both a positioning support and a conductive conductor, and the inner bearing 53 having a smaller diameter may function only as a conductive conductor. Thus, as shown in fig. 7 and 8, two terminals 55 (e.g., connected to a power source) may be provided on the substrate 41, two wires 54 (only one wire 54 is shown in fig. 8) connected to the two terminals 55 may pass through the insulating substrate 41 and be connected to the lower rings of the outer bearing 51 and the inner bearing 53, respectively, and two other wires 56 (only one wire 56 is shown in fig. 8) connected to the upper rings of the outer bearing 51 and the inner bearing 53 may pass through the insulating base 45 of the second elevating mechanism 43 and be connected to two other terminals 57 on the insulating base 45. The two other terminals 57 are in turn connected to the electric motor of the electric cylinder by means of wires 58 in order to supply the electric cylinder with electricity. Thus, both the larger diameter outer bearing 51 and the smaller diameter inner bearing 53 can act as electrically conductive conductors to avoid twisting of the wires.

Due to the weight of the chip tray stack TS, in order to ensure that the chip tray stack TS can be reliably conveyed to a position above the tray 49, a fourth conveying mechanism 59 may also be provided in alignment with the third conveying mechanism 35 on the downstream side of the rotary elevating mechanism 39. The fourth transport mechanism 59 also includes two fourth

endless belt conveyors

59a,59b arranged in parallel. The fourth

endless conveyor

59a,59b is generally aligned with the third

endless conveyor

35a,35b. The fourth

endless belts

59a,59b are also driven by pulleys that are rotated by motor drives.

The imaging system 20 according to the first embodiment of the present invention further includes the first imaging device 61 which is mounted on the frame 21 in the extending direction of the third

endless conveyors

35a,35b and the fourth

endless conveyors

59a,59b and in which the lens faces the chip tray stack TS on the second conveying mechanism 27, and the first imaging device 61 is used to image the chip tray stack TS on the second conveying mechanism 27. The imaging system 20 according to the first embodiment of the present invention further includes a second imaging device 63 mounted on the frame 21 outside the extending direction of the third

endless belt

35a,35b and having a lens facing the chip tray stack TS on the third conveying mechanism 35, the second imaging device 63 being for imaging the chip tray stack TS on the third conveying mechanism 35. Depending on the size of the chip tray stack, imaging devices having different resolutions may be selected. For example, for a chip tray stack having a small surface size, an imaging device having a resolution of five million pixels may be selected; for a chip tray stack having a larger surface size, an imaging device having a resolution of ten million pixels may be selected.

The operation of the imaging system 20 according to the first embodiment of the present invention will be described below.

When the stop mechanism 31 is moved to the release position below the plane of the first

endless belts

23a,23b, the chip tray stack TS from the packaging machine enters the chip tray stack detection device 1 through the entrance 3 and is conveyed on the first

endless belts

23a,23b towards the second conveyor mechanism 27. Subsequently, the stopper mechanism 31 moves to the blocking position above the plane of the first

endless belts

23a,23b until the chip tray stack TS that has entered completes all imaging, and then moves to the release position below the plane of the first

endless belts

23a,23b to allow entry of the next chip tray stack TS. In order to image the side surfaces of the stack of chip trays TS clearly, it is preferable to make the center surface of the stack of chip trays TS substantially perpendicular or parallel to the direction in which the first

endless belts

23a,23b extend. The stack of chip trays TS that has just moved onto the first

endless conveyor

23a,23b may be somewhat offset in position relative to the first

endless conveyor

23a, 23b. At this time, the orientation correcting plate 33 is moved toward and abutted against the chip tray stack TS in a direction substantially perpendicular to the extending direction of the first

endless belt

23a,23b so as to adjust the chip tray stack TS to a proper orientation required for imaging. With the operation of the first

endless belts

23a,23b, the chip tray stack TS is moved toward the first elevating mechanism 25. At this point, second

endless belts

27a,27b and baffles 29 are in a first position, i.e., second

endless belts

27a,27b are below the plane of first

endless belts

23a,23b, but baffles 29 extend above the plane of first

endless belts

23a, 23b. As a result, the chip tray stack TS is stopped by the stopper 29 and no longer moves with the first

endless belts

23a, 23b. Subsequently, the first elevating mechanism 25 is raised so that the second

endless belts

27a,27b and the flap 29 are both elevated to the second position. At this time, the chip tray stack TS is lifted by the second

endless belts

27a,27b to be separated from the first

endless belts

23a,23b, and enters the photographing position. The sensor detects the chip tray stack TS entering this position, thereby triggering an illumination source (not shown) and a first imaging device 61 directly opposite the first side of the chip tray stack TS, thereby completing imaging of the first side of the chip tray stack TS. To ensure imaging quality, the first imaging device 61 may be set to start, for example, 1 second later than the illumination source.

After the imaging of the first side of the chip tray stack TS is completed, the second

endless belts

27a,27b operate to convey the chip tray stack TS onto the third

endless belts

35a,35b of the third conveying mechanism 35. When the sensor detects that the chip tray stack TS is moved to a position opposite the second imaging device 63, the third

endless conveyor belts

35a,35b stop running and activate the illumination light source and the second imaging device 63 opposite the second side of the chip tray stack TS, thereby completing imaging of the second side of the chip tray stack TS.

Subsequently, the third

endless conveyors

35a,35b continue to run, and convey the chip tray stack TS above the rotary elevating mechanism 39 together with the fourth

endless conveyors

59a,59b of the fourth conveying mechanism 59. At this time, the second elevating mechanism 43 ascends to lift the chip tray stack TS off the third

endless conveyors

35a,35b and the fourth

endless conveyors

59a,59b by the carriage 49. Next, the motor drives the rotary elevating mechanism 39 to rotate 180 degrees, so that the chip tray stack TS on the carriage 49 also rotates 180 degrees. Then, the second elevating mechanism 43 descends to place the chip tray stack TS on the third

endless conveyors

35a,35b and the fourth

endless conveyors

59a,59b again. The fourth

endless belts

59a,59b and the third

endless belts

35a,35b run in reverse to move the chip tray stack TS again to the second imaging device 63 photographing range. When the sensor again detects that the chip tray stack TS is moved to a position facing the second imaging device 63, the third

endless conveyor belts

35a,35b stop running and activate the illumination light source and the second imaging device 63 facing the third side of the chip tray stack TS, thereby completing imaging of the third side of the chip tray stack TS.

Thereafter, the third

endless conveyors

35a,35b continue to run, so that the chip tray stack TS is conveyed again onto the second

endless conveyors

27a,27b at the second position and stopped by end stoppers, not shown. The sensor again detects the chip tray stack TS entering this position, thereby triggering the illumination source and the first imaging device 61 directly opposite the fourth side of the chip tray stack TS, thereby completing imaging of the fourth side of the chip tray stack TS.

Subsequently, the first elevator mechanism 25 is lowered so that the second

endless belts

27a,27b and the guards 29 are lowered to a third position below the level of the first

endless belts

23a, 23b. As a result, the chip tray stack TS is again placed on the first

endless belts

23a,23b, and is output from the outlet 5 of the chip tray stack detection apparatus 1 with the operation of the first

endless belts

23a, 23b.

The height of the guide strip is typically no higher than the height of the tray of the chip tray stack TS so as not to interfere with the imaging quality. When the chip tray stack TS is again placed on the third

endless belts

35a,35b as the second elevating mechanism 43 descends after rotating 180 degrees, the position of the chip tray stack TS is easily shifted due to inertia. For this, as shown in fig. 9, a height of a second guide bar 37a far from the second image forming device among the

second guide bars

37a, 37b outside the third

endless conveyors

35a,35b may be set higher than a height of a second guide bar 37b near the second image forming device. In this way, the stack of chip trays TS can be oriented against the second guide strip 37a remote from the second imaging device, thereby enabling the second guide strip 37a remote from the second imaging device to have an orientation-adjusting effect on the stack of chip trays TS.

In the first embodiment described above, two image forming devices are provided and the lifting mechanism and the rotary lifting mechanism are provided, but it is to be understood that a plurality of, for example, three or four image forming devices may be provided to omit the lifting mechanism and/or the rotary lifting mechanism and some conveying mechanisms. However, due to the higher cost of the imaging devices, the use of more imaging devices can significantly increase the cost and also significantly increase the size of the tray stack detection apparatus. Of course, it is also possible to provide only the first imaging device, in which case the rotary lifting mechanism can be rotated 90 degrees each time and it is contemplated that the side of the chip tray stack TS is imaged by the first imaging device while it is on the carriage.

A chip tray stack detecting device according to a second embodiment of the present invention will be described below. The chip tray stack detection apparatus according to the second embodiment of the present invention also includes a cover, which, in addition to the imaging system and the image processing system inside the package, protects the chip tray stack TS from external ambient light during imaging, thereby improving imaging quality. A display screen (not shown) may also be provided on one side of the outer cover for inputting a control command to a control unit of the chip tray stack detecting device located inside and displaying an operation state of the chip tray stack detecting device. An alarm device (not shown) may also be provided on the housing.

Fig. 10 schematically shows a chip tray stack detecting apparatus according to a second embodiment of the present invention, fig. 11 schematically shows the chip tray stack detecting apparatus shown in fig. 10 in a partial perspective view, and fig. 12 schematically shows the chip tray stack detecting apparatus shown in fig. 10 in a plan view. As shown in fig. 10 to 12, an imaging system 100 of a chip tray stack detecting apparatus according to a second embodiment of the present invention includes a frame 101 and a first conveying mechanism 103 provided on the frame 101. The first conveying mechanism 103 includes two first endless conveying

belts

103a,103b arranged in parallel, and the first endless conveying

belts

103a,103b are driven to run in synchronism by pulleys which are driven to rotate by a motor. On the outer side of the two first

endless conveyor belts

103a,103b arranged in parallel, there may also be provided a first guide strip, respectively, which is slightly higher than the plane (bearing surface) of the first endless conveyor belts in order to guide the chip tray stack TS from the packaging machine when it is transported on the first endless conveyor belts. As in the first embodiment, a stopper mechanism (not shown) is further provided near the input side of the first conveying mechanism 103 and between the two first

endless conveyor belts

103a,103b arranged in parallel to prevent or allow the chip tray stack TS from the packaging machine from entering the chip tray stack detecting device for detection.

Immediately downstream of the two parallel first

endless conveyor belts

103a,103b, a rotary platform 105 is arranged, which rotary platform 105 can be driven by an electric motor to rotate. A second conveying mechanism 107 is mounted on the rotary platform 105, and the second conveying mechanism 107 also includes two second

endless belts

107a,107b arranged in parallel. The second

endless belts

107a,107b are also driven in synchronism by pulleys which are in turn driven in rotation by an electric motor. The bearing surfaces of the second

endless conveyor belts

107a,107b are substantially flush with the bearing surfaces of the first

endless conveyor belts

103a, 103b.

On the rotating platform 105, outside the second

endless conveyors

107a,107b, chip tray stacking orientation adjusting push rods 109 are respectively provided, each chip tray stacking orientation adjusting push rod 109 being capable of moving toward each other in a folded state and moving upward, or moving away from each other in a separated state and moving downward. When the chip tray stack TS is conveyed from the first conveying mechanism 103 onto the second

endless belts

107a,107b, each chip tray stack orientation adjusting push rod 109 can be folded toward each other and elevationally moved so as to abut against the chip tray stack TS on the opposite sides thereof, so that the chip tray stack TS is adjusted to a proper orientation required for image formation. When it is necessary to image the chip tray stack TS toward the side of the chip tray stack orientation adjusting push rod 109 after the chip tray stack TS is adjusted to a desired orientation, the chip tray stack orientation adjusting push rods 109 may be moved apart and downward away from each other to avoid interference with the imaging of the chip tray stack TS.

Two chip tray stacking gripper heads 111 are provided on the rotary platform 105 at a spacing between the second

endless belts

107a,107b. Each chip tray stack clamping head 111 can be raised to a position above the plane of the second

endless conveyor

107a,107b and moved towards each other to clamp the chip tray stack TS to prevent displacement of the clamped chip tray stack TS as the rotary platform 105 rotates. When it is necessary to image the side of the chip tray stack TS close to the chip tray stack clamping head 111, the chip tray stack clamping head 111 may be lowered to avoid interference with the imaging of the chip tray stack TS.

A third conveyor 113 is arranged downstream of the rotary platform 105, and the third conveyor 113 also comprises two third

endless belts

113a,113b arranged in parallel, which third

endless belts

113a,113b run synchronously by being driven by pulleys, which in turn are rotated by being driven by a motor. A damper 115 may be provided at the end of the third conveyance mechanism 113.

The imaging system 100 according to the second embodiment of the present invention further includes an imaging device 117 mounted on the frame 101 and having a lens facing the chip tray stack TS placed on the second conveying mechanism 107, the imaging device 117 being for imaging the chip tray stack TS placed on the second conveying mechanism 107. Also, depending on the size of the chip tray stack, an imaging device having an appropriate resolution may be selected. For example, for a chip tray stack having a smaller surface size, an imaging device having a resolution of five million pixels may be selected; for a chip tray stack with a larger surface size, an imaging device with a resolution of ten million pixels may be selected.

The operation of the imaging system 100 according to the second embodiment of the present invention will be described below.

The stack TS of chip trays from the packaging machine is transferred onto the second

endless conveyor belts

107a,107b via the first

endless conveyor belts

103a, 103b. While the chip tray stack TS is conveyed on the second

endless conveyors

107a,107b, the two chip tray stack orientation adjusting push rods 109 are folded and raised toward the chip tray stack TS so as to abut against the chip tray stack TS on the opposite sides thereof, so that the chip tray stack TS is adjusted to a proper orientation required for image formation. When the chip tray stack TS is moved to the imaging area, the second

endless belts

107a,107b stop running, the chip tray stack orientation adjustment push rod 109 also moves away from the chip tray stack TS, and the sensor triggers the imaging device 117 to image the first side of the chip tray stack TS.

Subsequently, each chip tray stack clamping head 111 is raised to a position above the plane on which the second

endless belts

107a,107b are positioned and moved toward the chip tray stack TS to clamp the chip tray stack TS. The rotary stage 105 is rotated by 90 degrees by the motor so that the second side of the chip tray stack TS is turned to an imaging position facing the imaging device 117. The chip tray stack gripper head 111 is lowered and the sensor triggers the imaging device 117 to image the second side of the chip tray stack TS.

Each chip tray stack clamping head 111 is raised again to clamp the chip tray stack TS, and the rotary platform 105 is rotated by 90 degrees again by the motor so that the third side of the chip tray stack TS is turned to the imaging position directly opposite to the imaging device 117. The sensor triggers the imaging device 117 to image the third side of the chip tray stack TS.

The rotary stage 105 is further rotated by 90 degrees by the motor so that the fourth side of the chip tray stack TS is turned to an imaging position facing the imaging device 117. The chip tray stack gripper head 111 is lowered and the sensor triggers the imaging device 117 to image the fourth side of the chip tray stack TS.

Finally, the rotary platform 105 continues to rotate 90 degrees again under the drive of the motor to return the second

endless belts

107a,107b to a position generally parallel or aligned with the first and third

endless belts

103a,103b, 113a, 113b. The second

endless belts

107a,107b continue to run to convey the second

endless belts

107a,107b to and from the third

endless belts

113a,113b, and the baffle 115 prevents the chip tray stack TS from falling off the third

endless belts

113a, 113b. At the same time, the next chip tray stack TS enters the imaging system 100 for imaging.

In the imaging system 100 according to the second embodiment of the present invention, since the chip tray stack TS enters the imaging area as the rotary stage 105 rotates 90 degrees at a time, the imaging device 117 can be disposed at any suitable spatial position with respect to the rotary stage 105. Further, it is also understood that the first conveyance mechanism 103 and the third conveyance mechanism 113 in the image forming system 100 according to the second embodiment of the present invention may be omitted.

In the imaging system 100 according to the second embodiment of the present invention, the rotating platform 105 is provided, and the second conveying mechanism 107 including the second

endless belts

107a,107b is mounted on the rotating platform 105 and rotates together with the rotating platform 105, but it is understood that the rotating platform 105 may not be provided and the second

endless belts

107a,107b may be fixedly mounted with respect to the frame. In this case, a rotary elevating mechanism as in the first embodiment may be provided between the second

endless belts

107a,107b. The rotary lifting mechanism can lift the chip tray stack TS and rotate 90 degrees at a time to sequentially image the four sides of the chip tray stack TS.

In actual use, the chip tray is injection molded, for example, from a black or gray plastic material. Black materials are not reflective, but grey materials may be reflective. Chip trays for loading different types of chips will typically have different design features, such as an outer length, width, and height dimension, an inner pocket dimension to accommodate the chip, and the like. However, all chip trays in the stack TS should be of the same specification and have the same design features.

Fig. 13 schematically shows a chip tray stack TS in which 30 chip trays containing chips are stacked together, the stacked chip trays are placed on a base T and covered at the top by a top cover C, and then bundled together by three transverse strapping bands B and one longitudinal strapping band B. When all of the chips in the tray are in the tray pockets, the spacing between adjacent trays in the chip tray stack TS is substantially uniform or constant. However, when the chips in the tray are removed from the tray pockets, the interval between the corresponding tray and the adjacent tray is different from the interval between the other trays. By analyzing the imaging of the chip tray stack TS, it is possible to find out whether or not there is an abnormal interval between adjacent trays in the chip tray stack TS, and the position of the abnormal interval. Although fig. 13 schematically shows 30 chip trays stacked together, it is to be understood that each chip tray stack TS may include less or more than 30 chip trays, e.g., a minimum of 2 chip trays.

In order to analyze the imaging of the chip tray stack TS, it is critical to obtain a clear high quality image. And a clear high-quality image is obtained which is affected by the background light and the illumination light. Displaying a black background and white light further contributes to obtaining a clear high-quality image by using background light and illumination light such as a black background and an adjustable combined light, a white background and an adjustable combined light, a black background and a white light, and the like, and finally by comparing and analyzing the obtained images.

An area array light source or a coaxial light source may be selected, and the use of an area array light source is considered in view of the higher cost of the coaxial light source. The chip tray stack TS is preferably illuminated using an area array light source with a luminous flux of about 400 lumens. If the luminous flux of the area array light source is below 342 lumens or above 457 lumens, the imaging of the design features of the chip tray stack TS may be blurred. The area array light source can be powered by 12V direct current, the power is 120 watts, and the working temperature can be set to be 0-60 ℃.

Fig. 14 schematically shows the positional relationship of the chip tray stack, the light source, and the imaging device. The minimum area array light source size can be calculated from the positional relationship shown in fig. 14. In the figure, BA denotes an edge of the chip tray stack TS, whose size is FOV, MN denotes an edge of the area-array light source corresponding to the edge of the chip tray stack TS, whose size is L, C denotes the imaging device, WD denotes a distance from the imaging device C to the chip tray stack TS, and WD denotes a distance from the area-array light source to the chip tray stack TS. In the present invention, the imaging device is preferably a camera, and may be a video camera. The size of L can be determined from the formula L = FOV (WD + WD)/WD.

For example, a chip tray stack TS, which is typically formed by stacking 30 chip trays, has a length, width, and height dimension of 34cm × 14cm × 17cm. If an imaging device of five million pixels is used for photographing a side surface of a chip tray stack TS having a size of 14cm x 17cm, WD is 60cm, fov has a maximum side length of 17cm, WD is 53cm, and L has a size of 30cm. If an imaging device of ten million pixels is used to photograph a side of the chip tray stack TS size of 34cm 17cm, WD is 45cm, FOV is 34cm on the largest side and WD is 21cm, then L is 50cm in size. Thus, the smallest area array light source size may be 30cm 50cm.

In the stacked state, one chip tray is stacked on another chip tray, and if the imaging device lens is directed toward the chip tray stack TS, the captured image is less likely to reflect the removal of the chip from the chip tray. In order to reflect the state where the chip is removed from the chip tray more clearly, the imaging device is mounted such that the imaging device lens is in the range of 60 to 80 degrees from the photographed side.

Fig. 15 schematically shows a flow of processing an image by the image processing system of the chip tray stack detecting apparatus according to the present invention. As shown in fig. 15, in step S1, the image processing system of the chip tray stack detection apparatus receives an image captured by the imaging device from the imaging system of the chip tray stack detection apparatus. In step S2, the image processing system of the chip tray stack detection apparatus preprocesses the image. In step S3, the image processing system of the chip tray stack detection apparatus processes and analyzes the image that has been preprocessed. In step S4, the image processing system of the chip tray stack detection apparatus issues a result of processing and analyzing the image.

Fig. 16 schematically shows an image of one side of the chip tray stack taken by the imaging device. The image captured by the imaging device is transferred to an image processing system where it is pre-processed. The preprocessing of the image firstly needs to remove noise from the image, which is mainly to perform binarization processing on the image by selecting a proper threshold value, that is, to set the gray value of all points on the image to be 0 or 255, so that the whole image presents an obvious black-and-white effect, thereby reflecting the overall and local characteristics of the image of the photographed chip tray stack TS to highlight the target contour. And then, analyzing the high-gray-value points in the image subjected to binarization processing through connected domain analysis, and eliminating the single high-gray-value pixel points. Next, a region of interest (ROI) of the image is extracted using an algorithm, a target region is found, and a particular feature of interest may also be highlighted. The region of interest of the extracted image is moved (including translation and rotation) into a designed tool (also called an algorithmic tool). In this way, the tool only computes and analyzes pixels in the region of interest to free up or reduce the computer resources required for image processing. FIG. 17 schematically shows an interface for pre-processing an image, where each small box marks the noise to be removed from the image.

Next, the image that has been preprocessed is subjected to image processing and analysis. First, the derivative of the gray values of the respective pixels in the X and Y directions is obtained and superimposed, thereby obtaining a derivative profile of the image, as shown in fig. 18. The derived profile is then analyzed.

One method of analyzing the derived profile is to compare the slope of each of the obtained lines with a preset slope of the corresponding type of chip tray stack by line fitting the image derived profile and removing the Y-axis information. The preset slope is a slope that is previously extracted and stored in the image processing system according to the same straight line fitting method for the corresponding type of chip tray stack without the chip being removed from the tray pocket. And determining whether the chip is moved out of the tray pit according to whether the slope of each straight line in the obtained straight lines is consistent with a preset slope. FIG. 19 schematically shows a line fit plot where no chip has been removed from a tray pocket, where the lines are generally parallel to each other and their slope is constant. Fig. 20 schematically shows a line fit plot with a chip removed from a tray pocket, where the slope of the line changes at the corresponding location where there is a chip removed from the tray pocket and thus is not parallel to the other lines.

Another method of analyzing the derived profile is to divide the derived profile into a plurality of columns and each column is subdivided into a plurality of sub-regions, for example into 30 sub-regions (rectangular regions) for a stack comprising 30 trays. And determining the coordinates of the center of each sub-region according to the coordinate positions of the four corners of each sub-region. The difference between the Y values of the center coordinates of the adjacent sub-regions is found as shown in fig. 20. Then, each difference is compared with a preset difference. The preset difference value is a difference value previously extracted in the same sub-region dividing method and stored in the image processing system for the corresponding type of chip tray stack without the chip being removed from the tray pit. When the corresponding difference value is beyond the predetermined range compared with the preset difference value, it can be determined that the chip is moved out of the tray pit at the position of the corresponding difference value.

Finally, the image processing system outputs the image processing and analyzing results, including sending mails and short messages to operators, sending flashing lights and audio alarms, and the like, and can also display the processing and analyzing results or fault tracking marks and the like on a display screen in a GUI (graphical user interface) mode.

In the method of analyzing the derived profile, the magnitude of the preset slope or the preset difference may be adjusted to adjust the accuracy of the image processing and analysis. Additionally, the results of the image processing and analysis may be stored in the image processing system for subsequent tracking and recall.

The above-described method for processing and analyzing images of a stack of chip trays is mainly done by means of a computer program. Thus, an image processing system according to the invention comprises a memory for storing instructions and a processor coupled to the memory, which processor performs the steps of processing an image as described above when the processor executes the instructions.

While the invention has been described in detail in connection with the preferred embodiments thereof, it should be understood that the detailed description is only illustrative of the invention and should not be taken as limiting the invention. The scope of the invention is determined by the claims.

Claims

1. An imaging system of a chip tray stack inspection apparatus, a chip tray stack including a plurality of chip trays stacked together, each chip tray containing a plurality of chips in respective tray pockets, the imaging system comprising:

a first conveying mechanism including two first endless belts arranged in parallel on which the chip trays are stacked to be conveyed;

a first lifting mechanism disposed between the first endless conveyor belts;

a second conveying mechanism mounted on the first elevating mechanism, the second conveying mechanism including two second endless belts arranged in parallel, the second endless belts being arranged in a direction substantially perpendicular to the first endless belts, the chip trays being conveyed while being stacked on the second endless belts;

a third conveying mechanism provided outside one of the first endless belts, the third conveying mechanism including two third endless belts arranged in parallel, the third endless belts being arranged in a direction substantially perpendicular to the first endless belts, the chip trays being conveyed stacked on the third endless belts;

a rotary lifting mechanism disposed at an end of the third conveyance mechanism remote from the second conveyance mechanism, the rotary lifting mechanism including a second lifting mechanism rotatably supported, and a carrier mounted atop the second lifting mechanism and for receiving and supporting the stack of chip trays;

a first imaging device provided in an extending direction of the third endless belt and having a lens toward the second conveying mechanism and/or the third conveying mechanism;

a second imaging device provided outside an extending direction of the third endless belt and having a lens facing the third conveying mechanism, wherein an orientation of a side of the chip tray stack can be changed by rotation of the rotary elevating mechanism so that each of four sides of the chip tray stack is imaged by the first imaging device or the second imaging device;

wherein, be provided with the baffle next to the downstream side of second conveying mechanism on first elevating system, the baffle is a little higher than in vertical direction the planar height in second annular conveyer belt place, first elevating system goes up and down and makes second annular conveyer belt with the baffle removes three positions: in a first position, the second endless conveyor is below the plane of the first endless conveyor, but the baffle extends above the plane of the first endless conveyor; in the second position, the second endless conveyor and the baffle are both elevated above the plane of the first endless conveyor; in a third position, both the second endless conveyor and the baffle are lowered below the plane of the first endless conveyor.

2. The imaging system of the chip tray stack inspection device of claim 1, wherein the imaging device is a camera.

3. The imaging system of the chip tray stack inspection device according to claim 1, wherein the imaging system further comprises:

4. The imaging system of the chip tray stack inspection device according to claim 1, wherein the imaging system further comprises:

5. The imaging system of the chip tray stack inspection device of claim 1, further comprising:

6. The imaging system of the chip tray stack detecting apparatus according to claim 1, wherein a guide strip is provided on an outer side of each of the first endless belt and the third endless belt, the guide strip being slightly higher than a plane of the corresponding endless belt.

7. The imaging system of the chip tray stack detecting apparatus according to claim 1, wherein the rotary elevating mechanism rotates 90 degrees or 180 degrees at a time.

8. The imaging system of the chip tray stack detecting apparatus according to claim 1, wherein the bracket of the rotary elevating mechanism has a cross shape, and when the bracket is lowered to a position below the plane of the third endless belt, one branch of the cross-shaped bracket is located in a space between the third endless belts.

9. The imaging system of the chip tray stack detecting apparatus according to claim 1, wherein the second elevating mechanism of the rotary elevating mechanism is an electric cylinder, a base of the second elevating mechanism is rotatably mounted on the base plate by two thrust bearings concentrically disposed, lower rings of the two bearings are respectively connected to a power source, upper rings of the two bearings are respectively connected to a motor of the electric cylinder, so that the two bearings also function as a conductive body.

10. The imaging system of the chip tray stack inspection device of claim 1, wherein a black background is used in imaging four sides of the chip tray stack, and further comprising an area array light source emitting white light.

11. The imaging system of the chip tray stack inspection device of claim 10, wherein the area array light source has a luminous flux of about 400 lumens.

12. The imaging system of the chip tray stack detecting apparatus according to claim 1, wherein the imaging device is mounted such that an imaging device lens is in a range of 60 to 80 degrees from a side to be photographed.

13. A chip tray stack detection apparatus, comprising:

the imaging system of any one of claims 1-12; and