TWI817265B - Object tracking and guiding system on conveyor belt and method thereof - Google Patents

Abstract

An object tracking and guiding system on a conveyor belt includes a data acquisition unit, a data calculation unit and a display unit. The data acquisition unit includes a color/depth camera and a barcode reading device. The depth camera is used to capture a color image information and a depth information of a target object. The barcode reading device is used to read an object category of the target object. The data calculation unit is used for receiving the color image information and the depth information to track a position of the target object on the conveyor belt, and generating a mark information according to the object category of the target object. The display unit includes at least one projector, and the projector projects the marking information on the target object according to the position of the target object.

Description

Object tracking and guidance system and method on conveyor belt

The present invention relates to logistics management, and in particular to an object tracking and guidance system and method on a conveyor belt.

Before delivering items to consumers in traditional logistics management operations, the items need to be identified and sorted, and the sorted items must be concentrated in the corresponding freight area to wait for shipment. However, existing methods can only sort when the conveyor belt is running at a constant speed, and then use the speed and time of the conveyor belt to calculate the location of the objects, which in turn reduces the efficiency of sorting. In addition, the existing method uses a color camera to detect objects on the conveyor belt. However, when the object moves highly, it will cause misjudgment due to blurred images, or the object cannot be detected normally because the color of the object is similar to the color of the conveyor belt. Detection issues.

The invention relates to an object tracking and guidance system and method on a conveyor belt to improve sorting efficiency and reduce misjudgments.

According to one aspect of the present invention, an object tracking and guidance system on a conveyor belt is proposed, including a data acquisition unit, a data computing unit and a display display unit. The data acquisition unit includes a depth camera and a barcode reading device. The depth camera is used to capture an object image and depth information of a target object. The barcode reading device is used to read an object type of the target object. The data computing unit is used to receive the object image and the depth information to track a position of the target object on the conveyor belt, and generate a mark information according to the object type of the target object. The display unit includes at least one projector, which projects the marking information on the target object according to the position of the target object.

The function of the present invention is to improve the efficiency of sorting and reduce misjudgments.

In order to have a better understanding of the above and other aspects of the present invention, examples are given below and are described in detail with reference to the accompanying drawings:

100: Object tracking and guidance system

101:Target object

101a: First target object

101b: Second target object

101c: The third target object

110: Data acquisition unit

112:Color/Depth Camera

113: Color image information

114:Barcode reading device

115:In-depth information

117:Category information

120: Data operation unit

122:Information processing module

123: Binarized matrix

124:Object detection and tracking module

125:Location information

126:Information matching module

127: Mark information

130:Display unit

132:Projector

132a:First projector

132b: Second projector

132c:Third projector

A, B: mark

Figures 1A and 1B illustrate a schematic diagram of an object tracking and guidance system according to an embodiment of the present invention; Figure 2 illustrates a component block diagram of an object tracking and guidance system according to an embodiment of the present invention; Figure 3 Figure 4 illustrates a schematic diagram of a data processing process according to an embodiment of the present invention; Figure 4 illustrates a flow chart of an object tracking and guidance method according to an embodiment of the present invention; and Figures 5A and 5B respectively illustrate a method according to the present invention. A block diagram of component operation of the data operation unit according to an embodiment.

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in this application, all other embodiments that can be easily understood by a person with ordinary skill in the art shall fall within the scope of protection of this application. The following description uses the same/similar symbols to indicate the same/similar components.

Please refer to Figures 1A, 1B, 2 and 3. Figures 1A and 1B respectively illustrate a schematic side view and a top view of an object tracking and guidance system according to an embodiment of the present invention. Figure 2 illustrates an object tracking and guidance system according to an embodiment of the present invention. A component block diagram of the object tracking and guidance system 100 according to an embodiment. Figure 3 is a schematic diagram of a data processing flow according to an embodiment of the present invention. The object tracking and guidance system 100 can be used on a conveyor belt 103 and includes a data acquisition unit 110, a data calculation unit 120 and a display unit 130. The data acquisition unit 110 may include a color/depth camera 112 and a barcode reading device 114. The data computing unit 120 is, for example, a combination of a microprocessor, a digital signal processor and/or a graphics processor, and the display unit 130 may include a projector 132 .

In Figures 1A and 1B, before the logistics management operation delivers the items to consumers, the items need to be identified and sorted, and the sorted items are gathered into the corresponding freight area to wait for shipment. The object tracking and guidance system 100 of this embodiment identifies and tracks the target object 101 through the color/depth camera 112 of the data acquisition unit 110, and then projects the corresponding mark information 127 (such as mark A) through the projector 132 of the display unit 130. , B) on the target object 101 for sorting personnel or sorting equipment to classify objects.

The color/depth camera 112 is arranged above the conveyor belt 103, and the number thereof can be one or more. Each color/depth camera 112 can capture the target object 101 within a predetermined viewing angle range, and each color/depth camera 112 and its front color/depth camera 112 or rear color/depth camera 112 can be arranged on the conveyor belt 103 at intervals. Above, to identify and track the moving target object 101.

In addition, the projector 132 is disposed above the conveyor belt 103, and the number of the projector 132 may be one or more. Each projector 132 can project one or more marking information 127 on at least one target object 101 according to the location of the target object 101 for sorting personnel or sorting equipment to classify objects.

Please refer to Figures 2 and 3. The color/depth camera 112 of the data acquisition unit 110 can identify a target object 101 on the conveyor belt 103, and obtain the color image information 113 and depth information 115 of the target object 101 based on the image of the target object 101. . The color image information 113 may include information such as a red-green-blue (RGB) image or a red-green-blue-white (RGBW) image of the target object 101, and the depth information 115 is, for example, a depth map, which calculates the distance between the target object 101 and the color/ distance between the depth cameras 112, and convert the relative distance between the target object 101 and the color/depth camera 112 into corresponding depth values to obtain a depth map. In this embodiment, the data acquisition unit 110 obtains the actual boundary of the target object 101 by referring to the color image information 113 of the target object 101, and then correctly corrects the depth information 115 of the target object 101 to obtain a more accurate depth. Figure.

In addition, in this embodiment, the data acquisition unit 110 can also assist the image recognition of the target object 101 through the depth information 115, which can solve the problem of the target object 101. In the case of high movement, misjudgment may occur due to image blur. On the other hand, it can also avoid the problem that the target object 101 cannot be detected normally when the color of the target object 101 is similar to the color of the conveyor belt 103 .

Referring to Figures 2 and 3, the barcode reading device 114 of the data acquisition unit 110 can also generate a category information 117 according to the category of the target object 101. The target object 101 has, for example, a two-dimensional barcode or a matrix code (such as a QR code), which records the product name, model, specification, item and other information of the target object 101. When the target object 101 passes through the barcode reading device 114 When downward, the barcode reading device 114 can emit infrared light to scan the two-dimensional barcode or matrix pattern code, so the barcode reading device 114 can obtain the category information 117 of the target object 101 based on the two-dimensional barcode or matrix pattern code. The barcode reading device 114 is, for example, an optical scanner or a laser scanner.

Referring to Figures 2 and 3, the data computing unit 120 is used to receive the color information and depth information 115 of the target object 101 to track the target object 101, and obtain a position information 125 of the target object 101 on the conveyor belt 103. That is to say, when the target object 101 moves on the conveyor belt 103, even if the conveyor belt 103 does not move at a constant speed or moves intermittently (such as accelerating or moving after being stationary), the data computing unit 120 performs real-time tracking and positioning through images. In order to detect the source of the position information 125 of the target object 101, it is not necessary to use the moving speed and moving time of the conveyor belt 103 to calculate the position of the target object 101, so as to achieve the effect of real-time tracking and rapid positioning of the target object 101.

In addition, the data computing unit 120 can also generate a tag information 127 according to the category information 117 of the target object 101 . That is to say, when the class of target object 101 After the identification information 117 has been obtained (for example, by scanning a two-dimensional barcode or a matrix image code), the data operation unit 120 generates a tag information 127 corresponding to the category information 117 of the target object 101 . The mark information 127 may be, for example, symbols, characters, numbers, or geometric patterns, and the present invention is not limited thereto.

Referring to Figures 2 and 3, after the data computing unit 120 obtains the position information 125 and the mark information 127 of the target object 101, it can transmit the position information 125 and the mark information 127 of the target object 101 to the projector 132 of the display unit 130. The projector 132 of the display unit 130 can project mark information 127 (eg, marks A, B) on the target object 101 according to the position information 125 of the target object 101 for sorting personnel or sorting equipment to classify objects.

Referring to Figure 1B, there are, for example, three projectors 132, and each projector 132 can respectively project a piece of marker information (eg, markers A, B) on the target object 101 moved to a predetermined position. For example, three target objects 101a, 101b, and 101c move sequentially from right to left along the conveyor belt 103 and pass through three cameras. When the first target object 101a moves to a position below or adjacent to the first projector 132a, The first projector 132a projects the A mark information on the first target object 101a according to the type of the first target object 101a; when the first target object 101a moves to below or adjacent to the second projector 132b, the second projector 132b projects the A mark information according to the type of the first target object 101a. The type of the first target object 101a projects the A mark information on the first target object 101a; when the first target object 101a moves to a position below or near the third projector 132c, the third projector 132c according to the type of the first target object 101a The category projects A mark information on the first target object 101a; similarly, when the second target object 101b moves to below or adjacent to the first projector 132a When, the first projector 132a projects the B mark information on the second target object 101b according to the type of the second target object 101b; when the second target object 101b moves to below or adjacent to the second projector 132b, the second projector 132b projects the B mark information on the second target object 101b according to the type of the second target object 101b; when the second target object 101b moves to below or adjacent to the third projector 132c, the third projector 132c The category of 101b projects the B tag information on the second target object 101b. When the third target object 101c moves to a position below or adjacent to the first projector 132a, the first projector 132a projects C mark information on the third target object 101c according to the type of the third target object 101c; when the third target object 101c When moving to a position below or adjacent to the second projector 132b, the second projector 132b projects the C mark information on the third target object 101c according to the type of the third target object 101c; when the third target object 101c moves to the third projector When the third projector 132c is below or near the third target object 101c, the third projector 132c projects the C mark information on the third target object 101c according to the type of the third target object 101c.

Afterwards, the first target object 101a to which the A mark information is projected can be sorted into a first category area (not shown); the second target object 101b to which the B mark information is projected can be sorted to the second category. In the area; the third target object 101c on which the C mark information is projected can be sorted into the third category area to complete the sorting of all objects. Although in this embodiment, the color/depth camera 112 and the projector 132 are placed above the conveyor belt 103 as an example, the color/depth camera 112 and the projector 132 can also be placed elsewhere, as long as they are placed along the moving direction of the target object 101, and Able to complete target objects 101 identification, position tracking and marking information 127 can be projected, and the present invention does not limit this.

In the above embodiment, although a single projector 132 projects a single mark information 127, the projector 132 can also project multiple mark information 127 for multiple target objects 101, as long as the data computing unit 120 simultaneously projects multiple target objects 101. The position information 125 and mark information 127 of 101 are transmitted to the projector 132, and the projector 132 can project the corresponding mark information 127 onto the respective target objects 101 respectively. In addition, in a similar situation, if multiple target objects 101 move to a position below a projector 132 at the same time, the projector 132 can also project the corresponding mark information 127 onto the respective target objects 101. This is not the case in the present invention. be restricted.

Please refer to Figure 4 to illustrate a flow chart of an object tracking and guidance method according to an embodiment of the present invention. The object tracking and guidance method includes the following steps. In step S210, a depth camera 112 is used to capture a color image information 113 and a depth information 115 of the target object 101. In step S220, a barcode reading device 114 is used to read a category information 117 of the target object 101. In step S230 , the color image information 113 and the depth information 115 are received to track the target object 101 to obtain a position information 125 of the target object 101 on the conveyor belt 103 . In step S240, a mark information 127 is generated according to the category information 117 of the target object 101. In step S250, the mark information 127 is projected on the target object 101 according to the position information 125 of the target object 101.

The above-mentioned steps S210 and S220 are performed by the data acquisition unit 110 in Figure 2 to obtain the color image information 113 and depth of the target object 101. Information 115 and category information 117, the above-mentioned steps S230 and S240 correspond to being performed by the data computing unit 120 in Figure 2 to obtain the position information 125 and mark information 127 of the target object 101, and the above-mentioned step S250 corresponds to being performed by the data operation unit 120 in Figure 2 The display unit 130 is configured to project the mark information 127 .

Please refer to Figures 2 and 5A and 5B to further explain the functions of each component module of the data computing unit 120. Figures 5A and 5B respectively illustrate component operation block diagrams of the data computing unit 120 according to an embodiment of the present invention. . In this embodiment, the data computing unit 120 includes an information processing module 122, an object detection and tracking module 124, and an information matching module 126. In Figure 5A, the information processing module 122 is used to perform grayscale conversion and gamma correction on the color image information 113, and then through image processing such as binarization and background segmentation to convert the color image information 113 belonging to the target Parts of the object 101 and background parts that do not belong to the target object 101 are classified. In addition, the information processing module 122 is also used to perform depth filtering and binarization processing on the depth information 115 to obtain the depth information 115 of the target object 101, and then perform a weighted average of the color image information 113 and the depth information 115 of the target object 101. , to obtain the binarized matrix 123.

In addition, the object detection and tracking module 124 is used to perform morphology processing and contour detection on the target object 101 with the binary matrix 123, and store the contour image and its position coordinates of the target object 101, so as to For subsequent tracking and comparison of the target object 101. Afterwards, the object detection and tracking module 124 combines the recognized object image with its corresponding category information 117 and performs object tracking calculations to obtain the position information of the target object 101 on the conveyor belt 103 in real time. Message 125 and its object category.

In one embodiment, the morphological processing, for example, performs dilation and erosion on the binarized matrix 123 through closing operations to achieve the purpose of eliminating noise. Contour detection uses, for example, a border following algorithm (Border Following Algorithms) to perform topological analysis (Topological Analysis) on the binarized matrix 123 to find the contour of the target object 101 . The tracking algorithm uses the Channel and Spatial Reliability (CSRT) tracker for object tracking. The CSRT tracker will calculate the histogram features and color names features of the selected area, and compare them with the features of the previous image to determine the current location of the current object.

In addition, please refer to Figure 5B. The information matching module 126 is used to receive the location information 125 and its category information 117 of the target object 101, and use the category information 117 to query the corresponding tag information 127 in the database. As shown in Figure 5, the database stores a table of each object category and corresponding tag information 127. For example: when the object category is category_1, the corresponding tag is projection tag_1; when the object category is category_ When 2, the corresponding mark is projection mark_2, and so on. Therefore, the information matching module 126 can generate corresponding tag information 127 according to the category information 117 of the target object 101 through a table lookup method. Subsequently, the projector 132 can project the corresponding mark information 127 on the target object 101 according to the position information 125 of the target object 101, as described above, which will not be described again here.

It can be seen from the above description that the object tracking and guidance system and method thereof according to the above embodiments of the present invention can solve the problem of image distortion when the target object moves highly. On the other hand, it can also avoid the problem that the target object cannot be detected normally when the color of the target object is similar to the color of the conveyor belt. In addition, the object tracking and guidance system and method of the above embodiments of the present invention only need to use real-time tracking and recognition of images as the source of location information to detect the target object, and do not need to use the moving speed and moving time of the conveyor belt to calculate the target object. Therefore, even if the conveyor belt does not move at a constant speed or moves intermittently (such as accelerating or moving after being stationary), the position coordinates of the target object can still be accurately calculated to achieve real-time tracking and rapid positioning of the target object.

In summary, although the present invention has been disclosed above through embodiments, they are not intended to limit the present invention. Those with ordinary knowledge in the technical field to which the present invention belongs can make various modifications and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be determined by the appended patent application scope.

100: Object tracking and guidance system

110: Data acquisition unit

112:Color/Depth Camera

114:Barcode reading device

117:Category information

120: Data operation unit

122:Information processing module

124:Object detection and tracking module

126:Information matching module

130:Display unit

132:Projector

Claims (10)

An object tracking and guidance system for use on a conveyor belt. The system includes: a data acquisition unit, including a color/depth camera and a barcode reading device. The color/depth camera is used to capture a target object. A color image information and a depth information, the barcode reading device is used to read a type of information of the target object; a data computing unit is used to receive the color image information and the depth information to track the target object to obtain A position information of the target object on the conveyor belt, and a mark information is generated according to the category information of the target object, wherein the data processing unit includes an information processing module, and the information processing module is used to convert the color image The information is subjected to gray scale conversion and gamma correction, and then is subjected to image processing of binarization and background segmentation to obtain a binarization matrix, and the data operation unit includes an object detection and tracking module. The object detection and tracking module The tracking module is used to perform morphological processing and contour detection of the target object on the binary matrix, and store the contour image and position coordinates of the target object, and the data computing unit includes an information matching module , the information matching module is used to receive the location information and the category information of the target object, and use the category information to query the corresponding mark information in a database; and a display unit including at least one projector, the The projector projects the mark information on the target object according to the position information of the target object. The system of claim 1, wherein the depth information is used to assist image recognition of the target object. The system according to claim 1, wherein the target object has a two-dimensional barcode or a matrix pattern code, and the barcode reading device can emit infrared light to scan the two-dimensional barcode or the matrix pattern code. The system as described in claim 1, wherein the data computing unit detects the location information of the target object through real-time image tracking and positioning. The system of claim 1, wherein the data computing unit generates different tag information according to different types of information of the target object. An object tracking and guidance method, used on a conveyor belt, the method includes: using a depth camera to capture a color image information and a depth information of the target object; using a barcode reading device to read the target object A type of information; receiving the color image information and the depth information to track the target object to obtain a position information of the target object on the conveyor belt; generating a mark information based on the type of information of the target object, wherein the method It is used to perform grayscale conversion and gamma correction on the color image information, and then perform image processing of binarization and background segmentation to obtain a binarization matrix; wherein the method is used to perform the binarization matrix on the target Morphological processing and contour detection of objects, and storing the contour image and position coordinates of the target object; where this method is used Receive the location information and the category information of the target object, and use the category information to query the corresponding tag information in a database; and project the tag information on the target object based on the location information of the target object. The method of claim 6, wherein the depth information is used to assist image recognition of the target object. The method of claim 6, wherein the target object has a two-dimensional barcode or a matrix pattern code, and the barcode reading device can emit infrared light to scan the two-dimensional barcode or the matrix pattern code. The method described in claim 6, wherein the method detects the location information of the target object through real-time image tracking and positioning. The method described in claim 6, wherein the method generates different tag information according to different types of information of the target object.

Images