TW202320919A - Object tracking and guiding system on conveyor belt and method thereof - Google Patents

Abstract

An object tracking and guiding system on a conveyor belt includes a data acquisition unit, a data calculation unit and a display unit. The data acquisition unit includes a color/depth camera and a barcode reading device. The depth camera is used to capture a color image information and a depth information of a target object. The barcode reading device is used to read an object category of the target object. The data calculation unit is used for receiving the color image information and the depth information to track a position of the target object on the conveyor belt, and generating a mark information according to the object category of the target object. The display unit includes at least one projector, and the projector projects the marking information on the target object according to the position of the target object.

Description

Object Tracking and Guidance System and Method on Conveyor Belt

The present invention relates to logistics management, and in particular to an object tracking and guiding system and method on a conveyor belt.

Before delivering items to consumers in traditional logistics management operations, it is necessary to identify and sort the items, and gather the sorted items to the corresponding shipping area for shipment. However, in the existing methods, the sorting can only be performed when the conveyor belt is running at a constant speed, and then the position of the objects is calculated using the speed and time of the conveyor belt, thereby reducing the efficiency of sorting. In addition, in the existing methods, a color camera is used to detect objects on the conveyor belt. However, when the object moves at a high degree, it will cause misjudgment due to blurred images, or the object cannot be detected normally because the color of the object is similar to the color of the conveyor belt. detection problem.

The invention relates to an object tracking and guiding system and method on a conveyor belt, which are used to improve sorting efficiency and reduce misjudgment.

According to one aspect of the present invention, an object tracking and guiding system on a conveyor belt is provided, including a data acquisition unit, a data calculation unit, and a display unit. The data acquisition unit includes a depth camera and a barcode reading device. The depth camera is used to capture an object image and a depth information of a target object. The barcode reading device is used to read an object category of the target object. The data operation unit is used to receive the object image and the depth information to track a position of the target object on the conveyor belt, and generate a mark information according to the object type of the target object. The display unit includes at least one projector, and the projector projects the marking information on the target object according to the position of the target object.

The efficacy of the present invention is to improve the efficiency of sorting and reduce misjudgment.

In order to have a better understanding of the above-mentioned and other aspects of the present invention, the following specific examples are given in detail with the accompanying drawings as follows:

The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of them. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art on the premise of being obvious belong to the protection scope of the present application. The same/similar symbols are used to represent the same/similar components in the following description.

Please refer to Figures 1A, 1B, 2 and 3, wherein Figures 1A and 1B show a schematic side view and a schematic top view of an object tracking and guidance system according to an embodiment of the present invention, and Figure 2 shows a schematic view according to the present invention A component block diagram of an object tracking and guidance system 100 according to an embodiment, and FIG. 3 shows a schematic diagram of a data processing flow according to an embodiment of the present invention. The object tracking and guiding system 100 can be used on a conveyor belt 103 , and includes a data acquisition unit 110 , a data calculation unit 120 and a display unit 130 . The data acquisition unit 110 may include a color/depth camera 112 and a barcode reading device 114 . The data computing unit 120 is, for example, a combination of a microprocessor, a digital signal processor and/or a graphics processor, and the display unit 130 may include a projector 132 .

In Figures 1A and 1B, before the logistics management operation delivers the items to consumers, the items need to be identified and sorted, and the sorted items are collected in the corresponding shipping area for shipment. The object tracking and guidance system 100 of this embodiment identifies and tracks the target object 101 through the color/depth camera 112 of the data acquisition unit 110, and then projects the corresponding marker information 127 (such as marker A) through the projector 132 of the display unit 130. , B) on the target object 101 for sorting personnel or sorting equipment to classify objects.

The color/depth camera 112 is arranged above the conveyer belt 103, and its number can be one or more. Each color/depth camera 112 can capture the target object 101 within a predetermined viewing angle range, and each color/depth camera 112 and its front color/depth camera 112 or rear color/depth camera 112 can be arranged at intervals on the conveyor belt 103 above, to identify and track the moving target object 101 .

In addition, the projector 132 is disposed above the conveyor belt 103 , and its quantity can be one or more. Each projector 132 can project one or more marking information 127 on at least one target object 101 according to the location of the target object 101 for sorting personnel or sorting equipment to classify objects.

Please refer to Figures 2 and 3, the color/depth camera 112 of the data acquisition unit 110 can identify a target object 101 on the conveyor belt 103, and obtain the color image information 113 and depth information 115 of the target object 101 based on the image of the target object 101 . The color image information 113 may include information such as a red-green-blue (RGB) image or a red-green-blue-white (RGBW) image of the target object 101, and the depth information 115 is, for example, a depth map, which calculates the relationship between the target object 101 and the color/color image according to time-of-difference ranging. The distance between the depth cameras 112 and the relative distance between the target object 101 and the color/depth cameras 112 are converted into corresponding depth values to obtain a depth map. In this embodiment, the data acquisition unit 110 obtains the actual boundary of the target object 101 by referring to the color image information 113 of the target object 101, and then correctly corrects the depth information 115 of the target object 101 to obtain a more accurate depth picture.

In addition, in this embodiment, the data acquisition unit 110 can also assist the image recognition of the target object 101 through the depth information 115, which can solve the misjudgment caused by the blurred image when the target object 101 moves at a high degree. On the other hand, it can also To avoid the problem that the target object 101 cannot be normally detected when the color of the target object 101 is similar to that of the conveyor belt 103 .

Please refer to FIGS. 2 and 3 , the barcode reading device 114 of the data acquisition unit 110 can also generate a category information 117 according to the category of the target object 101 . The target object 101 has, for example, a two-dimensional barcode or a matrix code (such as QR code), which records the product name, model, specification, item and other information of the target object 101. When the target object 101 passes through the barcode reading device 114 When down, the barcode reading device 114 can emit infrared light to scan the two-dimensional barcode or matrix code, so the barcode reading device 114 can obtain the category information 117 of the target object 101 according to the two-dimensional barcode or matrix code. The barcode reading device 114 is, for example, an optical scanner or a laser scanner.

Please refer to FIG. 2 and FIG. 3 , the data operation unit 120 is used to receive the color information and the depth information 115 of the target object 101 to track the target object 101 , and obtain a position information 125 of the target object 101 on the conveyor belt 103 . That is to say, when the target object 101 is moving on the conveyor belt 103, even if the conveyor belt 103 is moving at a non-constant speed or intermittently (such as acceleration motion or moving after a standstill), the data computing unit 120 performs real-time tracking and positioning through the image. In order to detect the source of the location information 125 of the target object 101 , it is not necessary to use the moving speed and moving time of the conveyor belt 103 to calculate the location of the target object 101 , so as to achieve the effect of real-time tracking and fast positioning of the target object 101 .

In addition, the data operation unit 120 can also generate a mark information 127 according to the type information 117 of the target object 101 . That is to say, when the type information 117 of the target object 101 is obtained (for example, by scanning a two-dimensional barcode or a matrix code), the data operation unit 120 generates a mark information 127 corresponding to the type information 117 of the target object 101 . The marking information 127 is, for example, symbols, characters, numbers or geometric patterns, etc., which are not limited in the present invention.

Referring to FIGS. 2 and 3 , after the data operation unit 120 obtains the position information 125 and the mark information 127 of the target object 101 , it can transmit the position information 125 and mark information 127 of the target object 101 to the projector 132 of the display unit 130 . The projector 132 of the display unit 130 can project mark information 127 (such as mark A, B) on the target object 101 according to the position information 125 of the target object 101 for sorting personnel or sorting equipment to classify objects.

Please refer to FIG. 1B , for example, there are three projectors 132 , and each projector 132 can respectively project a mark information (for example, mark A, B) on the target object 101 moving to a predetermined position. For example, three target objects 101a, 101b, 101c move from right to left along the conveyor belt 103 and pass through three cameras. When the first target object 101a moves to a position below or adjacent to the first projector 132a, The first projector 132a projects A mark information on the first target object 101a according to the category of the first target object 101a; when the first target object 101a moves to the position below or adjacent to the second projector 132b, the second projector 132b The category of the first target object 101a projects the A mark information on the first target object 101a; when the first target object 101a moves to the position below or adjacent to the third projector 132c, the third projector 132c according to the first target object 101a Class projection A mark information on the first target object 101a; similarly, when the second target object 101b moves below or adjacent to the first projector 132a, the first projector 132a projects B according to the category of the second target object 101b Mark information on the second target object 101b; when the second target object 101b moves to the position below or adjacent to the second projector 132b, the second projector 132b projects B mark information on the second target according to the category of the second target object 101b On the object 101b; when the second target object 101b moves to the position below or adjacent to the third projector 132c, the third projector 132c projects B mark information on the second target object 101b according to the type of the second target object 101b. When the third target object 101c moves to the position below or adjacent to the first projector 132a, the first projector 132a projects the C mark information on the third target object 101c according to the category of the third target object 101c; when the third target object 101c When moving to the position below or adjacent to the second projector 132b, the second projector 132b projects the C mark information on the third target object 101c according to the category of the third target object 101c; when the third target object 101c moves to the third projector 132c, the third projector 132c projects the C mark information on the third target object 101c according to the category of the third target object 101c.

Afterwards, the first target object 101a projected with the A mark information can be sorted into a first category area (not shown in the figure); the second target object 101b projected with the B mark information can be sorted into the second category In the zone; the third target object 101c projected with the C mark information can be sorted into the third category zone to complete the sorting of all objects. Although this embodiment takes the color/depth camera 112 and the projector 132 above the conveyor belt 103 as an example, the color/depth camera 112 and the projector 132 can also be installed elsewhere, as long as they are arranged along the moving direction of the target object 101, and It only needs to be able to realize the recognition of the target object 101 , the position tracking and the projection of the marker information 127 , which is not limited in the present invention.

In the above-mentioned embodiment, although a single projector 132 is shown to project a single mark information 127, the projector 132 can also project multiple mark information 127 for multiple target objects 101, as long as the data operation unit 120 simultaneously converts multiple target objects The position information 125 and its mark information 127 of 101 are transmitted to the projector 132 , and the projector 132 can respectively project the corresponding mark information 127 onto the respective target objects 101 . In addition, in a similar situation, if multiple target objects 101 move to a position below a projector 132 at the same time, the projector 132 can also project the corresponding marking information 127 onto the respective target objects 101, which is not covered by the present invention. be restricted.

Please refer to FIG. 4 for a flow chart of an object tracking and guiding method according to an embodiment of the present invention. The object tracking and guidance method includes the following steps. In step S210 , a color image information 113 and a depth information 115 of the target object 101 are captured by a depth camera 112 . In step S220 , a category information 117 of the target object 101 is read by the barcode reading device 114 . In step S230 , the color image information 113 and the depth information 115 are received to track the target object 101 to obtain a position information 125 of the target object 101 on the conveyor belt 103 . In step S240 , generate tag information 127 according to the category information 117 of the target object 101 . In step S250 , project marker information 127 on the target object 101 according to the location information 125 of the target object 101 .

The above-mentioned steps S210 and S220 correspond to the data acquisition unit 110 in the second figure to obtain the color image information 113, depth information 115 and category information 117 of the target object 101, and the above-mentioned steps S230 and S240 correspond to the data acquisition unit 110 in the second figure. The data operation unit 120 is performed to obtain the position information 125 and the marker information 127 of the target object 101 , and the above step S250 is correspondingly performed by the display unit 130 in FIG. 2 to project the marker information 127 .

Please refer to Figures 2, 5A, and 5B to further illustrate the functions of the various component modules of the data computing unit 120, wherein Figures 5A and 5B respectively illustrate a block diagram of the component operations of the data computing unit 120 according to an embodiment of the present invention . In this embodiment, the data computing unit 120 includes an information processing module 122 , an object detection and tracking module 124 and an information matching module 126 . In Figure 5A, the information processing module 122 is used to perform grayscale conversion and Gamma correction on the color image information 113, and then through image processing such as binarization and background segmentation, the color image information 113 belongs to the target The parts of the object 101 and the background parts that do not belong to the target object 101 are classified. In addition, the information processing module 122 is also used to perform depth filtering and binarization processing on the depth information 115 to obtain the depth information 115 of the target object 101, and then carry out weighted average of the color image information 113 and the depth information 115 of the target object 101 , to obtain the binarization matrix 123.

In addition, the object detection and tracking module 124 is used to process the morphology (morphology) and contour detection of the target object 101 with the binarization matrix 123, and store the contour image and position coordinates of the target object 101, so as to For tracking and comparison of subsequent target objects 101 . Afterwards, the object detection and tracking module 124 combines the recognized object image with its corresponding category information 117, and performs object tracking calculations to obtain the location information 125 and object category of the target object 101 on the conveyor belt 103 in real time .

In one embodiment, the morphological processing, for example, performs dilation and erosion on the binarization matrix 123 through a closing operation, so as to eliminate noise. For the contour detection, for example, Border Following Algorithms are used to perform Topological Analysis on the binary matrix 123 to find the contour of the target object 101 . Tracking algorithms such as Channel and Spatial Reliability (CSRT) trackers for object tracking. The CSRT tracker will calculate the histogram features and color names features of the selected area, and compare them with the features of the previous frame to determine the current position of the object.

In addition, please refer to FIG. 5B , the information matching module 126 is used to receive the location information 125 and the category information 117 of the target object 101 , and use the category information 117 to query the corresponding tag information 127 in the database. As shown in FIG. 5, the database stores a table of each object category and the corresponding label information 127, for example: when the object category is category_1, the corresponding label is projected label_1; when the object category is category_ 2, the corresponding marker is projected marker_2, and so on. Therefore, the information matching module 126 can generate corresponding tag information 127 according to the category information 117 of the target object 101 through a look-up table method. Subsequently, the projector 132 can project the corresponding marking information 127 on the target object 101 according to the position information 125 of the target object 101 , as mentioned above, and will not be repeated here.

From the above description, it can be seen that the object tracking and guidance system and method of the above embodiments of the present invention can solve the misjudgment caused by the blurred image when the target object moves at a high degree. On the other hand, it can also avoid the color and transportation of the target object The problem that the target object cannot be detected normally when the color is similar. In addition, the object tracking and guidance system and method of the above-mentioned embodiments of the present invention only need to use real-time image tracking and identification as the source of location information for detecting target objects, and do not need to use the moving speed and moving time of the conveyor belt to calculate the target object Therefore, even if the conveyor belt moves at a non-constant speed or intermittently (such as acceleration motion or moving after a standstill), the position coordinates of the target object can be accurately calculated to achieve real-time tracking and fast positioning of the target object.

To sum up, although the present invention has been disclosed by the above embodiments, it is not intended to limit the present invention. Those skilled in the art of the present invention can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention should be defined by the scope of the appended patent application.

100: Object Tracking and Guidance System 101: target object 101a: first target object 101b: Second target object 101c: The third target object 110: Data acquisition unit 112:Color/depth camera 113:Color image information 114: Barcode reading device 115: In-depth information 117: Category information 120: data computing unit 122:Information processing module 123: Binarization matrix 124: Object detection and tracking module 125: location information 126: Information matching module 127: Mark information 130: display unit 132:Projector 132a: The first projector 132b: second projector 132c: The third projector A, B: mark

FIG. 1A and FIG. 1B show schematic diagrams of an object tracking and guidance system according to an embodiment of the present invention; FIG. 2 shows a component block diagram of an object tracking and guidance system according to an embodiment of the present invention; FIG. 3 shows a schematic diagram of a data processing flow according to an embodiment of the present invention; FIG. 4 shows a flowchart of an object tracking and guidance method according to an embodiment of the present invention; and FIGS. 5A and 5B are block diagrams of device operations of a data computing unit according to an embodiment of the present invention, respectively.

100: Object Tracking and Guidance System

110: Data acquisition unit

112:Color/depth camera

114: Barcode reading device

117: Category information

120: data computing unit

122:Information processing module

124: Object detection and tracking module

126: Information matching module

130: display unit

132:Projector

Claims (16)

An object tracking and guiding system for use on a conveyor belt, the system comprising: A data acquisition unit, including a color/depth camera and a barcode reading device, the color/depth camera is used to capture a color image information and a depth information of a target object, and the barcode reading device is used to read the A category of information about the target object; A data computing unit, used to receive the color image information and the depth information to track the target object, obtain a position information of the target object on the conveyor belt, and generate a mark information according to the type information of the target object ;as well as A display unit includes at least one projector, and the projector projects the mark information on the target object according to the position information of the target object. The system as claimed in claim 1, wherein the depth information is used to assist image recognition of the target object. The system according to claim 1, wherein the target object has a two-dimensional barcode or a matrix code, and the barcode reading device can emit infrared light to scan the two-dimensional barcode or the matrix code. The system as described in claim 1, wherein the data computing unit detects the position information of the target object through real-time tracking and positioning of images. The system as described in claim 1, wherein the data computing unit correspondingly generates different tag information according to the different type information of the target object. The system as described in Claim 1, wherein the data operation unit includes an information processing module, and the information processing module is used to perform grayscale conversion and gamma correction on the color image information, and then perform binarization and background segmentation image processing to obtain a binarization matrix. The system as described in claim 6, wherein the data operation unit includes an object detection and tracking module, and the object detection and tracking module is used to perform morphological processing and contour of the target object on the binarization matrix Detect and store the outline image of the target object and its position coordinates. The system as described in claim 7, wherein the data operation unit includes an information matching module, the information matching module is used to receive the location information and the category information of the target object, and use the category information in a database Query the corresponding tag information in . A method for object tracking and guidance used on a conveyor belt, the method comprising: capturing a color image information and a depth information of the target object with a depth camera; Read a type of information of the target object with a barcode reading device; receiving the color image information and the depth information to track the target object, so as to obtain a position information of the target object on the conveyor belt; generating tag information according to the class information of the target object; and Projecting the mark information on the target object according to the position information of the target object. The method according to claim 9, wherein the depth information is used to assist image recognition of the target object. The method according to claim 9, wherein the target object has a two-dimensional barcode or a matrix code, and the barcode reading device can emit infrared light to scan the two-dimensional barcode or the matrix code. The method as described in claim 9, wherein the method detects the position information of the target object through real-time tracking and positioning of images. The method as described in claim 9, wherein the method correspondingly generates different tag information according to the different type information of the target object. The method as described in Claim 9, wherein the method is used to perform grayscale conversion and gamma correction on the color image information, and then perform image processing of binarization and background segmentation to obtain a binarization matrix. The method as described in claim 14, wherein the method is used to perform morphological processing and contour detection of the target object on the binarization matrix, and store the contour image and position coordinates of the target object. The method as described in claim 15, wherein the method is used to receive the location information and the category information of the target object, and use the category information to query the corresponding tag information in a database.

Images