AU2021368390B2 - Multi-target recognition system and method for follow-up robot based on coded thermal infrared mark - Google Patents
Multi-target recognition system and method for follow-up robot based on coded thermal infrared mark Download PDFInfo
- Publication number
- AU2021368390B2 AU2021368390B2 AU2021368390A AU2021368390A AU2021368390B2 AU 2021368390 B2 AU2021368390 B2 AU 2021368390B2 AU 2021368390 A AU2021368390 A AU 2021368390A AU 2021368390 A AU2021368390 A AU 2021368390A AU 2021368390 B2 AU2021368390 B2 AU 2021368390B2
- Authority
- AU
- Australia
- Prior art keywords
- image
- thermal radiation
- heating wires
- radiation mark
- thermal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 230000005855 radiation Effects 0.000 claims abstract description 110
- 238000010438 heat treatment Methods 0.000 claims abstract description 76
- 238000003708 edge detection Methods 0.000 claims description 25
- 238000001914 filtration Methods 0.000 claims description 22
- 230000008569 process Effects 0.000 claims description 13
- 230000000877 morphologic effect Effects 0.000 claims description 9
- 238000004422 calculation algorithm Methods 0.000 claims description 5
- 230000000694 effects Effects 0.000 claims description 4
- 230000003287 optical effect Effects 0.000 claims description 4
- 230000010339 dilation Effects 0.000 claims 4
- 230000008030 elimination Effects 0.000 claims 2
- 238000003379 elimination reaction Methods 0.000 claims 2
- 230000003628 erosive effect Effects 0.000 claims 2
- 239000008186 active pharmaceutical agent Substances 0.000 claims 1
- 239000003550 marker Substances 0.000 abstract 4
- 238000010586 diagram Methods 0.000 description 8
- 238000004590 computer program Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 5
- 230000004048 modification Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 238000001931 thermography Methods 0.000 description 3
- 229920000049 Carbon (fiber) Polymers 0.000 description 2
- 239000004917 carbon fiber Substances 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- VNWKTOKETHGBQD-UHFFFAOYSA-N methane Chemical compound C VNWKTOKETHGBQD-UHFFFAOYSA-N 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000002310 reflectometry Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
A following robot multi-target identification system and method based on a coded thermal infrared marker. The system comprises: a thermal radiation mark, which is arranged on a target to be identified, wherein a plurality of heating coils are arranged in the thermal radiation mark, and the heating coils are heated according to different coding rules; a thermal infrared image collection module, which is used for collecting a thermal infrared image of said target and transferring the thermal infrared image to a data processing module; and the data processing module, which is used for identifying a thermal radiation marker image from the thermal infrared image, identifying coding information of the thermal radiation marker according to the thermal radiation marker image, and identifying said target by means of the coding information. A thermal radiation mark worn by a target to be identified is identified, and coding information in the thermal radiation mark is extracted, such that the identification of different targets by a following robot is realized according to the coding information.
Description
The present disclosure relates to the technical field of target recognition of robots, in particular to a multi-target recognition system and method for a follow-up robot based on a coded thermal infrared mark.
The description in this section merely provides background information related to the present disclosure and does not necessarily constitute the prior art. Target person recognition is the key to realize follow-up walking tasks of robots. However, the existing target person recognition technology is still in the laboratory stage, and cannot meet the application requirements of follow-up walking robots in real circumstances. In addition, when the number of the follow-up robots operated in the same scene increases, the number of targets followed up by the robots will increase correspondingly. However, an existing target recognition method for the follow-up robots is usually applicable for a single target object, and a satisfactory solution is not provided to solve the autonomous recognition problem of the robots when there are multiple target persons at the same time. In order to improve the stability in target person recognition and achieve a multi-target recognition function, a target recognition system and method for a follow-up walking robot based on a thermal radiation mark and thermal infrared image recognition is provided. The target recognition method based on a thermal infrared image overcomes the defect that a traditional color image recognition method is easily affected by the ambient light variation, can meet the application requirements in indoor and outdoor environments and has good environmental adaptability. The inventor finds that the thermal imaging technology also has the defects that the edges of the infrared thermal images are blurred, and the features are difficult to extract; and the infrared thermal images are easily affected by the surface characteristics of objects and external factors such as the radiation wavelength and the like, and the brightness of the images can be affected by the navigator clothing thickness and the direction of movement as
I well as materials of clothing. Therefore, human detection based on infrared thermal imaging is still a very challenging problem.
In order to solve the problems, the present disclosure provides a multi-target recognition system and method for a follow-up robot based on a coded thermal infrared mark. A thermal radiation mark worn by a to-be-recognized target is recognized, coded information in the thermal radiation mark is extracted, and the follow-up robot recognizes different targets according to the coded information. To achieve the foregoing objective, the present disclosure uses the following technical solutions: In a first aspect, a multi-target recognition system for a follow-up robot based on a coded thermal infrared mark is provided, the system including: a thermal radiation mark arranged on a to-be-recognized target, where a plurality of heating wires are arranged in the thermal radiation mark, and the heating wires are heated according to different coding rules; a thermal infrared image collecting module used for collecting a thermal infrared image of the to-be-recognized target and transmitting to a data processing module; and the data processing module used for recognizing a thermal radiation mark image from the thermal infrared image, recognizing coded information of the thermal radiation mark according to the thermal radiation mark image and recognizing the to-be-recognized target by means of the coded information. In a second aspect, a multi-target recognition method for the follow-up robot based on the coded thermal infrared mark is provided, the method including: collecting the thermal infrared image of the to-be-recognized target; recognizing the thermal radiation mark image from the thermal infrared image; recognizing the coded information of the thermal radiation mark according to the thermal radiation mark image; and recognizing the to-be-recognized target by means of the coded information. In a third aspect, an electronic device is provided, the electronic device including a memory, a processor and a computer instruction stored in the memory and run on the processor, where when the computer instruction is run by the processor, the steps of the multi-target recognition method for the follow-up robot based on the coded thermal infrared mark are completed. In a fourth aspect, a computer-readable storage medium is provided, the computer-readable storage medium being used for storing the computer instruction, where when the computer instruction is executed by the processor, the steps of the multi-target recognition method for the follow-up robot based on the coded thermal infrared mark are completed. Compared with the prior art, the present disclosure has the beneficial effects: 1. The present disclosure has the multi-target recognition capability by coding the heating wires in the thermal radiation marks, and recognizes the coded information of the thermal radiation marks heated according to the coding rules, so that the follow-up robot recognizes different targets according to the coded information. The advantages of the additional aspects of the present disclosure will be set forth in part in the description below, parts of which will become apparent from the description below, or will be understood by the practice of the present disclosure.
The accompanying drawings constituting a part of this application are used for providing further understanding for this application. Exemplary embodiments of this application and descriptions thereof are used for explaining this application and do not constitute any inappropriate limitation to this application. FIG. 1 is a structural diagram of Embodiment 1 of the present disclosure. FIG. 2 is a recognition flow diagram of Embodiment 1 of the present disclosure. FIG. 3 is the coding rule of the thermal radiation mark in Embodiment 1 of the present disclosure. FIG. 4 is the thermal infrared images collected by Embodiment 1 of the present disclosure. FIG. 5 is an image after median filtering is performed on the thermal infrared images by Embodiment 1 of the present disclosure. FIG. 6 is an edge detection image extracted by Embodiment 1 of the present disclosure. FIG. 7 is vertical edge pixel searching rules of Embodiment 1 of the present disclosure. FIG. 8 is an image after the vertical edge of the edge detection image is eliminated by Embodiment 1 of the present disclosure. FIG. 9 is an image after the image in FIG. 8 is expanded for three times by Embodiment
1 of the present disclosure. FIG. 10 is an image after the image in FIG. 9 is corroded for four times by Embodiment 1 of the present disclosure. FIG. 11 is an image, which is determined by Embodiment 1 of the present disclosure, of an area where the thermal radiation mark is located. FIG. 12 is a thermal radiation mark image extracted by Embodiment 1 of the present disclosure. FIG. 13 is a bright and dark stripe image recognized by Embodiment 1 of the present disclosure. FIG. 14 is a code recognition image verified by Embodiment 1 of the present disclosure. wherein: 1 represents the follow-up robot, 2 represents a thermal infrared camera, and 3 represents the thermal radiation mark.
The present disclosure is further described below with reference to the accompanying drawings and embodiments. It should be noted that the following detailed descriptions are all exemplary and are intended to provide a further understanding of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by a person of ordinary skill in the art to which this application belongs. It should be noted that terms used herein are only for describing specific implementations and are not intended to limit exemplary implementations according to this application. As used herein, the singular form is intended to include the plural form, unless the context clearly indicates otherwise. In addition, it should further be understood that terms "comprise" and/or "include" used in this specification indicate that there are features, steps, operations, devices, components, and/or combinations thereof. In the present disclosure, orientation or position relationships indicated by the terms such as "upper", "lower", "left", "right" "front", "rear", "vertical", "horizontal", "side", and "bottom" are based on orientation or position relationships shown in the accompanying drawings, and are merely relationship words that are determined for ease of describing the structural relationship between components or elements in the present disclosure, and are not intended to specifically refer to any component or element in the present disclosure. Therefore, such terms should not be construed as a limitation on the present disclosure.
In the present disclosure, terms such as "fixedly connected", "interconnection", and "connection" should be understood in a broad sense. The connection may be a fixing connection, an integral connection or a detachable connection; or the connection may be a direct connection, or an indirect connection by using an intermediary. Relevant scientific research or technical personnel in the art may determine the specific meanings of the foregoing terms in the present disclosure according to specific situations, and such terms should not be construed as a limitation on the present disclosure. Embodiment 1 In this embodiment, in order to achieve the objective that a follow-up robot recognizes multiple targets, a multi-target recognition system for a follow-up robot based on a coded thermal infrared mark is disclosed, the system including: a thermal radiation mark arranged on a to-be-recognized target, where a plurality of heating wires are arranged in the thermal radiation mark, and the heating wires are heated according to different coding rules; a thermal infrared image collecting module used for collecting a thermal infrared image of the to-be-recognized target and transmitting to a data processing module; and the data processing module used for recognizing a thermal radiation mark image from the thermal infrared image, recognizing coded information of the thermal radiation mark according to the thermal radiation mark image and recognizing the to-be-recognized target by means of the coded information. Furthermore, the heating wires in the thermal radiation mark are coded by using a binary rule. Furthermore, the coded information carried by the thermal radiation mark is in one-to-one correspondence to the to-be-recognized target. Furthermore, the heating wires in the thermal radiation mark are arranged in parallel at equal intervals. Furthermore, the specific process that the data processing module recognizes the thermal radiation mark image from the thermal infrared image includes: performing filtering processing on the thermal infrared image; extracting the edges of the heating wires from the thermal infrared image after filtering processing so as to obtain an edge detection image; and eliminating the vertical edge and divergent edge of the edge detection image, and extracting the thermal radiation mark image from the edge detection image according to the largest area where the thermal radiation mark is located.
Furthermore, the specific process that the data processing module recognizes the coded information of the thermal radiation mark according to the thermal radiation mark image includes: recognizing the image of the heated heating wires and the image of the non-heated heating wires in the thermal radiation mark image according to pixel gray values, and determining the coded information of the thermal radiation mark according to height information of the image of the heated heating wires and the image of the non-heated heating wires in a vertical direction. Furthermore, the data processing module judges whether the image of the heated heating wires and the image of the non-heated heating wires in the thermal radiation mark image satisfy the morphological constraint, and determines the coded information of the thermal radiation mark according to the height information of the image of the heated heating wires and the image of the non-heated heating wires in the vertical direction if the image of the heated heating wires and the image of the non-heated heating wires in the thermal radiation mark image satisfy the morphological constraint. The multi-target recognition system for the follow-up robot based on the coded thermal infrared mark is described detailedly with reference to FIG. 1 to FIG. 14. As shown in FIG. 2, the multi-target recognition system for the follow-up robot based on the coded thermal infrared mark includes a thermal radiation mark 3 worn by a to-be-recognized target, where a plurality of parallel carbon fiber heating wires are arranged in the thermal radiation mark, the carbon fiber heating wires are heated according to different coding rules, so that gray images with obvious feature differences can be obtained by the thermal infrared camera, and thus, the independent coded information carried by different thermal radiation marks is extracted in an image processing manner so as to recognize different targets. The heating wires in the thermal radiation mark are coded by using the binary rule. As shown in FIG. 3, the coding rule is described by taking the thermal radiation mark containing five heating wires as an example, the uppermost heating wire and the lowest heating wire as boundary determination marks of the thermal radiation mark are always heated, the internal three heating wires are heated by using the binary coding rule, eight combinations from 0 to 7 are provided, i.e., all the heating wires are not heated if the number is 0, only the lowest heating wire is heated if the number is 1, only the middle heating wire is heated if the number is 2, ... , and all the three heating wires are heated if the number is 7. The multi-feature differentiation of the same thermal radiation mark can be achieved by using the method.
Therefore, the recognition requirements of the robot in the presence of multiple target persons can be met. FIG. 4 shows the thermal infrared images, which are obtained by the thermal infrared camera and have the corresponding numbers of 0, 2, 5 and 7, of the to-be-recognized target when the thermal radiation mark formed by five heating wires uses the coding rule. It can be seen from FIG. 3 that the heated heating wires are clearly distinguished from the surrounding environment, and the feasibility of the recognition solution is proved. Although human eyes can easily recognize the positions of barcode areas in the images, the features of the images need to be analyzed step by step by using a digital image processing technique if the robot needs to achieve the recognition function. Thus, the thermal infrared images are recognized by the data processing module of the follow-up robot 1. The specific principle of the data processing module for recognizing the thermal infrared images is described below only by taking the recognition process of the thermal infrared image of the to-be-recognized target when the number is 7 as an example. The first step: performing image filtering. Generally, the actually obtained image is inevitably interfered by external noise and internal noise in the processes of forming, transmission, receiving and treatment, thus, the quality of the image is affected, the image is blurred, the features are submerged, and the analysis is difficult. It is very important to remove noise from the image polluted by noise and recover the image during image preprocessing. Here, median filtering is mainly used to remove impulse noise brought by an optical acquisition system from the barcode image. When strong impulse noise interference exists in the image, the gray values corresponding to these interference points greatly differ from that of adjacent pixels. Therefore, by using the method of sorting to remove median, the gray values of these interference points become similar to that of some adjacent pixels so as to achieve the effect of removing noise. Median filtering is a nonlinear smoothing method for reducing edge blur. Its basic idea is
as follows: a one-dimensional sequence A'f2,13---f, is set, the window length is set to m, median filtering is performed on the one-dimensional sequence, i.e., m numbers
i-"- -1 9A9f+1--9 fie are sequentially extracted from the sequence as center point values
of a window, then the m numbers are sorted according to the value size, the number in the middle is taken as the filtering output, i.e.:
y, = median{ f_- . f , whereE Z,v=(m-1)/2 The effect of median filtering depends on two factors: the spatial range of a neighborhood and the number of pixels involved in median calculation. Median filtering can suppress random noise without blurring edges. Median filtering is performed on the original image with noise by using a 3*3 template, and the obtained processed result is shown in FIG. 5. The second step: performing edge detection. For the heated thermal radiation mark image, the gradient on edge points of the heating wire is consistent, the color reflectivity on both sides of the edge of the heating wire has great difference, and the edge is relatively dense. The described features can be processed by extracting pixels with obvious features and large amount of information in the thermal infrared image after filtering processing in the edge detection manner, thereby reducing the amount of calculation and improving the algorithm efficiency. Common edge detection algorithms include classical edge detection operators such as a Prewitts operator, a Roberts operator, a Soble operator, a Canny operator, etc. The heating wire is basically horizontal when moving with a human body. Therefore, the Prewitts operator is more favorable for edge extraction of the thermal radiation mark. The Prewitt operator is a differential operator for image edge detection. Its principle is to realize edge detection by using the difference generated by the gray values of pixels in a specific area. The Prewitt operator uses the 3*3 template to calculate the pixel values in the area. Therefore, the edge detection results are more obvious in horizontal and vertical directions. The Prewitt operator is suitable for recognizing the image with more noise and gray gradient. FIG. 6 is the edge detection image after edge detection is performed by using the Prewitt operator. The third step: eliminating the vertical edge. The original edge information of the above edge detection image includes more edge data unrelated to the thermal radiation mark, such as body edge information of the target person and edge information of other objects. These edges may introduce unnecessary interference information to subsequent processing. Therefore, a certain filtering operation needs to be used to eliminate excessive edge data. It can be seen from FIG. 6 that the edge of the heating wire is approximately horizontal, therefore, the obvious vertical edge is deleted by using the following rules, I is set as an edge pixel, 0 is set as a background pixel, and if the pixelP(, A satisfies the following formula, the pixel is considered as a vertical edge point:
P(i-1, j-1) & (P(i-2,j-2)=1 P(i-2, j-1)=1 P(i-2, j)=1) or P(i-1, j) & (P(i-2,j-1)=1P(i-2,j)=1|P(i-2, j+1)=1) or P(i-1, j+1) & (P(i-2, jl)=1|P(i-2,j+1)=1| P(i-2, j+2)=1) or P(i+1,j-1) & (P(i+2,j-2)=1 P(i+2,j-1)=1 P(i+2,j)=1) or P(i+1,j) & (P(i+2,j-1)=1P(i+2,j)=1|P(i+2, j+1)=1) or P(i+1,j+1) & (P(i+2,j)=1|P(i+2,j+1)=1|P(i+2,j+2)=1) The image description of the formula is shown in FIG. 7. The result after the above method is used is shown in FIG. 8. Compared with FIG. 7, it can be seen that the vertical edge information can be effectively eliminated by using the algorithm. The fourth step: eliminating the discrete edge. The upper and lower edges of a single heating wire are close to each other in thermal imaging, and the edges of other objects are relatively dispersed. Therefore, the edges of the heating wire will fuse to form an area with a larger connected area after expansion calculation is used, and the pixels after the edges of other objects are expanded will not overlap and thus do not form a connected area. Thus, the discrete edge is eliminated by expanding the image in FIG. 8 for n times and then corroding the image for n+1 times. FIG. 9 shows the result after the image is expanded for three times. FIG. 10 shows the result after the image is corroded for four times. The fifth step: extracting the thermal radiation mark image The image shown in FIG. 10 has multiple connected areas of different sizes, and the area where the thermal radiation mark is located is the largest. Therefore, the connection of the image pixels can be analyzed, and the area with the largest connected area in the image is extracted, which is the area where the thermal radiation mark is located. The result is shown in FIG. 11. The connected area shown in FIG. 11 is the result after the edge image is expanded for three times and then is corroded for four times. Therefore, the edge area of the thermal radiation mark is formed after expanding for one time again on the basis of FIG. 11, thereby obtaining the thermal radiation mark image shown in FIG. 12. The sixth step: recognizing bright and dark stripes. The average gray value of pixels in the thermal radiation mark area in the thermal radiation mark image is calculated. Then, the pixel with the gray value higher than the average value is considered as a heated heating wire image, and the remaining pixels are considered as non-heated heating wire and background images. The processing result is shown in FIG. 13. Bright stripes refer to the heated heating wire image, and dark stripes refer to the non-heated heating wire image. The seventh step: verifying the thermal radiation mark image. According to a column image where the center pixel in the thermal radiation mark area is located, the number L(L1 ,L 2 ,---,L,) of pixels in each bright stripe and the corresponding number D(D, DD2,--,D,) of pixels in each dark stripe in the vertical direction are counted in sequence from top to bottom, and the average value A of the numbers of the pixels in the bright stripes and the pixel sum T of all the stripes are calculated.
A=$Lk/X k=1
T=0Lk+ D k=1 i1
The five heating wires are arranged in parallel at equal intervals. Therefore, in the vertical direction, the average value of pixel heights occupied by the bright stripes is proportional to the pixel height of the whole thermal radiation mark, so that it is considered that the above stripe information satisfies the morphological constraint of thermal radiation mark when the following formula is satisfied. 6*A T 14 *A The eighth step: performing code recognition. For the thermal radiation mark image that satisfies the morphological constraint as
shown in FIG. 14, the coded information Num of the current thermal radiation mark is obtained according to the height information of the bright and dark stripes in the vertical direction by using the following formula.
if x=2 => Num=O if L 2 > 3.5*A= Num =1 if x = 3 else if L 2 > 2.5* A = Num = 2 if L 4 > 3.5 *A= Num = 4 if L 2 > 2.5*A=> Num = 3 if x=4Iif L>2.5*A= Num=6 if L4 > 2.5 *A= Num = 5 if x=5 => Num=7 The complete recognition steps are shown in FIG. 1. The data processing module recognizes the to-be-recognized target by means of the coded information. According to the embodiment, in order to solve the inherent problems that the edges of the infrared thermal images are blurred, the features are difficult to extract, the infrared thermal images are easily affected by the surface characteristics of objects and external factors such as the radiation wavelength and the like, and the brightness of the images can be affected by the navigator clothing thickness and the direction of movement as well as materials of clothing in the target recognition field based on the thermal infrared images, the present disclosure provides the multi-target recognition system for the follow-up robot based on the coded thermal infrared mark. The thermal infrared images recognized by the system have the characteristics of clear structures, obvious features, easiness in distinguishing and the like so as to be accurately distinguished from the background. The heating wires in the thermal radiation mark are coded in a binary form, thereby having the multi-target recognition capability. According to the system, by implementing the steps of performing image filtering, performing edge detection, eliminating the vertical edge, eliminating the discrete edge, extracting the thermal radiation mark image, recognizing bright and dark stripes, determining whether the morphological constraint is satisfied, verifying the thermal radiation mark image and performing code recognition on the infrared image of the thermal radiation mark heated according to the coding rule, the thermal radiation mark can be accurately recognized, and the coded information can be extracted, so that the robot can recognize the target. Embodiment 2 In this embodiment, a multi-target recognition method for a follow-up robot based on a coded thermal infrared mark is disclosed, the method including: collecting a thermal infrared image of a to-be-recognized target; recognizing a thermal radiation mark image from the thermal infrared image; recognizing coded information of the thermal radiation mark according to the thermal radiation mark image; and recognizing the to-be-recognized target by means of the coded information. Furthermore, the specific process of recognizing the thermal radiation mark image from the thermal infrared image includes: performing filtering processing on the thermal infrared image; performing heating wire edge detection on the thermal infrared image after filtering processing so as to obtain an edge detection image; and eliminating the vertical edge and divergent edge of the image after edge detection, and extracting the thermal radiation mark image from the image after edge detection according to the largest area where the thermal radiation mark is located. Furthermore, the specific process of recognizing the coded information of the thermal radiation mark according to the thermal radiation mark image includes: recognizing the curve of the heated heating wires and the curve of the non-heated heating wires in the thermal radiation mark image according to pixel gray values, and determining the coded information of the thermal radiation mark according to height information of the curve of the heated heating wires and the curve of the non-heated heating wires in a vertical direction. Embodiment 3 In this embodiment, an electronic device is disclosed, the electronic device including a memory, a processor and a computer instruction stored in the memory and run on the processor, where when the computer instruction is run by the processor, the steps of the multi-target recognition method for the follow-up robot based on the coded thermal infrared mark in Embodiment 2 are completed. Embodiment 4 In this embodiment, a computer-readable storage medium is disclosed, the computer-readable storage medium being used for storing the computer instruction, where when the computer instruction is executed by the processor, the steps of the multi-target recognition method for the follow-up robot based on the coded thermal infrared mark in Embodiment 2 are completed. The above descriptions are merely preferred embodiments of this application and are not intended to limit this application. For a person skilled in the art, this application may have various modifications and changes. Any modification, equivalent replacement, or improvement made without departing from the spirit and principle of this application shall fall within the protection scope of this application. A person skilled in the art can understand that the embodiments of this application may be provided as a method, a system, or a computer program product. Therefore, this application may use a form of hardware-only embodiments, software-only embodiments, or embodiments combining software and hardware. Moreover, this application may use a form of a computer program product that is implemented on one or more computer-usable storage media (including but not limited to a disk memory, a CD-ROM, an optical memory, and the like) that include computer-usable program code. This application is described with reference to flowcharts and/or block diagrams of the method, the device (system), and the computer program product according to the embodiments of this application. It should be understood that computer program instructions can implement each procedure and/or block in the flowcharts and/or block diagrams and a combination of procedures and/or blocks in the flowcharts and/or block diagrams. These computer program instructions may be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or a processor of another programmable data processing device to generate a machine, so that an apparatus configured to implement functions specified in one or more procedures in the flowcharts and/or one or more blocks in the block diagrams is generated by using instructions executed by the computer or the processor of another programmable data processing device. These computer program instructions may alternatively be stored in a computer-readable memory that can instruct a computer or another programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate an artifact that includes an instruction apparatus. The instruction apparatus implements a specific function in one or more procedures in the flowcharts and/or in one or more blocks in the block diagrams. These computer program instructions may further be loaded onto a computer or another programmable data processing device, so that a series of operations and steps are performed on the computer or the another programmable device, thereby generating computer-implemented processing. Therefore, the instructions executed on the computer or the another programmable device provide steps for implementing a specific function in one or more processes in the flowcharts and/or in one or more blocks in the block diagrams. Finally, it should be noted that: The foregoing embodiments are merely used for describing the technical solution of the present disclosure, but are not intended to limit the present disclosure. Although the present disclosure is described in detail with reference to the foregoing embodiments, it should be appreciated by a person ordinarily skilled in the art that: modifications or equivalent replacements can still be made to the detailed description of the present disclosure, and any modification or equivalent replacement without departing from the spirit and scope of the present disclosure shall fall within the protection scope of claims of the present disclosure.
Claims (5)
1. A multi-target recognition system for a follow-up robot based on a coded thermal infrared mark, comprising: a thermal radiation mark arranged on a to-be-recognized target, a plurality of heating wires arranged in the thermal radiation mark, and the heating wires heated according to different coding rules; and, the heating wires in the thermal radiation mark are arranged in parallel at equal intervals; a thermal infrared image collecting module used for collecting a thermal infrared image of the to-be-recognized target and transmitting to a data processing module; and the data processing module used for recognizing a thermal radiation mark image from the thermal infrared image, recognizing coded information of the thermal radiation mark according to the thermal radiation mark image, and recognizing the to-be-recognized target by means of the coded information; wherein, a specific process that the data processing module recognizes the thermal radiation mark image from the thermal infrared image comprises: performing filtering processing on the thermal infrared image, comprising: removing, by using a median filtering, impulse noise brought by an optical acquisition system from the barcode image; by using a method of sorting to remove median, gray values of interference points become similar to gray values of some adjacent pixels of the interference points, so as to achieve an effect of removing the noise; extracting the edges of the heating wires from the thermal infrared image after filtering processing so as to obtain an edge detection image; eliminating the vertical edge and discrete edge of the edge detection image; wherein, a specific process of the eliminating the vertical edge, comprising: setting 1 as an edge pixel, and setting 0 as a background pixel, and if the pixel P(i,j) satisfies the following formula, the pixel is considered as a
vertical edge point:
P(i-1, j-1) & (P(i-2,j-2)=1|P(i-2,j-1)=1 P(i-2, j)=1) or P(i-1, j) & (P(i-2, j-1)=1 P(i-2, j)= 1l P(i-2, j+1)=1) or P(i-1,j+l) & (P(i-2,jl)=1|P(i-2,j+1)=1|P(i-2,j+2)=1) or P(i, j)=1 P(i+1,j-1) & (P(i+2,j-2)=1P(i+2,j-1)=1|P(i+2, j)=1) or & P(i+1,j) & (P(i+2,j-1)=1P(i+2,j)= 1l P(i+2,j+1)=1) or P(i+l,j+1) & (P(i+2,j)=1|P(i+2,j+1)=1|P(i+2, j+2)=1) and, a specific process of the eliminating the discrete edge, comprising: carrying out a dilation for a set number of times by using a dilation algorithm, then carrying out an erosion for the set number of times + 1 time, so as to realize the elimination of the discrete edge; and, extracting the thermal radiation mark image from the edge detection image according to the largest area where the thermal radiation mark is located; wherein, a specific process that the data processing module recognizes the coded information of the thermal radiation mark according to the thermal radiation mark image, comprises: recognizing the image of the heated heating wires and the image of the non-heated heating wires in the thermal radiation mark image according to pixel gray values, and determining the coded information of the thermal radiation mark according to height information of the image of the heated heating wires and the image of the non-heated heating wires in a vertical direction; wherein, judging, by the data processing module, whether the image of the heated heating wires and the image of the non-heated heating wires in the thermal radiation mark image satisfy a morphological constraint; according to a column image where the center pixel in the thermal radiation mark area is located, the number L of pixels in each bright stripe and the corresponding number D of pixels in each dark stripe in the vertical direction are counted in sequence from top to bottom, and the average value A of the numbers of the pixels in the bright stripes and the pixel sum T of all the stripes are calculated,
A=$LjkIx,and k=1
X y T=$Lk+ DS ; k=1 i1
determining the coded information Num of the thermal radiation mark according to the height information of the image of the heated heating wires and the image of the non-heated heating wires in the vertical direction if the image of the heated heating wires and the image of the non-heated heating wires in the thermal radiation mark image satisfy the morphological constraint 6*A T 14*A; wherein, if x=2 = Num=O, if L 2 >3.5* Ao Num=1 if x=3 else if L 2 > 2.5* A = Num=2, LfL 4 >3.5* A = Num=4 fL2 > 2.5* A = Num=3 if x=4f L6 > 2.5* A = Num=6, LfL4 > 2.5* A = Num=5 if x=5 = Num=7.
2. The multi-target recognition system for the follow-up robot based on the coded thermal infrared mark according to claim 1, wherein the heating wires in the thermal radiation mark are coded by using a binary rule.
3. A multi-target recognition method for the follow-up robot based on the coded thermal infrared mark, wherein the method comprises: collecting a thermal infrared image of a to-be-recognized target; recognizing a thermal radiation mark image from the thermal infrared image, comprising: performing filtering processing on the thermal infrared image, comprising: removing, by using a median filtering, impulse noise brought by an optical acquisition system from the barcode image; by using a method of sorting to remove median, gray values of interference points become similar to gray values of some adjacent pixels of the interference points, so as to achieve an effect of removing the noise; extracting the edges of the heating wires from the thermal infrared image after filtering processing so as to obtain an edge detection image; eliminating the vertical edge and discrete edge of the edge detection image; wherein, a specific process of the eliminating the vertical edge, comprising: setting 1 as an edge pixel, and setting 0 as a background pixel, and if the pixel P(i,j) satisfies the following formula, the pixel is considered as a vertical edge point: P(i-1, j-1) & (P(i-2,j-2)=1|P(i-2,j-1)=1 P(i-2, j)=1) or P(i-1, j) & (P(i-2, j-1)=1 P(i-2, j)= 1l P(i-2, j+1)=1) or P(i-1,j+1)& (P(i-2,jl)=1|P(i-2,j+1)=1|P(i-2,j+2)=1) or P(i, j)=1 & P(i+1,j-1) & (P(i+2,j-2)=1P(i+2,j-1)=1|P(i+2,j)=1) or P(i+1,j) & (P(i+2,j-1)=1P(i+2,j)= 1l P(i+2,j+1)=1) or P(i+1,j+1) & (P(i+2,j)=1|P(i+2,j+1)=1|P(i+2, j+2)=1) and, a specific process of the eliminating the discrete edge, comprising: carrying out a dilation for a set number of times by using a dilation algorithm, then carrying out an erosion for the set number of times + 1 time, so as to realize the elimination of the discrete edge; recognizing coded information of the thermal radiation mark according to the thermal radiation mark image, comprising: recognizing the image of the heated heating wires and the image of the non-heated heating wires in the thermal radiation mark image according to pixel gray values, and determining the coded information of the thermal radiation mark according to height information of the image of the heated heating wires and the image of the non-heated heating wires in a vertical direction; wherein, judging, by the data processing module, whether the image of the heated heating wires and the image of the non-heated heating wires in the thermal radiation mark image satisfy a morphological constraint; according to a column image where the center pixel in the thermal radiation mark area is located, the number L of pixels in each bright stripe and the corresponding number D of pixels in each dark stripe in the vertical direction are counted in sequence from top to bottom, and the average value A of the numbers of the pixels in the bright stripes and the pixel sum T of all the stripes are calculated,
A= Lk/Ix,and k-i
T=$Lk+ D ; k=1 n=1
determining the coded information Num of the thermal radiation mark according to the height information of the image of the heated heating wires and the image of the non-heated heating wires in the vertical direction if the image of the heated heating wires and the image of the non-heated heating wires in the thermal radiation mark image satisfy the morphological constraint 6*A T 14*A, wherein, if x=2 = Num=O,
L 2 >3.5* Ao Num=1 if if x=3 elseif L 2 >2.5*A= Num=2, LfL 4 >3.5* A = Num=4
L 2 > 2.5* A = Num=3 if x=4f L6 > 2.5* A = Num=6, LfL4 > 2.5* A = Num=5
if x = 5 = Num=7;
and, recognizing the to-be-recognized target by means of the coded information.
4. An electronic device, comprising a memory, a processor and a computer instruction stored in the memory and run on the processor, wherein when the computer instruction is run by the processor, the steps of the multi-target recognition method for the follow-up robot based on the coded thermal infrared mark according to claim 3 are completed.
5. A computer-readable storage medium, wherein the computer-readable storage medium is used for storing the computer instruction, and when the computer instruction is executed by the processor, the steps of the multi-target recognition method for the follow-up robot based on the coded thermal infrared mark according to claim 3 are completed.
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202011181170.6A CN112287831B (en) | 2020-10-29 | 2020-10-29 | Following robot multi-target identification system and method based on coded thermal infrared mark |
CN202011181170.6 | 2020-10-29 | ||
PCT/CN2021/074877 WO2022088544A1 (en) | 2020-10-29 | 2021-02-02 | Following robot multi-target identification system and method based on coded thermal infrared marker |
Publications (3)
Publication Number | Publication Date |
---|---|
AU2021368390A1 AU2021368390A1 (en) | 2023-06-22 |
AU2021368390B2 true AU2021368390B2 (en) | 2024-03-28 |
AU2021368390A9 AU2021368390A9 (en) | 2024-09-12 |
Family
ID=74352388
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
AU2021368390A Active AU2021368390B2 (en) | 2020-10-29 | 2021-02-02 | Multi-target recognition system and method for follow-up robot based on coded thermal infrared mark |
Country Status (3)
Country | Link |
---|---|
CN (1) | CN112287831B (en) |
AU (1) | AU2021368390B2 (en) |
WO (1) | WO2022088544A1 (en) |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112287831B (en) * | 2020-10-29 | 2022-11-04 | 齐鲁工业大学 | Following robot multi-target identification system and method based on coded thermal infrared mark |
CN118314332B (en) * | 2024-06-07 | 2024-08-30 | 海信集团控股股份有限公司 | Target identification method and device and intelligent equipment |
Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110298334A (en) * | 2019-07-05 | 2019-10-01 | 齐鲁工业大学 | Tracking robot multi-targets recognition device based on infrared image processing |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN208402864U (en) * | 2018-06-25 | 2019-01-22 | 龙勇 | Unmanned plane warm-blooded animal based on thermal imaging monitors system |
CN110320523B (en) * | 2019-07-05 | 2020-12-11 | 齐鲁工业大学 | Target positioning device and method for following robot |
CN110834132B (en) * | 2019-11-28 | 2021-07-09 | 西南交通大学 | Method for manufacturing aluminum alloy flange arc fuse wire additive on bottom of ellipsoidal box |
CN111354011A (en) * | 2020-05-25 | 2020-06-30 | 江苏华丽智能科技股份有限公司 | Multi-moving-target information capturing and tracking system and method |
CN112287831B (en) * | 2020-10-29 | 2022-11-04 | 齐鲁工业大学 | Following robot multi-target identification system and method based on coded thermal infrared mark |
-
2020
- 2020-10-29 CN CN202011181170.6A patent/CN112287831B/en active Active
-
2021
- 2021-02-02 WO PCT/CN2021/074877 patent/WO2022088544A1/en active Application Filing
- 2021-02-02 AU AU2021368390A patent/AU2021368390B2/en active Active
Patent Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110298334A (en) * | 2019-07-05 | 2019-10-01 | 齐鲁工业大学 | Tracking robot multi-targets recognition device based on infrared image processing |
Also Published As
Publication number | Publication date |
---|---|
AU2021368390A9 (en) | 2024-09-12 |
AU2021368390A1 (en) | 2023-06-22 |
CN112287831B (en) | 2022-11-04 |
WO2022088544A1 (en) | 2022-05-05 |
CN112287831A (en) | 2021-01-29 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN112712512A (en) | Hot-rolled strip steel scab defect detection method and system based on artificial intelligence | |
CN106934795B (en) | A kind of automatic testing method and prediction technique of glue into concrete beam cracks | |
CN110349207B (en) | Visual positioning method in complex environment | |
CN110315525A (en) | A kind of robot workpiece grabbing method of view-based access control model guidance | |
CN110084243B (en) | File identification and positioning method based on two-dimensional code and monocular camera | |
EP2570966A2 (en) | Fast obstacle detection | |
AU2021368390B2 (en) | Multi-target recognition system and method for follow-up robot based on coded thermal infrared mark | |
CN112861654B (en) | Machine vision-based famous tea picking point position information acquisition method | |
CN108734172A (en) | Target identification method, system based on linear edge feature | |
Li et al. | Road markings extraction based on threshold segmentation | |
Kortli et al. | Efficient implementation of a real-time lane departure warning system | |
Duong et al. | Near real-time ego-lane detection in highway and urban streets | |
CN114529715B (en) | Image identification method and system based on edge extraction | |
CN110705553B (en) | Scratch detection method suitable for vehicle distant view image | |
CN115661110A (en) | Method for identifying and positioning transparent workpiece | |
CN116188763A (en) | Method for measuring carton identification positioning and placement angle based on YOLOv5 | |
Gal | Automatic obstacle detection for USV’s navigation using vision sensors | |
KR100691855B1 (en) | Apparatus for extracting features from image information | |
Gupta et al. | Robust lane detection using multiple features | |
CN113538483B (en) | Coding and decoding method and measuring method of high-precision close-range photogrammetry mark | |
Damaryam et al. | A Pre-processing Scheme for Line Detection with the Hough Transform for Mobile Robot Self-Navigation | |
CN115019306A (en) | Embedding box label batch identification method and system based on deep learning and machine vision | |
Antunes et al. | All-in-focus imaging using a series of images on different focal planes | |
CN113505811A (en) | Machine vision imaging method for hub production | |
CN107480672A (en) | Image-recognizing method and system and autofocus control method and system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
DA3 | Amendments made section 104 |
Free format text: THE NATURE OF THE AMENDMENT IS: AMEND THE INVENTION TITLE TO READ MULTI-TARGET RECOGNITION SYSTEM AND METHOD FOR FOLLOW-UP ROBOT BASED ON CODED THERMAL INFRARED MARK |
|
FGA | Letters patent sealed or granted (standard patent) | ||
SREP | Specification republished |