TWI802514B - Processing method of target identification for unmanned aerial vehicle (uav) - Google Patents

Abstract

This invention includes the steps of image pre-process, target image acquisition, target image-based tracking, color identification, body segment identification and length calculation, target position identification, and target image refinement. By these steps, the unmanned aerial vehicle (UAV) can track a target person using an on-board light- weight micro-controller unit (MCU). By utilizing color database and dressing database and target position prediction, the MCU calculates and sums up the individual scores corresponding to color, length, and target position. Finally, the person with the highest score within the captured image from the on-board camera is recognized, locked and tracked. Hence, this invention has presented an effective and low-cost scoring method which is feasible to be realized within on-board MCU of the UAV for the purpose of as a personal security guard. Accuracy of target identification of this method is relatively high while considering the limited computational resource of on-board MCU. In addition, parameters of the scoring system can be flexibly adjusted.

Description

The Processing Method of UAV Target Recognition

The present invention relates to a processing method for UAV target identification, especially a processing method for UAV target identification with a simple and fast scoring system, high judgment accuracy, reduced computer computing resources, and flexible fine-tuning of parameters. method.

In recent years, the development of drones has made great progress, not only can be used in military reconnaissance, police search and rescue, commercial photography, agricultural applications (spraying pesticides or seeding) and other fields, but also can be used in media (or self-recorded programs) The host or Internet celebrity) tracking and filming. To put it simply, a person walks in front, followed by a drone, and introduces it while walking (such as an introduction and tour of a city's attractions) or as a personal bodyguard. At this time, the drone must have tracking The ability of a specific (person) target, and always maintain a fixed distance (for example, follow someone 5 meters behind, and maintain a height of 2 meters).

For traditional UAV target recognition, it is no problem to lock on one person to follow the shot. However, when other people mix into the UAV's shooting range, there may be confusion, and the situation of following the wrong person may occur.

Secondly, drones are small in size, limited in weight, and have limited battery capacity, so they cannot carry heavy image processing (such as distinguishing specific targets) equipment; Artificial Intelligence (AI for short; and it is a well-known AI technology, will not be described in detail) Calculation-based identification is an example. AI-specific microcomputers need to waste a lot of computing resources, which is not suitable for the space of drones and their limited battery capacity.

However, there is currently no such UAV target recognition technology that can quickly distinguish different people on the market.

The traditional UAV target recognition technology is mainly based on location information. Therefore, when the target person overlaps with another person and then separates, the UAV target recognition often makes mistakes.

Therefore, the industry has been looking for a simpler and more accurate discrimination mechanism to be applied to the target recognition of UAVs.

In view of this, must develop the technology that can solve above-mentioned conventional shortcoming.

The purpose of the present invention is to provide a processing method for UAV target recognition, which has the advantages of simple and fast judgment mechanism based on the scoring system, high judgment accuracy, and can reduce computer computing resources, so it is suitable for on-board computers and can flexibly fine-tune parameters Etc. In particular, the problem to be solved by the present invention is that the traditional UAV target recognition technology is mainly based on location information, so when the target person overlaps with another person and then separates, the UAV target recognition often makes mistakes. However, recognition based on artificial intelligence (AI) calculations and microcomputers need to waste a lot of computing resources, and are not suitable for drones with limited space and battery capacity.

The technical means to solve the above problems is to provide a processing method for UAV target recognition, which includes the following steps: 1. Preparation step; 2. Capture target image step; 3. Continuously track target image step; 4. Color recognition calculation step; 5. the calculation step of length identification; 6. the calculation step of position identification; and 7. the step of completing the target image identification.

The above objects and advantages of the present invention can be easily understood from the detailed description of the following selected embodiments and the accompanying drawings.

The present invention is hereafter described in detail with the following embodiments and accompanying drawings:

10: Drone

11: Color database

12: Clothing appearance database

13: Identification database

20: target person

31: Miscellaneous Characters

S1: Preparatory steps

S2: the step of capturing the target image

S3: Continuously track target image steps

S4: color recognition calculation steps

S5: length identification calculation steps

S6: Position identification calculation steps

S7: Complete the step of identifying the target image

M1: The first drone view image

M2: Second drone view image

A: The image of the target person

A1: target head image

A2: Target upper body image

A3: target lower body image

B1, B2: Miscellaneous figures images

B11: Miscellaneous Head Image

B12: Miscellaneous upper body images

B13: Miscellaneous lower body images

L11: target head length

L12: Target lower body length

L21: idle head length

L22: casual lower body length

(Xp, Yp, Zp): 3D coordinates of the target

(Xb, Yb, Zb): three-dimensional coordinates of idle characters

Fig. 1 is a flowchart of the processing method of the present invention.

Fig. 2 is a schematic diagram of the captured target image of the present invention.

Figure 3 is a schematic diagram of the first UAV field of view image obtained in Figure 2.

Figure 4 is an enlarged schematic diagram of the image of the person in Figure 3.

Fig. 5 is a schematic diagram of the continuous tracking target image of the present invention.

Figure 6 is a schematic diagram of the second UAV field of view image obtained in Figure 5.

Fig. 7 is an enlarged schematic diagram of one of the figures in Fig. 6.

Fig. 8 is a graph of the sigmoid function of the present invention.

Attachment: It is a reference photo of the second UAV field of view image of the present invention.

Referring to Figures 1, 2, 3 and 4, the present invention is a processing method for UAV target identification, which includes:

1. Preparation step S1 : prepare a drone 10 , which has a color database 11 , a clothing appearance database 12 and an identification database 13 connected to each other. The color database 11 is built with multiple color systems, and the multiple color systems at least include red, orange, yellow, green, blue, purple, black and white. The clothing and appearance database 12 is built with multiple items of clothing and appearance information, and the multiple items of appearance information at least include long hair, short hair, trousers and shorts.

2. Capturing target image step S2: controlling the UAV 10 to track a target person 20 at a time=t to obtain a first UAV field of view image M1, the first UAV field of view image M1 has a The target person image A, the target person image A includes a target head image A1 , a target upper body image A2 and a target lower body image A3 . in: The target head image A1 has a first length multiplied by a first width of pixels, and the color system occupying the largest number of pixels in the target head image A1 is defined as a target head color; the first length is is defined as a target header length L11.

The target upper body image A2 has a second length multiplied by a second width pixels, and the color system occupying the largest number of pixels in the target upper body image A2 is defined as a target upper body color.

The target lower body image A3 has a third length multiplied by a third width of pixels, the color system occupying the largest number of pixels in the target lower body image A3 is defined as a target lower body color, and the third length is defined as A target lower body length is L12.

Also, by obtaining the shooting angle of the UAV 10 and the distance between the UAV 10 and the target person 20, the spatial position of the target person 20 can be converted from the identification database 13 and defined as a target three-dimensional coordinate (Xp, Yp, Zp).

3. Continuously track the target image Step S3: Referring to Figure 5, control the drone 10 to continuously track the target person 20 at a time=t+1 to obtain a second drone field of view image M2 (as in Figure 6 As shown in the figure and the appendix), the second UAV field of view image M2 has the target person image A and at least one idle person image (as shown in Figure 6, such as B1, B2, ...), and the at least one idle person image is Corresponding to a person 31 adjacent to the target person 20, and the at least one person image includes a head image (refer to FIG. 7, for example, B11), an image of the upper body of the person (for example, B12) and a lower body image of the person Image (eg B13). in: The miscellaneous head image B11 has a fourth length multiplied by a fourth width of pixels, and the color system occupying the largest number of pixels in the miscellaneous head image B11 is defined as a miscellaneous head color; the fourth length is is defined as an idle header length L21.

The miscellaneous upper body image B12 has a fifth length multiplied by a fifth width of pixels, and the color system occupying the largest number of pixels in the miscellaneous upper body image B12 is defined as a miscellaneous upper body color.

The miscellaneous lower body image B13 has a sixth length multiplied by a sixth width pixel, and the color system occupying the largest number of pixels in the miscellaneous lower body image B13 is defined as a miscellaneous lower body color, and the sixth length is defined as A leisurely lower body length L22.

Also, by obtaining the shooting angle of the UAV 10 and the distance between the UAV 10 and the at least one idle person 31, the spatial position of the idle person 31 can be converted from the identification database 13, and it can be defined as an idle person. The three-dimensional coordinates (Xb, Yb, Zb) of the character.

4. Color recognition calculation step S4: the recognition database 13 has a built-in color score, which is preset as 0 points, and when the recognition database 13 recognizes that the color of the idle head is equal to the color of the target head, then Add 1 point to the color score, otherwise add 0 points; when the recognition database 13 recognizes that the upper body color of the idle miscellaneous is equal to the upper body color of the target, then add 1 point, otherwise add 0 points; when the identification database 13 identifies the idle miscellaneous When the color of the lower body is equal to the color of the lower body of the target, add 1 point, otherwise add 0 points.

5. Length identification calculation step S5: the identification database 13 has a built-in length score, which is preset as 0 points, and when the identification database 13 identifies that the length L21 of the idle head is equal to the length L11 of the target head When ±N%, then add 1 point to the length score, otherwise add 0 points; and 0<N<20; when the identification database 13 identifies that the idle lower body length L22 is equal to ±N% of the target lower body length L12, Add 1 point, otherwise add 0 points; and 0<N<20.

6. Position identification calculation step S6: The identification database 13 is at least built in the following (Formula 1) and (Formula 2):

Among them: position error=position error; ε=coefficient, which is 0.0001<ε<0.1.

PS = A × sigmoid ( s ( position error-bias ))- A /2 (Formula 2)

Where: A=(color score+length score)×2; s=steepness; and bias=offset.

And the identification database 13 uses the above (formula 1) to calculate the position error (position error), and then preset the offset to 0, the steepness to 1, and the A=color score+length score, Substituting into formula 2, the position score (PS) can be calculated;

7. Complete the step S7 of identifying the target image: the identification database 13 is built with the following (Formula 3):

Among them: w ₁ , w ₂ and w ₃ are weighting factors ( w ₁ + w ₂ + w ₃ =1); S=total score; RR=recognition rate; CS=color score; CFS=color full score; LS=length score ; LFS = Length Full Score; PS = Position Score; and PFS = Position Full Score.

Using formula 3, the total score (S) can be calculated.

The person image corresponding to the highest total score in the second UAV field of view image M2 is the target person image A, and then the recognition is completed.

For example: first, the first person is a specific target, and the known conditions are: black hair (the color database 11), short hair (the clothing appearance database 12), orange tops (the color database 11 and the clothing appearance database 12) database 12), blue trousers (the color database 11 and the clothing appearance database 12), trousers (the clothing appearance database 12).

In the photo, there are 6 people to be tested (as shown in Figure 6), as shown in Table 1 and Table 2 below, and the first 3 people are used as an example to illustrate:

Then the first person: black hair, short hair, orange jacket, blue pants, trousers; color score is 3 points; length score is 2 points; So a total of five points (3+2=5).

位置誤差=1/(0+0.01)=100；Sigmoid(100)=1(如第8圖所示)；位置分數=(5*2)*Sigmoid(1(100-0))-5；=10*1-5；=5。 A=(color score + length score)x2; A=5*2; S=1; Bias=0; And the three-dimensional coordinates of idle characters (Xb, Yb, Zb)=(0,0,0); so according to the following formula:

Position error=1/(0+0.01)=100; Sigmoid(100)=1 (as shown in Figure 8); Position score=(5*2)*Sigmoid(1(100-0))-5;= 10*1-5;=5.

Therefore, the total score of the first person = 3 + 2 + 5 = 10 points.

Person 2: Black hair, short hair, blue jacket, blue pants, long pants; 2 points for color; (black hair and blue pants); 2 points for length; (short hair and long pants); 4 points in total.

A=(color score + length score)x2; A=(2+2)*2; A=4*2; S=1; Bias=0; because the three-dimensional coordinates of the target character (Xp, Yp, Zp)=(0,0,0); and the three-dimensional coordinates of the idle characters (Xb,Yb,Zb)=(0 ,1,0); Position error=1/(1+0.01)=0.99 is approximately equal to 1; Sigmoid(1)=0.5; Position score=(4*2)*Sigmoid(1(1-0))-4; =8*0.5-4;=0.

Therefore, the total score of the third person = 2 + 2 + 0 = 4 points.

Person 3: black hair, long pants, short hair; 1 point for color; (black hair); 2 points for length; (short hair and long pants); 3 points for (black hair, long pants, and short hair); A =3*2; S=1; Bias=0; because the three-dimensional coordinates of the target character (Xp, Yp, Zp)=(0,0,0); and the three-dimensional coordinates of the idle characters (Xb,Yb,Zb)=(2, 0,0); Position error=1/(4+0.01)=0.25 is approximately equal to 1/4; Sigmoid(0.25)=0.125; Position score=(3*2)*Sigmoid(1(0.25-0))-3;=3*0.125-3;=-2.625; therefore, the total score of the third person=1+2+(-2.625)= 0.375 points.

Advantage and effect of the present invention are as follows:

[1] The judgment of the scoring system is quite special. In this case, in addition to the judgment of the spatial position (equivalent to the position score), the judgment of "color score" and "length score" is also introduced, which is different from the recognition of the whole image in the known technology. Therefore, it is quite special to adopt the scoring system to judge.

[2] The accuracy of judgment is high. This case mainly uses the comprehensive judgment of "position score", "color score" and "length score". Therefore, the overall accuracy is higher than the traditional judgment that only uses spatial position (equivalent to position score).

[3] It can reduce computer computing resources and is therefore suitable for on-board computers. In this case, only the "Position Score", "Color Score" and "Length Score" in the image are extracted for calculation and comparison. There is no need to perform calculation and comparison on a large number of images, which can greatly reduce the amount of calculation. Therefore, the computing resource of the computer can be reduced and it is applicable to the on-board computer.

[4] The parameters can be fine-tuned flexibly. Since the S-shaped function (Sigmoid function or S-function) is used in this case, there is a steepness (s) and an offset (bias) that can be adjusted. If the original preset steepness (s) = 1 and the offset When the amount (bias)=0 is not suitable, you can flexibly modify the steepness (s) and offset (bias), and it may be adjusted to a better situation. Therefore, you can flexibly fine-tune the parameters.

The above is only a detailed description of the present invention through preferred embodiments, and any simple modifications and changes made to the embodiments will not depart from the spirit and scope of the present invention.

S1: Preparatory steps

S2: the step of capturing the target image

S3: Continuously track target image steps

S4: color recognition calculation steps

S5: length identification calculation steps

S6: Position identification calculation steps

S7: Complete the step of identifying the target image

Claims (3)

A processing method for UAV target identification, which includes: 1. Preparing steps: prepare a UAV, which has a color database, a clothing and appearance database, and an identification database connected to each other; the color database A plurality of color systems are built in; the clothing and appearance database is built with a plurality of clothing and appearance information; 2. The step of capturing target images: control the drone to track a target person at a time = t to obtain a The first UAV field of view image, the first UAV field of view image has a target person image, and the target person image includes a target head image, a target upper body image and a target lower body image; wherein: the target head image It has a first length multiplied by a first width of pixels, and the color system occupying the largest number of pixels in the target head image is defined as a target head color; the first length is defined as a target head length; the target upper body image has a second length multiplied by a second width of pixels, and the color system occupying the largest number of pixels in the target upper body image is defined as a target upper body color; the target lower body image has a first Three lengths multiplied by a third width pixel, the color system occupying the largest number of pixels in the target lower body image is defined as a target lower body color, and the third length is defined as a target lower body length; The shooting angle of the drone and the distance between the drone and the target person can be converted from the recognition database to the spatial position of the target person, and defined as a target three-dimensional coordinates (Xp, Yp, Zp); 3. The step of continuously tracking the target image: controlling the UAV to continuously track the target person at a time=t+1 to obtain a second UAV field of view image, the second UAV field of view image has the target person image and at least An image of a person, the at least one image of a person corresponds to a person adjacent to the target person, and the at least one image of a person includes a head image, an image of an upper body and an image of a lower body; wherein: the The miscellaneous head image has a fourth length multiplied by a fourth width of pixels, the color system occupying the largest number of pixels in the miscellaneous head image is defined as a miscellaneous head color, and the fourth length is defined as A length of the head; the upper body image has a fifth length multiplied by a fifth width pixels, and the color system occupying the largest number of pixels in the upper body image is defined as an upper body color; the lower body image It has a sixth length multiplied by a sixth width pixel, the color system occupying the largest number of pixels in the lower body image is defined as a lower body color, and the sixth length is defined as a lower body length; and , obtain the shooting angle of the UAV, and the distance between the UAV and the at least one idle person, the spatial position of the idle person can be converted from the identification database, and be defined as a three-dimensional coordinate of an idle person (Xb, Yb, Zb); 4. Calculation steps for color recognition: a color score is built into the recognition database, which is preset to 0 points, and when the recognition database recognizes that the color of the idle head is equal to the color of the target head , then add 1 point to the color score, otherwise add 0 points; when the recognition database recognizes that the color of the upper body of the idler is equal to the color of the upper body of the target, add 1 point, otherwise add 0 points; when the recognition database identifies the lower body of the idler When the color is equal to the color of the lower body of the target, add 1 point, otherwise add 0 points; 5. Length identification calculation steps: The identification database has a built-in length score, which is 0 points by default, and is used as the identification database When it is identified that the head length of the idler is equal to ±N% of the head length of the target, add 1 point to the length score, otherwise add 0 points, and 0<N<20; when the identification database identifies the length of the lower body of the idler as equal to When the length of the lower body of the target is ±N%, add 1 point, otherwise add 0 points, and 0<N<20; 6. Position recognition calculation steps: The recognition database is built with at least the following (formula 1) and (formula 2):

Among them: position error=position error; ε=coefficient, its system is 0.0001<ε<0.1; PS = A × sigmoid ( s ( position error-bias ))- A /2 (formula 2) where: A=(color score+ length fraction)×2; s=steepness; and bias=offset; and the identification database uses the above (formula 1), first calculates the position error (position error), and then presets the offset to 0, Steepness is preset to 1, and the A=color score+length score, and then substituted into formula 2, the position score (PS) can be calculated; 7. Complete the steps of identifying the target image: the identification database is built in the following (formula 3 ):

Among them: w ₁ , w ₂ and w ₃ are weighting factors ( w ₁ + w ₂ + w ₃ =1); S=total score; RR=recognition rate; CS=color score; CFS=color full score; LS=length score ;LFS=Length Full Score; PS=Position Score; and PFS=Position Full Score; use formula 3 to calculate the total score (S); The image of the person, and then complete the identification. The processing method for UAV target recognition according to Claim 1, wherein the plurality of colors at least includes red, orange, yellow, green, blue, purple, black and white. The method for processing drone target recognition as described in Claim 1, wherein the plurality of pieces of clothing and appearance information at least include long hair, short hair, trousers and shorts.

Images