CN108961838B - Road pedestrian classification system - Google Patents
Road pedestrian classification system Download PDFInfo
- Publication number
- CN108961838B CN108961838B CN201810936562.5A CN201810936562A CN108961838B CN 108961838 B CN108961838 B CN 108961838B CN 201810936562 A CN201810936562 A CN 201810936562A CN 108961838 B CN108961838 B CN 108961838B
- Authority
- CN
- China
- Prior art keywords
- pedestrians
- pedestrian
- probability
- model
- magnetic force
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/16—Anti-collision systems
- G08G1/166—Anti-collision systems for active traffic, e.g. moving vehicles, pedestrians, bikes
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/005—Traffic control systems for road vehicles including pedestrian guidance indicator
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
- H04W4/02—Services making use of location information
- H04W4/021—Services related to particular areas, e.g. point of interest [POI] services, venue services or geofences
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
- H04W4/02—Services making use of location information
- H04W4/025—Services making use of location information using location based information parameters
- H04W4/027—Services making use of location information using location based information parameters using movement velocity, acceleration information
Landscapes
- Engineering & Computer Science (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Traffic Control Systems (AREA)
Abstract
A road pedestrian classification system belongs to the field of moving target tracking and processing, and is used for solving the problem of enriching the existing road pedestrian classification, a plurality of instructions are stored, and the instructions are suitable for being loaded and executed by a processor: the vehicle-mounted camera shoots images of the pedestrians on the roads, and the pedestrians on the roads are classified according to the magnetic force relation existing among the pedestrians on the roads displayed by the images; wherein: the magnetic force relationship is characterized by the moving speed of the pedestrian, the relative distance between the pedestrians in the image and the relative distance between the pedestrian and the camera, the moving speed of the pedestrian, the relative distance between the pedestrians in the image and the relative distance between the pedestrian and the camera are classified through the magnetic force relationship, and the magnetic force relationship is determined.
Description
Technical Field
The invention belongs to the field of moving target tracking processing, and particularly relates to a classification method for distinguishing potential danger degrees of pedestrians on a road by using a magnetic model.
Background
The moving target tracking processing technology is an important research subject in the field of machine vision, and with the application of autonomous automobiles and auxiliary driving systems, how to reasonably apply the target tracking processing technology to protect the safety of pedestrians and vehicles is a popular research direction at present.
At present, under the condition of only using a vehicle-mounted camera, classifying pedestrians by analyzing information such as historical moving tracks and moving speeds of pedestrians is a main approach for protecting the safety of pedestrians and vehicles by utilizing a target tracking processing technology. Firstly, the moving track and the moving speed of the pedestrian are analyzed to calculate the probability that the pedestrian collides with the vehicle, and then the pedestrian is classified into different types by utilizing the collision probability and establishing corresponding classification rules. Most of the existing road pedestrian classification methods in the target tracking processing technology calculate the probability that a pedestrian will directly collide with a vehicle, or only detect special pedestrians with specific characteristics.
The patent application number is CN201610048194.1, and the name is 'blind person detection and identification method and system based on combined features and vehicle-mounted camera', firstly, the HOG features are utilized to detect pedestrians, and then three classification features, namely a blind guiding dog, a blind stick and a blind mirror, are used near the detection result of the pedestrians to detect whether blind persons exist.
The patent application number is CN201610048233.8, entitled traffic police detection method and system based on coat-hat characteristic posture detection, and uses the police hat and fluorescent waistcoat of the traffic police in the road as classification characteristics to judge whether the pedestrian detection result has the traffic police. Although a specific target can be well identified by detecting a specific feature of the target, the method has a great limitation because only the specific target can be detected.
In the article "Analysis of pedestrian condensation using Fuzzy information model", haiyono et al analyze the moving speed and moving direction of pedestrians and vehicles and the relative distance between the pedestrians and vehicles, calculate the Collision probability of the pedestrians and the vehicles, and classify the pedestrians according to the size of the Collision probability. However, in order to accurately acquire speed and position information of pedestrians, the method uses a camera erected on the roadside to shoot the road condition in front of the vehicle from the side, so that the method is greatly influenced by the environment.
In an article, Hariyono and the like in the Estimation of fusion Risk for Improving Driver's Safety establishes a dangerous area in front of a vehicle, and pedestrians entering the dangerous area are judged as dangerous pedestrians so as to be classified. However, the existence of many sudden conditions on the road may cause the pedestrian to be instantly changed from a normal pedestrian to a dangerous pedestrian. For example, a pedestrian holding a mobile phone suddenly decelerates while passing through a road, causing the pedestrian who is in a safe state to suddenly become a pedestrian who may collide with the vehicle.
The road pedestrian classification method used by the invention is to establish a magnetic model, and judge the potential danger of the pedestrian by analyzing the physical logic relationship between the pedestrians, rather than estimating the danger of direct collision between the pedestrian and the vehicle. The pedestrians with high potential danger are used as main analysis objects by establishing a new road pedestrian classification rule, so that the classification results of the existing road pedestrian classification method are enriched, and the safety of vehicles and pedestrians is better protected.
Disclosure of Invention
In order to solve the problem of enriching the existing road pedestrian classification, the invention provides the following technical scheme: a road pedestrian classification system storing a plurality of instructions adapted to be loaded and executed by a processor: the vehicle-mounted camera shoots images of the pedestrians on the roads, and the pedestrians on the roads are classified according to the magnetic force relation existing among the pedestrians on the roads displayed by the images;
wherein: the magnetic force relationship is characterized by the moving speed of the pedestrians, the relative distance between the pedestrians in the image and the relative distance between the pedestrians and the camera.
Further, the magnetic force relationship is represented by a magnetic force model, the magnetic force model comprises a repulsion force model, an attraction force model and a non-magnetic force model, the probability of each specific model in the magnetic force relationship displayed in the current image is calculated, and if the probability of the specific model exceeds a probability threshold value, the magnetic force relationship of the road pedestrian in the current image is reflected by the current model.
Further, the repulsion model is defined as: the pedestrians who run in opposite directions have collision routes in a plurality of traveling routes, and the moving speed of the pedestrians is reduced or even stopped when the pedestrians meet each other until the speed is recovered after meeting is finished.
Further, the magnetic force relationship of the pedestrian displayed by the current image is the probability of the repulsive force model, which is calculated by the following formula:
when the pedestrians i and j move in opposite directions and approach each other continuously, the probability P of repulsion isrWill gradually increase until the probability of repulsion exceeds and is judged as a magnetic pedestrian; wherein: priProbability of repulsion for pedestrian i; sijIs the distance between pedestrians i and j; dijIs the difference between the distances of the pedestrians i and j, respectively, from the camera, and dij＝|zi-zjL, wherein ziIs the distance between the pedestrian i and the camera, zjTo moveThe distance between a person j and the camera, m and n are constant coefficients, and m, n ∈ (0, 1); viAnd vjThe moving speeds, v, of pedestrians i and j, respectivelyivj< 0 indicates that the pedestrians i and j are traveling in opposite directions.
Further, the suction model is defined as: among a plurality of pedestrians walking in the same direction, two or more pedestrians walk in an overlapped or adjacent state, and the moving speeds of the pedestrians are similar in direction and magnitude.
Further, the magnetic force relationship of the pedestrian displayed by the current image is the probability of the attraction model calculated by the following formula:
when pedestrians i and j continuously walk in the same-direction overlapped or adjacent state, the suction probability PsiGradually increasing until the suction probability exceeds so as to be judged as a suction pedestrian; wherein: psiThe probability of suction being a pedestrian i; sijIs the distance between pedestrians i and j; dijIs the difference between the distances of the pedestrians i and j and the camera, and dij＝|zi-zjL, wherein ziIs the distance between the pedestrian i and the camera, zjIs the distance between the pedestrian j and the camera, m, n and l are constant coefficients, and m, n, l ∈ (0, 1); viAnd vjThe moving speeds, v, of pedestrians i and j, respectivelyivj> 0 indicates that pedestrians i and j are traveling in the same direction; k is a radical ofijIs a parameter of suction, and kij＝-[min(T|vi|,T|vj|)/sij]Where T is the elapsed time, min (T | v)i|,T|vj|) represents a smaller displacement value of the pedestrians i and j after the time T.
Further, the nonmagnetic model is defined as: among the several pedestrians, there are at least three pedestrians, one of which has a higher or lower speed than the other at least two pedestrians.
Further, the probability that the magnetic force relationship of a pedestrian displayed by the current image is a no-magnetic force model is calculated by the following formula:
the pedestrians i, j and h walk in the same direction, the speed of one pedestrian is obviously higher than or lower than the speeds of the other two pedestrians, the speeds of the other two pedestrians are equal, the current speeds of the three pedestrians are kept unchanged, and the magnetic probability P is not generatedmExceeds to be determined as a non-magnetic pedestrian; wherein: pmiThe magnetic force free probability of the pedestrian i; v. ofi、vjAnd vhThe moving speeds of the pedestrians i, j and h respectively, the speed of the pedestrian i is obviously higher than or lower than the speeds of the other two pedestrians, and the speeds of the three pedestrians meet the following conditions:
(vi 2-vj 2)+(vi 2-vh 2)＞2
then P ismiEqual to 1, otherwise 0; is a constant coefficient.
Further, the specific method for classifying the pedestrian is as follows: calculating the magnetic force probability of a certain pedestrian in the current image:
PMM＝{Pr,Ps,Pm}
when no model probability exceeds a probability threshold, the pedestrian i is a common pedestrian; when only one model probability exceeds a probability threshold, the pedestrian i is judged as a corresponding type of magnetic pedestrian; when more than two model probabilities exceed the probability threshold, the pedestrian i is judged as a magnetic pedestrian with higher priority according to the priority of the model probabilities; setting: when P is presentrPedestrians are repulsive when exceeding; when P is presentsThe pedestrians are attracted by the people when the pedestrian exceeds the time; when P is presentmThe pedestrians are non-magnetic pedestrians when the vehicle exceeds the set time; three kinds of magnetic force probability PmOf highest priority, PsIs centered on the priority of PrIs lowest.
Has the advantages that: according to the invention, through analyzing common road condition environments, pedestrians which are easy to cause accidents are found out through the scheme, the pedestrians do not necessarily collide with vehicles, the moving speed of the pedestrians, the relative distance between the pedestrians in the image and the relative distance between the pedestrians and the camera are classified through the magnetic force relationship, and the determination of the magnetic force relationship can be used as an important reference basis for obstacle avoidance and path planning of an autonomous automobile or an auxiliary driving system.
Drawings
FIG. 1 is a logic diagram of a magnetic force model based road pedestrian classification method;
fig. 2 is a diagram illustrating classification results of repulsive pedestrians;
FIG. 3 is a diagram illustrating classification results of suction pedestrians;
FIG. 4 is a diagram illustrating classification results of a non-magnetic pedestrian;
FIG. 5 is a diagram illustrating classification results of various magnetic pedestrians;
Detailed Description
The invention is further described with reference to the accompanying drawings and the specific classification procedures:
a method for judging pedestrians of interest of a magnetic force model is shown in a logic block diagram of fig. 1, and the method is implemented by the following steps:
step 1: inputting a moving speed v of the pedestrian, a relative distance s between the pedestrians in the image, and a relative distance z between the pedestrian and the camera;
step 2: calculating the magnetic force probability P of each pedestrian in the imageMM；
And 3, step 3: according to PMMAnd classifying the pedestrians according to the three magnetic force probabilities and outputting classification results.
To describe the above scheme in more detail and distinguish the existing direct collision probability calculation of pedestrians and vehicles, the present invention provides a more complete description of the above scheme, that is, the present invention estimates the potential danger of pedestrians by establishing a magnetic model, and is a road pedestrian classification method based on the magnetic model, which can be implemented by software, and relates to a system executing the software, which stores a plurality of instructions, which are suitable for being loaded by a processor and executing the method or algorithm.
Under the condition of only using the vehicle-mounted camera, if the information of the moving speed, the moving direction, the distance from the camera, the relative position between pedestrians and the like of the pedestrians is known, the magnetic force model can be combined for analysis, the conformity degree of the classification conditions set by the pedestrians and the magnetic force model is calculated, the pedestrians are classified into common pedestrians and magnetic force pedestrians, the classification result can further enrich the information of the pedestrians on the road obtained by the existing target tracking processing technology, and more sufficient road condition reference information is provided for the autonomous automobile and the auxiliary driving system.
The above object is achieved by the following technical solutions.
In the first step, a moving speed v of the pedestrian, a relative distance s between the pedestrians in the image, and a relative distance z between the pedestrian and the camera are input.
And secondly, establishing a magnetic model. The magnetic force model is divided into three submodels of a repulsion force model, a suction force model and a non-magnetic force model, the three submodels are independent of each other and do not influence each other, and respectively correspond to a magnetic force probability: pr(probability of repulsion), Ps(suction probability), Pm(probability of no magnetic force). The total magnetic probability P of the pedestrian is the collection of the three magnetic probabilitiesMMAnd is and
PMM＝{Pr,Ps,Pm} (1)
the probability of magnetism is 0.7, and may be a constant coefficient. When P of a certain pedestrianr、PsOr PmWhen any one of the magnetic force probabilities exceeds (for the non-magnetic force probability P)mI.e. Pm1), the pedestrian is determined to be a magnetic pedestrian. The magnetic force model in the invention sets the pedestrian to move from the left side to the right side of the image as positive direction and from the right side to the left side as negative direction. For the value, taking the repulsive pedestrian as an example, when two pedestrians traveling in the same direction and having a collision in the traveling route are relatively close to each other, the two pedestrians can be determined as the repulsive pedestrian, the probability threshold value is about 0.7 converted by the method according to the general distance (more than 360 cm) in the social space distance of the human body as the determination basis, and similarly, when the two pedestrians traveling in the same direction and having a close distance, the probability threshold value can also be determined according to the general distance (more than 360 cm) in the social space distance of the human bodyThe conversion to probability threshold is also about 0.7. Therefore, when the pedestrians are relatively close to each other, the magnetic pedestrian probability determination threshold is used, and of course, the social space distance may be adaptively adjusted by using the current scheme as a reference in the implementation of the scheme, so that the general distance is adaptively modified, or the probability threshold may be set by itself.
(1) Definition of repulsion model: in the pedestrians which run in opposite directions and collide in a plurality of traveling routes, the moving speed of the pedestrians can be reduced or even stopped when the pedestrians meet each other until the original speed is recovered after meeting is finished, the characteristic that the moving directions of the pedestrians are opposite is similar to the same-polarity magnetic repulsion effect, and the situation is called as a repulsion model.
The number of pedestrians in the repulsion model is at least two, and pedestrians which run in opposite directions and collide with each other in the traveling route form the repulsion model. The determination of the repulsive force model takes the distance between pedestrians and the distance between a pedestrian and a vehicle, which are traveling in opposite directions, as the main basis: firstly, comparing the distances between two pedestrians walking in opposite directions and a vehicle to judge whether the traveling routes of the two pedestrians collide; if the traveling routes of the pedestrians walking in the opposite directions collide, the distance between the two pedestrians walking in the opposite directions is compared to judge whether the two pedestrians are about to collide. The formula (2) is a repulsive force probability calculation formula,
in the formula
Is the probability of repulsion of the pedestrian i. sijIs the distance between pedestrians i and j. dijIs the difference between the distances of the pedestrians i and j and the camera respectively and dij＝|zi-zjL, wherein ziIs the distance between the pedestrian i and the camera, zjIs the distance between the pedestrian j and the camera, m and n are constant coefficients, and m, n ∈ (0, 1). viAnd vjThe moving speeds, v, of pedestrians i and j, respectivelyivj< 0 indicates that the pedestrians i and j are traveling in opposite directions. Probability of repulsion existing only inIn the formula (2), m and n are constant coefficients, and m, n ∈ (0,1), in one embodiment of the invention, m is 0.4, and n is 0.4.
For pedestrian encounters on the road, pedestrians i and j move at a speed of 1.5m/s in a 640 x 480 image, where pedestrian i is (30, 90) and remains moving positively and pedestrian j is (600, 100) and remains moving negatively. When pedestrians i and j approach each other, the probability of repulsion PrAnd they are gradually increased, and their repulsive probability is exceeded when they approach a certain distance, thereby being determined as a magnetic pedestrian.
On the road, the pedestrian meets the condition very commonly, and the possibility that the pedestrian collides and then accidents occur may occur due to the difference of the conditions of the pedestrian, for example, the pedestrian who holds a mobile phone or hurried and hurried to the road exists in the meeting pedestrian, the former easily collides with other people due to the distraction, and the latter easily collides with other people due to the faster moving speed. The encountered ones of the pedestrians are thus distinguished by the repulsive force model.
(2) Defining a suction model: among a plurality of pedestrians walking in the same direction, two or more pedestrians walk in an overlapped or adjacent state, and the characteristics that the moving speed is similar in direction and magnitude are similar to the magnetic opposite attraction effect, and the condition is called as an attraction model.
The number of pedestrians in the suction model is at least two, and all the pedestrians in the same direction which keep overlapping or walking at an adjacent distance form the suction model. Whether the pedestrian is walking with the accompanying is the main basis for judging the suction model, and whether the pedestrian is walking with the accompanying is judged by comparing the distance between the pedestrians in the same direction and the distance between the pedestrian and the vehicle. The formula (3) is a suction probability calculation formula,
in the formula
Probability of attraction for pedestrian i, sijDistance between pedestrians i and j, dijIs the difference between the distances of the pedestrians i and j and the camera and dij＝|zi-zjL, wherein ziIs the distance between the pedestrian i and the camera, zjIs the distance between the pedestrian j and the camera. v. ofiAnd vjThe moving speeds, v, of pedestrians i and j, respectivelyivj> 0 indicates that pedestrians i and j are traveling in the same direction. k is a radical ofijIs a parameter of suction, and kij＝-[min(T|vi|,T|vj|)/sij]Where T is the elapsed time, min (T | v)i|,T|vjThe longer the time to maintain a close and fixed relative distance between pedestrians i and j traveling in the same direction, the greater the probability of attraction of the pedestrian, equation (3), where m, n, and l are constant coefficients, and m, n, l ∈ (0,1), in one embodiment of the invention m is 0.4, n is 0.4, and l is 0.2.
For pedestrian companions on the road, pedestrians i and j move in a 640 x 480 image at a speed of 1.5m/s, where pedestrian i is located at (30, 90) and pedestrian j is located at (45, 100), and they both maintain forward movement. When the pedestrians i and j are in the process of advancing and the relative distance between the pedestrians i and j does not change greatly, the probability P of suction force of the pedestrians i and jsWill gradually increase. When the two move forward for a certain distance, the probability of attraction of the two is exceeded, and the pedestrian is judged to be a magnetic pedestrian.
On the road, the condition that pedestrians walk together is common, and the walking together can shield the sight line, so that the attention of the pedestrians is weakened, and the danger of the pedestrians is improved. Meanwhile, the old and the children often accompany people to walk on the street, wherein the children are very easy to cause traffic accidents on the road, and the old is one of people who need to take care and give way. The accompanying one of the pedestrians is thus distinguished by the suction model.
(3) Definition of a non-magnetic force model: in a plurality of pedestrians, the moving speed of a certain pedestrian is obviously different from that of other pedestrians but does not necessarily satisfy a repulsion model or a suction model, and the situation that the certain pedestrian does not necessarily satisfy the repulsion or the suction is similar to the non-magnetic effect, and the situation is called a non-magnetic model.
At least three pedestrians exist in the non-magnetic force model, and when the speed of one pedestrian is higher or lower than that of at least two other pedestrians, the non-magnetic force model is formed. The determination of the non-magnetic force model is mainly based on the moving speed value of the pedestrian. The formula (4) is a nonmagnetic probability calculation formula,
in the formula
Is the nonmagnetic probability of the pedestrian i. v. ofi、vjAnd vhThe moving speeds of the pedestrians i, j, and h, respectively. Only when the speed of the pedestrian i is higher or lower than that of the other at least two pedestrians, and the speeds of the three pedestrians satisfy the formula
(vi 2-vj 2)+(vi 2-vh 2)＞2(5)
For a particular group of pedestrians on the road, their movement speed is generally slow, and in a 640 x 480 image, pedestrian i moves at 0.5m/s, and pedestrians j and h move at 1.5 m/s. Wherein pedestrian i is located at (30, 90), pedestrian j is located at (170, 240), pedestrian h is located at (310, 420), and they both remain moving in the forward direction. The non-magnetic probability P of the pedestrian i when the speeds of the three pedestrians remain unchangedmIt is over-set and thus determined as a magnetic pedestrian. For pedestrians moving faster on the road, in one 640 × 480 image, the pedestrian i moves at a speed of 2m/s, and the pedestrians j and h move at a speed of 1.5 m/s. WhereinThe pedestrian i is located at (310, 420), the pedestrian j is located at (170, 240), the pedestrian h is located at (30, 90), and they all keep moving forward. The non-magnetic probability P of the pedestrian i when the speeds of the three pedestrians remain unchangedmIt is exceeded and determined as a magnetic pedestrian.
On roads, special groups and pedestrians overtaking roads are also common, wherein the special groups are one of the pedestrians needing attention and care, the special groups tend to move at a slower speed, and the probability of causing traffic accidents due to the faster moving speed of the pedestrians is much higher than that of the pedestrians walking normally. The two pedestrians are therefore distinguished by a no-magnetic model.
The magnetic force probability of the same pedestrian may be different from that of different pedestrians through calculation, and the larger magnetic force probability is taken as the standard.
The method comprises the following steps of firstly, calculating a magnetic force model, then, enabling each pedestrian to have respective magnetic force probability, and classifying the pedestrians into four categories, namely ordinary pedestrians, repulsive pedestrians, attractive pedestrians and non-magnetic pedestrians according to the difference of the magnetic force probability, wherein the repulsive pedestrians, the attractive pedestrians and the non-magnetic pedestrians are three categories, and ① is set as PrPedestrian is repulsive when exceeding ② when PsWhen exceeding, the pedestrian is a suction pedestrian, ③ when PmThe pedestrian is a non-magnetic pedestrian when exceeding the time, and the P of ④ three magnetic probabilitiesmOf highest priority, PsIs centered on the priority of PrIs lowest. The probability of magnetism of the known pedestrian i is
When the probability of no magnetic force exceeds, the pedestrian i is a common pedestrian; when only one magnetic force probability is exceeded, the pedestrian i is judged as a corresponding type of magnetic force pedestrian according to the setting; when two or three kinds of magnetic force probabilities are exceeded, the pedestrian i is judged as a magnetic force pedestrian with a higher priority according to the priority of the magnetic force probabilities.
By the technical scheme, the road pedestrian classification method based on the magnetic force model has the beneficial effects that:
the existing road pedestrian classification method mainly analyzes and calculates the direct collision probability between pedestrians and vehicles, ignores the complexity and variability of the actual road, cannot well analyze the road condition, and is greatly influenced by the environment. The method classifies the pedestrians on the road by adopting a magnetic model, considers three common pedestrian conditions which are easy to cause danger, analyzes partial potential dangers of the pedestrians, and realizes attention to partial special groups on the road. The invention finds out the pedestrians which are easy to cause the accident by analyzing the common road condition environment, and the pedestrians are not necessarily collided with the vehicle, so the pedestrians cannot be classified by the existing method. Compared with the existing method, the pedestrian classification result can further enrich the pedestrian information obtained by the existing road pedestrian classification method, and provide more sufficient road condition information for the autonomous automobile and the assistant driving system. The method comprises the steps of firstly using the existing road pedestrian classification method to find out pedestrians which can directly collide with vehicles, and then using the method to find out other pedestrians which are easy to cause danger, so that a safer path scheme can be obtained when an autonomous automobile or an auxiliary driving system carries out obstacle avoidance and path planning. In the routing of the autonomous vehicle, different classes of magnetic pedestrians have different basic routing. The repulsion pedestrians and the attraction pedestrians can influence the path selection of the vehicles only when being positioned on the driving path of the vehicles, and for the repulsion pedestrians, the vehicles can preferentially select to pass through the pedestrians after the repulsion pedestrians are mutually staggered; for a suction pedestrian, the vehicle will preferentially choose to pass in the opposite direction of the direction of movement of the suction pedestrian. When a pedestrian without magnetism appears in the front of the vehicle, the vehicle can preferentially select to stop driving no matter whether the pedestrian is positioned on the driving route of the vehicle or not, and the pedestrian without magnetism is waited to leave the front visual angle and then continue driving. Because the pedestrian without magnetic force has two states of high moving speed or low moving speed, the pedestrian may be a special crowd and needs to be given a gift for the pedestrian with low moving speed; even if a pedestrian moving at a high speed is not on the driving route of the vehicle at the present time, the pedestrian may move to the vicinity of the front of the vehicle and collide with the vehicle at a later time, and therefore the vehicle may choose to stop when a pedestrian without magnetic force is present. The problem of selecting the path of the vehicle when the single magnetic force pedestrian exists is described above, when a plurality of magnetic force pedestrians occur and the basic paths of the magnetic force pedestrians conflict with each other, the vehicle can choose to stop traveling, otherwise, the vehicle can select to travel according to the basic paths. For example, non-magnetic pedestrians and repulsive pedestrians exist in the front of the vehicle, and the vehicle can choose to stop; repulsion pedestrian and suction pedestrian exist in the plantago, and the suction pedestrian walks to the left side from the right side of plantago visual angle, and the suction pedestrian is on repulsion pedestrian's right side, and the route selection conflict between them, and the vehicle can select to park this moment. The influence of the magnetic force pedestrian on the vehicle routing is described above, when a dangerous pedestrian and a magnetic force pedestrian which collide with the vehicle appear simultaneously, the vehicle considers the route capable of avoiding the dangerous pedestrian first, then judges whether the route accords with the routing of the magnetic force pedestrian, and stops traveling if the route does not accord with the routing of the magnetic force pedestrian. Take the example of a dangerous pedestrian in the center of the view in front of the vehicle, and the pedestrian walks to the right of the view in front of the vehicle. In order to avoid the pedestrian, the vehicle can choose to pass through the left side of the pedestrian, if the repulsive force pedestrian exists on the right side of the front of the vehicle, the route selection of the vehicle is not influenced, because the repulsive force pedestrian is not on the driving route, if the repulsive force pedestrian is on the left side of the front of the vehicle, the vehicle can choose to wait for the passage of the repulsive force pedestrian after being staggered; if a suction pedestrian exists on the front right side of the vehicle, no matter how the movement direction of the suction pedestrian exists, the path selection of the vehicle is not influenced, if the suction pedestrian walks on the front left side of the vehicle and to the right side of the front visual angle, the vehicle can choose to run from the left side of the suction pedestrian, if the suction pedestrian walks on the front left side of the vehicle and to the left side of the front visual angle, the vehicle can choose to pass between the suction pedestrian and a dangerous pedestrian, and meanwhile, the suction pedestrian and the dangerous pedestrian can meet the judgment requirement of a repulsion pedestrian at the previous moment, so the running route also meets the basic path selection of the repulsion of the pedestrian; if no-magnetic force pedestrians appear in the front of the vehicle, the vehicle can stop moving forward no matter whether the no-magnetic force pedestrians are located on the left side or the right side in front of the vehicle.
A road pedestrian classification system storing a plurality of instructions adapted to be loaded and executed by a processor: the vehicle-mounted camera shoots images of the pedestrians on the roads, and the pedestrians on the roads are classified according to the magnetic force relation existing among the pedestrians on the roads displayed by the images; wherein: the magnetic force relationship is characterized by the moving speed of the pedestrians, the relative distance between the pedestrians in the image and the relative distance between the pedestrians and the camera.
The magnetic force relation is represented by a magnetic force model, the magnetic force model comprises a repulsion force model, an attraction force model and a non-magnetic force model, the probability of each specific model in the magnetic force relation displayed by the current image is calculated, and if the probability of the specific model exceeds a probability threshold value, the magnetic force relation of the road pedestrian in the current image is reflected by the current model.
The repulsion model is defined as follows: the pedestrians who run in opposite directions have collision routes in a plurality of traveling routes, and the moving speed of the pedestrians is reduced or even stopped when the pedestrians meet each other until the speed is recovered after meeting is finished.
The magnetic force relationship of the pedestrian displayed by the current image is the probability of the repulsive force model calculated by the following formula:
when the pedestrians i and j move in opposite directions and approach each other continuously, the probability P of repulsion isrWill gradually increase until the probability of repulsion exceeds and is judged as a magnetic pedestrian; wherein:
probability of repulsion for pedestrian i; sijIs the distance between pedestrians i and j; dijIs the difference between the distances of the pedestrians i and j, respectively, from the camera, and dij＝|zi-zjL, wherein ziIs the distance between the pedestrian i and the camera, zjIs the distance between the pedestrian j and the camera, m and n are constant coefficients, and m, n ∈ (0, 1); viAnd vjThe moving speeds, v, of pedestrians i and j, respectivelyivj< 0 indicates that the pedestrians i and j are traveling in opposite directions.
The suction model is defined as follows: among a plurality of pedestrians walking in the same direction, two or more pedestrians walk in an overlapped or adjacent state, and the moving speeds of the pedestrians are similar in direction and magnitude.
The magnetic force relationship of the pedestrian displayed by the current image is that the probability of the suction model is calculated by the following formula:
when pedestrians i and j continuously walk in the same direction and overlap or in the adjacent state, the probability of suction force
Gradually increasing until the suction probability exceeds so as to be judged as a suction pedestrian;
wherein:
the probability of suction being a pedestrian i; sijIs the distance between pedestrians i and j; dijIs the difference between the distances of the pedestrians i and j and the camera, and dij＝|zi-zjL, wherein ziIs the distance between the pedestrian i and the camera, zjIs the distance between the pedestrian j and the camera; v. ofiAnd vjThe moving speeds, v, of pedestrians i and j, respectivelyivj> 0 indicates that pedestrians i and j are traveling in the same direction; k is a radical ofijIs a parameter of suction, and kij＝-[min(T|vi|,T|vj|)/sij]Where T is the elapsed time, min (T | v)i|,T|vj|) represents a smaller displacement value of the pedestrians i and j after the time T.
The definition of the nonmagnetic model is as follows: among the several pedestrians, there are at least three pedestrians, one of which has a higher or lower speed than the other at least two pedestrians.
The probability that the magnetic force relationship of a pedestrian displayed by the current image is a no-magnetic force model is calculated by the following formula:
the pedestrians i, j and h walk in the same direction, the speed of one pedestrian is obviously higher than or lower than the speeds of the other two pedestrians, the speeds of the other two pedestrians are equal, the current speeds of the three pedestrians are kept unchanged, and the magnetic probability P is not generatedmExceeds to be determined as a non-magnetic pedestrian; wherein:
the magnetic force free probability of the pedestrian i; v. ofi、vjAnd vhThe moving speeds of the pedestrians i, j and h respectively, the speed of the pedestrian i is obviously higher than or lower than the speeds of the other two pedestrians, and the speeds of the three pedestrians meet the following conditions:
(vi 2-vj 2)+(vi 2-vh 2)＞2
then
Equal to 1, otherwise 0;
is a constant coefficient.
The specific method for classifying the pedestrians is as follows: calculating the magnetic force probability of a certain pedestrian in the current image:
PMM＝{Pr,Ps,Pm}
when no model probability exceeds a probability threshold, the pedestrian i is a common pedestrian; when only one model probability exceeds a probability threshold, the pedestrian i is judged as a corresponding type of magnetic pedestrian; when more than two model probabilities exceed the probability threshold, the pedestrian i is judged as a magnetic pedestrian with higher priority according to the priority of the model probabilities;
setting: when P is presentrPedestrians are repulsive when exceeding; when P is presentsThe pedestrians are attracted by the people when the pedestrian exceeds the time; when P is presentmThe pedestrians are non-magnetic pedestrians when the vehicle exceeds the set time; three kinds of magnetic force probability PmOf highest priority, PsIs centered on the priority of PrIs lowest.
Example 1:
repulsive force pedestrian classification situation
The simulation results for the classification of repulsive pedestrians in this example are shown in fig. 2. Fig. 2 lists three frames of images in consecutive video frames and the pedestrian classification results for the frames, wherein the magnetic force probability of a pedestrian satisfies the determination requirement of a magnetic force pedestrian only by the repulsive force probability. Three pedestrian targets move at the speed of about 1.2m/s in the video, wherein two pedestrians move in the positive direction, one pedestrian moves in the negative direction, and the three pedestrian targets move in a straight line without changing the moving speed. From frame 8 to frame 33, pedestrians B and C approach each other. The repulsive probabilities of the pedestrians B and C are exceeded until the 33 th frame, and it is determined as a repulsive pedestrian. Similarly, at the 72 th frame, the pedestrian a and the pedestrian C have the exceeding repulsive probability, and are determined as repulsive pedestrians, and at this time, the pedestrian B and the pedestrian C have ended the meeting process, and the repulsive probability is decreased, and the pedestrian B is determined as a normal pedestrian.
Example 2:
suction force pedestrian classification situation
The simulation results for the classification of repulsive pedestrians in this example are shown in fig. 3. Fig. 3 lists three images in consecutive video frames and the pedestrian classification results for the frames, wherein the magnetic force probability and the attraction probability of the pedestrian meet the determination requirement of the magnetic force pedestrian. Three pedestrian objects move in the forward direction at a speed of about 1.2m/s in the video, and all keep moving straight without changing the moving speed. From frame 11 to frame 39, pedestrians B and C keep walking concomitantly. The suction probabilities of the pedestrians B and C are exceeded until the 39 th frame, and it is determined that the pedestrian is a suction pedestrian. The pedestrians B and C maintain the determination result of the sucking pedestrian at the 75 th frame thereafter.
Example 3:
non-magnetic pedestrian classification condition
The simulation result of the present example for the classification of the non-magnetic pedestrian is shown in fig. 4. Fig. 4 lists three frames of images in consecutive video frames and the pedestrian classification results for the frames, wherein the magnetic force probability of the pedestrian satisfies the determination requirement of the magnetic force pedestrian only without the magnetic force probability. In the video, three pedestrian targets move along the negative direction at different speeds, and all move linearly without changing the moving speed, wherein the speed of a pedestrian A is about 0.5m/s, and the speeds of pedestrians B and C are about 1.3 m/s. At the time of the 9 th frame, the calculated non-magnetic probability of the pedestrian a is 1, the pedestrian is determined to be a non-magnetic pedestrian, and the pedestrian a maintains the determination result of the non-magnetic pedestrian at the following 42 th frame and 103 th frame.
Example 4:
hybrid magnetic pedestrian classification scenario
The simulation results for the classification of the mixed magnetic force pedestrian are shown in fig. 5. Fig. 5 lists three frames of images in consecutive video frames and the pedestrian classification results of the frames, wherein the magnetic force probability of the pedestrian with repulsive force probability and the magnetic force probability without magnetic force satisfy the determination requirements of the magnetic force pedestrian. In the video, the pedestrian a moves in the positive direction at a speed of 1.19m/s, the pedestrian B moves in the positive direction at a speed of 1.83m/s, the pedestrian C moves in the negative direction at a speed of 1.21m/s, and the three pedestrians keep moving straight and do not change the moving speed. At frame 6, the probability of no magnetic force of the pedestrian B is calculated to be 1, and the pedestrian is determined to be a no magnetic force pedestrian. By frame 54, the probability of repulsion for pedestrians B and C is exceeded, wherein the probability of no magnetic force for pedestrian B is also exceeded, thus pedestrian B is determined to be a no magnetic pedestrian and pedestrian C is determined to be a repulsive pedestrian. By frame 109, pedestrian B and pedestrian C have already finished the meeting process, the repulsive probability of pedestrian B is reduced and lower, but his no-magnetic force probability is not changed and therefore still determined as a no-magnetic force pedestrian. And the pedestrian C and the pedestrian A are close to each other, so that the probability of repulsion of the pedestrians C and A is increased and exceeded, and the pedestrians C and A are judged as repulsive pedestrians.
Claims (8)
1. A road pedestrian classification system having stored thereon a plurality of instructions adapted to be loaded and executed by a processor to: the vehicle-mounted camera shoots images of the pedestrians on the roads, and the pedestrians on the roads are classified according to the magnetic force relation existing among the pedestrians on the roads displayed by the images;
the magnetic force relationship is characterized by the moving speed of the pedestrians, the relative distance between the pedestrians in the image and the relative distance between the pedestrians and the camera;
the magnetic force relation is represented by a magnetic force model, the magnetic force model comprises a repulsive force model, the probability of each specific model in the magnetic force relation displayed by the current image is calculated, and if the probability of the specific model exceeds a probability threshold value, the magnetic force relation of the road pedestrians in the current image is reflected by the current model;
the repulsion model is defined as follows: pedestrians walking in opposite directions have collision routes in a plurality of advancing routes, and the moving speed of the pedestrians is reduced or even stopped when meeting is carried out until the speed is recovered after the meeting is finished; the magnetic force relationship of the pedestrian displayed by the current image is the probability of the repulsive force model calculated by the following formula:
when the pedestrians i and j move in opposite directions and approach each other continuously, the probability P of repulsion isrWill gradually increase until the probability of repulsion exceeds the probability threshold and thus be determined as a magnetic pedestrian;
wherein:
Priprobability of repulsion for pedestrian i;
sijis the distance between pedestrians i and j;
dijis the difference between the distances of the pedestrians i and j, respectively, from the camera, and dij＝|zi-zjL, wherein ziIs the distance between the pedestrian i and the camera, zjIs the distance between the pedestrian j and the camera;
m and n are constant coefficients, and m, n belongs to (0, 1);
viand vjThe moving speeds, v, of pedestrians i and j, respectivelyivj< 0 indicates that the pedestrians i and j are traveling in opposite directions.
2. The road pedestrian classification system of claim 1, wherein the magnetic force model further comprises an attraction model.
3. The road pedestrian classification system of claim 2, wherein the suction model is defined by: among a plurality of pedestrians walking in the same direction, more than two pedestrians walk in an overlapped or adjacent state, and the moving speeds of the pedestrians are similar in direction and magnitude.
4. The road pedestrian classification system of claim 3, wherein the magnetic force relationship of the pedestrian displayed by the current image is a probability of the attraction model calculated by:
when pedestrians i and j continuously walk in the same-direction overlapped or adjacent state, the suction probability PsGradually increasing until the suction probability exceeds a probability threshold so as to be determined as a suction pedestrian;
wherein:
Psithe probability of suction being a pedestrian i;
sijis the distance between pedestrians i and j;
dijis the difference between the distances of the pedestrians i and j and the camera, and dij＝|zi-zjL, wherein ziIs the distance between the pedestrian i and the camera, zjIs the distance between the pedestrian j and the camera;
m, n and l are constant coefficients, and m, n, l are belonged to (0, 1);
viand vjThe moving speeds, v, of pedestrians i and j, respectivelyivj> 0 indicates that pedestrians i and j are traveling in the same direction;
kijis a parameter of suction, and kij＝-[min(T|vi|,T|vj|)/sij]Where T is the elapsed time, min (T | v)i|,T|vj|) represents a smaller displacement value of the pedestrians i and j after the time T.
5. The road pedestrian classification system of claim 1, wherein the magnetic models further include a no-magnetic model.
6. The road pedestrian classification system of claim 5, wherein the nonmagnetic model is defined as: among the several pedestrians, there are at least three pedestrians, one of which has a higher or lower speed than the other at least two pedestrians.
7. The road pedestrian classification system of claim 6, wherein the probability that the magnetic relationship of a pedestrian displayed by the current image is a no-magnetic model is calculated by:
the pedestrians i, j and h walk in the same direction, the speed of one pedestrian is obviously higher than or lower than the speeds of the other two pedestrians, the speeds of the other two pedestrians are equal, the current speeds of the three pedestrians are kept unchanged, and the magnetic probability P is not generatedmExceeds a probability threshold to be determined as a non-magnetic pedestrian;
wherein:
Pmithe magnetic force free probability of the pedestrian i;
vi、vjand vhThe moving speeds of the pedestrians i, j and h respectively, the speed of the pedestrian i is obviously higher than or lower than the speeds of the other two pedestrians, and the speeds of the three pedestrians meet the following conditions:
(vi 2-vj 2)+(vi 2-vh 2)＞2
then P ismiEqual to 1, otherwise 0;
is a constant coefficient.
8. The road pedestrian classification system of claim 1, wherein the specific method of classifying pedestrians is:
calculating the magnetic force probability of a certain pedestrian in the current image:
PMM＝{Pr,Ps,Pm}
Pris the probability of repulsion, PsIs the probability of attraction, PmIs the magnetic force free probability;
when no model probability exceeds a probability threshold, the pedestrian i is a common pedestrian; when only one model probability exceeds a probability threshold, the pedestrian i is judged as a corresponding type of magnetic pedestrian; when more than two model probabilities exceed the probability threshold, the pedestrian i is judged as a magnetic pedestrian with higher priority according to the priority of the model probabilities;
setting:
when P is presentrPedestrians are repulsive when exceeding;
when P is presentsThe pedestrians are attracted by the people when the pedestrian exceeds the time;
when P is presentmThe pedestrians are non-magnetic pedestrians when the vehicle exceeds the set time;
three kinds of magnetic force probability PmOf highest priority, PsIs centered on the priority of PrIs lowest.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810936562.5A CN108961838B (en) | 2018-08-16 | 2018-08-16 | Road pedestrian classification system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810936562.5A CN108961838B (en) | 2018-08-16 | 2018-08-16 | Road pedestrian classification system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108961838A CN108961838A (en) | 2018-12-07 |
| CN108961838B true CN108961838B (en) | 2020-09-22 |
Family
ID=64469202
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810936562.5A Active CN108961838B (en) | 2018-08-16 | 2018-08-16 | Road pedestrian classification system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108961838B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108985271B (en) * | 2018-08-16 | 2021-10-08 | 大连民族大学 | Method for judging pedestrian of interest of magnetic model |
Family Cites Families (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4568637B2 (en) * | 2005-04-26 | 2010-10-27 | 富士重工業株式会社 | Road recognition device |
| US9126525B2 (en) * | 2009-02-27 | 2015-09-08 | Magna Electronics Inc. | Alert system for vehicle |
| CN102096803B (en) * | 2010-11-29 | 2013-11-13 | 吉林大学 | Safe state recognition system for people on basis of machine vision |
| CN102765365B (en) * | 2011-05-06 | 2014-07-30 | 香港生产力促进局 | Pedestrian detection method based on machine vision and pedestrian anti-collision warning system based on machine vision |
| CN102496001B (en) * | 2011-11-15 | 2015-02-25 | 无锡港湾网络科技有限公司 | Method of video monitor object automatic detection and system thereof |
| US9786178B1 (en) * | 2013-08-02 | 2017-10-10 | Honda Motor Co., Ltd. | Vehicle pedestrian safety system and methods of use and manufacture thereof |
| CN103473953B (en) * | 2013-08-28 | 2015-12-09 | 奇瑞汽车股份有限公司 | A kind of pedestrian detection method and system |
| DE102014201159A1 (en) * | 2014-01-23 | 2015-07-23 | Robert Bosch Gmbh | Method and device for classifying a behavior of a pedestrian when crossing a roadway of a vehicle and personal protection system of a vehicle |
| CN104834887B (en) * | 2014-02-11 | 2018-09-11 | 株式会社理光 | Move pedestrian's representation method, recognition methods and its device |
| KR101611194B1 (en) * | 2014-04-04 | 2016-04-11 | 주식회사 와이즈오토모티브 | Apparatus and method for peripheral image generation of vehicle |
| US10444346B2 (en) * | 2014-07-25 | 2019-10-15 | Robert Bosch Gmbh | Method for migrating radar sensor limitations with video camera input for active braking for pedestrians |
| CN104134360A (en) * | 2014-08-14 | 2014-11-05 | 奇瑞汽车股份有限公司 | Intersection pedestrian recognition safety control system and method based on short-range communication |
| CN106909929A (en) * | 2015-12-22 | 2017-06-30 | 比亚迪股份有限公司 | Pedestrian's distance detection method and device |
| CN106428000B (en) * | 2016-09-07 | 2018-12-21 | 清华大学 | A kind of vehicle speed control device and method |
| CN108122431A (en) * | 2016-11-30 | 2018-06-05 | 中国移动通信有限公司研究院 | A kind of traffic accident method for early warning and device |
| CN106709478A (en) * | 2017-02-22 | 2017-05-24 | 桂林电子科技大学 | Pedestrian image feature classification method and system |
| CN107169439A (en) * | 2017-05-11 | 2017-09-15 | 南宁市正祥科技有限公司 | A kind of Pedestrians and vehicles detection and sorting technique |
-
2018
- 2018-08-16 CN CN201810936562.5A patent/CN108961838B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN108961838A (en) | 2018-12-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Yu et al. | Occlusion-aware risk assessment for autonomous driving in urban environments | |
| LU101647B1 (en) | Road pedestrian classification method and top-view pedestrian risk quantitative method in two-dimensional world coordinate system | |
| US20200211395A1 (en) | Method and Device for Operating a Driver Assistance System, and Driver Assistance System and Motor Vehicle | |
| US10796574B2 (en) | Driving assistance method and device | |
| JP6679152B1 (en) | Accident analysis device, accident analysis method and program | |
| CN113306552B (en) | Ultra-low speed creeping method of unmanned vehicle under mixed road congestion state | |
| CN108803626B (en) | System for planning a route for an autonomous vehicle or a driver assistance system | |
| Guo et al. | Humanlike behavior generation in urban environment based on learning-based potentials with a low-cost lane graph | |
| CN108961838B (en) | Road pedestrian classification system | |
| CN109165591B (en) | Road pedestrian classification method | |
| CN109145980B (en) | Magnetic force probability algorithm and system for road pedestrian classification | |
| CN109147388B (en) | Method and system for judging road pedestrian attraction relationship | |
| CN109147389B (en) | Method for planning route by autonomous automobile or auxiliary driving system | |
| CN109145807B (en) | Method and system for judging no-magnetic relation of road pedestrians | |
| CN108909708B (en) | Method and system for judging road pedestrian repulsive force relationship | |
| CN108985271B (en) | Method for judging pedestrian of interest of magnetic model | |
| Karaduman et al. | Interactive risky behavior model for 3-car overtaking scenario using joint Bayesian network | |
| Yue et al. | An augmentation function for active pedestrian safety system based on crash risk evaluation | |
| Karaduman et al. | Approaching car detection via clustering of vertical-horizontal line scanning optical edge flow | |
| Hegazy et al. | Classifying aggressive drivers for better traffic signal control | |
| CN114475588A (en) | Automobile unmanned recognition system and automobile unmanned recognition method | |
| Akikawa et al. | Smartphone-based risky traffic situation detection and classification | |
| Jyothi et al. | Driver assistance for safe navigation under unstructured traffic environment | |
| Siriwardana et al. | Driving Through a Bend: Detection of Unsafe Driving Patterns and Prevention of Heavy Good Vehicle Rollovers | |
| Malathy et al. | Pedestrian safety system with crash prediction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |