CN107256030A - Remote terminal, flight assisting system and the method for unmanned vehicle - Google Patents
Remote terminal, flight assisting system and the method for unmanned vehicle Download PDFInfo
- Publication number
- CN107256030A CN107256030A CN201710457599.5A CN201710457599A CN107256030A CN 107256030 A CN107256030 A CN 107256030A CN 201710457599 A CN201710457599 A CN 201710457599A CN 107256030 A CN107256030 A CN 107256030A
- Authority
- CN
- China
- Prior art keywords
- unmanned vehicle
- flight
- information
- positional information
- relative
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
- G05D1/10—Simultaneous control of position or course in three dimensions
- G05D1/101—Simultaneous control of position or course in three dimensions specially adapted for aircraft
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
- G05D1/0094—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
-
- G—PHYSICS
- G08—SIGNALLING
- G08C—TRANSMISSION SYSTEMS FOR MEASURED VALUES, CONTROL OR SIMILAR SIGNALS
- G08C17/00—Arrangements for transmitting signals characterised by the use of a wireless electrical link
Abstract
The invention discloses a kind of remote terminal of unmanned vehicle, flight assisting system and method, belong to unmanned vehicle field, the remote terminal includes data obtaining module, message processing module and message output module, data obtaining module is used for the positional information for obtaining observation position, and the positional information and state of flight information of unmanned vehicle are obtained by wireless network；Message processing module is used to obtain positional information of the unmanned vehicle relative to observation position according to the positional information of unmanned vehicle and the positional information of observation position；Message output module is used for the positional information of state of flight information and unmanned vehicle relative to observation position for exporting the unmanned vehicle.Using the present invention, by grasping current flight position and the state of flight of unmanned vehicle, manipulator is helped to control the flight path of unmanned vehicle, it is to avoid blind to fly and fly to lose, while improving the flight experience of manipulator.
Description
The application is the Application No. 201310282462.2 submitted to China Intellectual Property Office, and the applying date is 2013 07
The moon 05, invention and created name is the application for a patent for invention of " remote terminal, flight assisting system and the method for unmanned vehicle "
Divisional application.
Technical field
The present invention relates to unmanned vehicle, more particularly to a kind of remote terminal of unmanned vehicle, flight assisting system and
Method.
Background technology
In recent years, unpiloted aircraft (such as fixed wing aircraft, rotor craft includes helicopter), motor vehicle
, submarine or ship, and satellite, space station, or airship etc. are widely used, such as in detecting, the field such as search and rescue.
Manipulation for these movable bodies is usual by user is realized by remote control.
Remotely-piloted vehicle, submarine, or motor vehicles with carrying carrier, for example, can carry the carrier dress of camera or illuminating lamp
Put.For example, camera can be loaded with a kind of telecontrolled aircraft to be taken photo by plane.
Manipulator is during operational movement body, such as unmanned vehicle, because unmanned vehicle average physique is smaller,
(such as four or five hundred meters) are difficult with the naked eye to see in the case of flying far, in this case, and manipulator is difficult to observe nobody to fly
The course heading of row device, be equivalent to it is blind fly, if the supplementary means do not flown, unmanned vehicle is just easy to fly to lose.Separately
Outside, if if flying, being too absorbed in display screen using FPV (First Person View, first person) patterns,
The position for not making clear that unmanned vehicle is current may also finally be caused, cause to get lost or even fly to lose, and while see
FPV, while noting the position of unmanned vehicle, then can allow Consumer's Experience to have a greatly reduced quality.
The content of the invention
In view of this, the technical problem to be solved in the present invention is to provide a kind of remote terminal of unmanned vehicle, flight are auxiliary
Auxiliary system and method, to help manipulator to grasp the current flight position of unmanned vehicle and state, and can control nobody accordingly
The flight of aircraft.
The technical scheme that present invention solution above-mentioned technical problem is used is as follows：
According to an aspect of the present invention there is provided a kind of unmanned vehicle remote terminal include data obtaining module,
Message processing module and message output module, wherein：
Data obtaining module, the positional information for obtaining observation position obtains the positional information of unmanned vehicle and flown
Row status information；
Message processing module, the positional information for the positional information according to unmanned vehicle and observation position obtains nobody
Positional information of the aircraft relative to observation position；
Message output module, for receiving and showing that the state of flight information of unmanned vehicle flies with nobody by display screen
Positional information of the row device relative to observation position；
Message processing module, is additionally operable to judge azimuth and the unmanned vehicle of screen datum line of display screen relative to sight
Whether the azimuthal difference for examining position is less than predetermined threshold value, and display screen elevation angle and unmanned vehicle relative to observation
The difference of the elevation angle of position is less than predetermined threshold value, and target prompting information is captured if it is, being produced on screen.
Preferably, the position that observation position is taken off for the position of manipulator or unmanned vehicle.
Preferably, the remote terminal also includes position measuring module, for measuring the positional information of remote terminal as behaviour
The positional information of control person, and send message processing module to.
Preferably, message output module specifically for：By the state of flight information of unmanned vehicle and relative to observation position
The positional information put is shown on a display screen by the form of figure.
Preferably, unmanned vehicle includes between unmanned vehicle and observation position relative to the positional information of observation position
Distance and unmanned vehicle relative to observation position angle；The course of state of flight information including unmanned vehicle and/or
The elevation information relative to ground of unmanned vehicle.
Preferably, data obtaining module also includes terminal attitude measurement module, wherein：
Terminal attitude measurement module, the azimuth of the screen datum line for detecting display screen；
Message processing module, azimuth is reversely rotated for the positive optical axis by figure with respect to display screen.
Preferably, predetermined threshold value is 10 degree.
According to another aspect of the present invention there is provided the flight assisting system of unmanned vehicle a kind of include：Above-mentioned skill
Remote terminal in art scheme and the state measurement sensor being arranged on unmanned vehicle, wherein：
State measurement sensor, for measuring the positional information and state of flight information of unmanned vehicle, and nobody is flown
The positional information and state of flight information transmission of row device are to remote terminal；
Remote terminal, is additionally operable to state of flight information and unmanned vehicle according to unmanned vehicle relative to observation position
Positional information, control unmanned vehicle flight.
Preferably, when observing the position that position is manipulator, the positional information of manipulator by be attached to manipulator or
Sensor on the object entrained by manipulator is obtained；When it is the position that unmanned vehicle takes off to observe position, nobody flies
The position that row device takes off is obtained by state measurement sensor.
According to a further aspect of the invention there is provided a kind of unmanned vehicle flight householder method, it is characterised in that
This method includes：
Obtain the state of flight information of observation positional information, the positional information of unmanned vehicle and unmanned vehicle；
Unmanned vehicle is obtained relative to observation according to the positional information of the positional information of unmanned vehicle and observation position
The positional information of position；And by the state of flight information of unmanned vehicle and relative to the positional information output of observation position；
Position letter according to the state of flight information and unmanned vehicle of the unmanned vehicle of output relative to observation position
Breath, controls the flight of unmanned vehicle；
This method also includes：Judge azimuth and the unmanned vehicle of screen datum line of display screen relative to observation position
Azimuthal difference whether be less than predetermined threshold value, and display screen elevation angle and unmanned vehicle relative to observation position
The difference of elevation angle is less than predetermined threshold value, and target prompting information is captured if it is, being produced on screen.
It is preferred that, the positional information and state of flight information of unmanned vehicle pass through the state that is arranged on unmanned vehicle
Measurement sensor is obtained；The position that observation position is taken off for the position of manipulator or unmanned vehicle, when observation position is manipulation
During the position of person, the positional information of manipulator is obtained by being attached to manipulator or the sensor on the object entrained by manipulator
；When it is the position that unmanned vehicle takes off to observe position, the position that unmanned vehicle takes off is by setting on board the aircraft
State measurement sensor obtain.
Preferably, the positional information by the state of flight information and unmanned vehicle of unmanned vehicle relative to observation position
Output includes：Included by the state of flight of unmanned vehicle and relative to the positional information of observation position by the form of figure
On display screen.
Preferably, unmanned vehicle includes between unmanned vehicle and observation position relative to the positional information of observation position
Distance and unmanned vehicle relative to observation position angle；The course of state of flight information including unmanned vehicle and/or
The elevation information relative to ground of unmanned vehicle.
Preferably, this method also includes：The azimuth of the screen datum line of display screen is obtained, and by figure with respect to display screen
The positive optical axis reversely rotate azimuth.
Preferably, predetermined threshold value is 10 degree.
Remote terminal, flight assisting system and the method for the unmanned vehicle of the embodiment of the present invention, by grasping nothing in real time
The current flight position of people's aircraft and state of flight, help manipulator to control the flight path of unmanned vehicle, improve behaviour
The flight experience of control person, while avoiding blind winged and flying to lose.
Brief description of the drawings
Fig. 1 is a kind of structural representation of the remote terminal of unmanned vehicle provided in an embodiment of the present invention；
Fig. 2 present invention is preferable to carry out a kind of unmanned vehicle accessory system structural representation provided；
A kind of display schematic diagram for display screen east-west direction horizontal positioned that Fig. 3 provides for the preferred embodiment of the present invention；
A kind of display schematic diagram for display screen North and South direction horizontal positioned that Fig. 4 provides for the preferred embodiment of the present invention；
Fig. 5 is azimuth schematic diagram provided in an embodiment of the present invention；
A kind of azimuth of the screen datum line for display screen that Fig. 6 provides for the preferred embodiment of the present invention and unmanned vehicle
Schematic diagram when azimuth relative to observation position is consistent；
A kind of schematic diagram of the positive optical axis for display screen that Fig. 7 preferred embodiment of the present invention is provided just to unmanned vehicle；
Fig. 8 is the schematic diagram of the elevation angle of unmanned vehicle provided in an embodiment of the present invention；
Fig. 9 is the schematic diagram of the elevation angle of display screen provided in an embodiment of the present invention；
A kind of flow chart of the flight householder method for unmanned vehicle that Figure 10 provides for the preferred embodiment of the present invention.
Embodiment
In order that technical problems, technical solutions and advantages to be solved are clearer, clear, tie below
Drawings and examples are closed, the present invention will be described in further detail.It should be appreciated that specific embodiment described herein is only
To explain the present invention, it is not intended to limit the present invention.
It is a kind of remote terminal structural representation of unmanned vehicle provided in an embodiment of the present invention as shown in Figure 1, this is distant
Control terminal includes data obtaining module 10, message processing module 20 and message output module 30, wherein：
Data obtaining module 10, the positional information for obtaining observation position, unmanned vehicle is obtained by wireless network
Positional information and state of flight information；
Specifically, data obtaining module 10 further comprises observing position information acquisition module 101, and flight position shape
State acquisition module 102, wherein, the positional information or unmanned vehicle that position information acquisition module 101 is used to obtain manipulator rise
Winged positional information, when observing the position that position is manipulator, the positional information of manipulator by be attached to manipulator or
Sensor on object entrained by manipulator is obtained；Because this remote terminal is hand-held device, thus manipulator position with it is distant
The positional information of control terminal is same information, can be realized by the positional information of this remote terminal, therefore remote terminal is also
Including position measuring module (such as GPS module), for measuring the positional information of remote terminal as the positional information of manipulator,
And send message processing module 20 to.When it is the position that unmanned vehicle takes off to observe position, the position that unmanned vehicle takes off
Put and obtained by being arranged on carry-on state measurement sensor.The positional information of unmanned vehicle takeoff point is unmanned flight
Device searches the positional information recorded during enough position location satellites after the power-up, for the first time.
Flight position state acquisition module 102 obtains positional information and the flight of unmanned vehicle by wireless network in real time
Status information, the positional information and state of flight information of unmanned vehicle are passed by the state measurement being arranged on unmanned vehicle
Sensor measurement is obtained.Flight position information includes the longitude and latitude of flight position, and state of flight includes pitching, roll and boat
To the elevation information relative to ground of unmanned vehicle can also be included.Wireless network includes Wi-Fi (Wireless Fidelity) net
Network, certainly, still can be communicated with the 5G networks of 2G networks, 3G, 4G and future.
Message processing module 20, the positional information for the positional information according to unmanned vehicle and observation position obtains nothing
Positional information of people's aircraft relative to observation position；
Wherein, unmanned vehicle includes relative to the positional information of observation position：Between unmanned vehicle and observation position
Distance and unmanned vehicle relative to observation position angle.
Message output module 30, for exporting the state of flight information and unmanned vehicle of unmanned vehicle relative to observation
The positional information of position.
In order to more directly perceived, this step preferably by between the course of unmanned vehicle, unmanned vehicle and observation position away from
Shown in a graphical form on a display screen from the angle with unmanned vehicle relative to observation position.Wherein, unmanned vehicle
Angle relative to observation position is included relative to observation location fix angle and the elevation angle relative to observation position, and figure can be with
It is planar graph or solid figure.Refer to the arrow in Fig. 3 and Fig. 4, the figure of display and represent unmanned vehicle institute
In position, the sensing of arrow represent be unmanned vehicle course, the center of circle represents to observe position position, arrow and the center of circle
Line represents the position of unmanned vehicle with observing projection of the line of position on ground level, line and the folder in the positive north in ground
Angle a represents azimuth of the unmanned vehicle relative to observation position, to observe several circles of the position as the center of circle, from the inside to surface table
Show unmanned vehicle from a distance from observation position from closely to remote (semidiameter such as adjacent circle is 50 meters), wherein empty
The safe distance of flight is represented in the range of line circle, when unmanned vehicle is when outside safe distance, except figure shows exceed
Scope, can also remind manipulator, the path that manipulator can be flown by Controlling model is controlled by voice or word
Flown in safe range, so as to avoid blind flying and flying to lose that (situation as shown in Figure 3, manipulator only needs to beat toward lower left
Bar, comes with regard to unmanned vehicle can be allowed to fly back).
As another preferred scheme of the present embodiment, data obtaining module 10 also includes terminal attitude measurement module 103,
For the azimuth for the screen datum line for measuring display screen, the azimuth of the screen datum line of display screen is sent to information processing
Module 20；The positive optical axis of the message processing module 20 by the figure of display with respect to display screen reversely rotates azimuth.To cause display
The relative position and course angle of screen display unmanned vehicle are kept using ground as reference, the level with the display screen of remote terminal
Placement direction and to be disposed vertically direction unrelated.
Wherein, the screen datum line of display screen refers to the reference line parallel with one side of display screen, can be on display screen
The line at following midpoint, bottom and upper segment refers to when the figure of display is forward direction, and screen datum line is for the positive to be up
(as shown in Figures 3 and 4).The azimuth of the screen datum line of display screen refer to screen datum line ground level projection with just
The north to angle (B angles as shown in Figure 5).The positive optical axis of display screen refers to the axle perpendicular to display screen, it is also assumed that being
People soon display screen when, be to be observed perpendicular to the angle of display screen (as shown in Figure 7) of display screen.
When the display screen horizontal positioned of remote terminal, terminal attitude measurement module 103 can by magnetometer 1031 come
Realize, using the sensing of magnetometer 1031 as benchmark, obtain the screen datum line of display screen in the horizontal direction relative to the positive north
To the azimuth of rotation.The positive optical axis of the message processing module 202 by the figure of display with respect to display screen reversely rotates azimuth.This
In, the azimuth of screen datum line actually just refers to the angle of screen datum line and direct north, refers to Fig. 3 and Fig. 4, magnetic force
Meter can be realized by compass, such as according to the sensing of compass as benchmark, to determine unmanned vehicle on a display screen
The position shown in real time, it is ensured that as long as the position and course of unmanned vehicle do not change, no matter manipulator holds remote terminal
Change angle how is rotated on horizontal positioned direction, if according to the azimuth of the screen datum line of display screen in real time relative to
The positive optical axis of display screen reversely rotates the figure of display, it is ensured that display screen upward arrow and the line in the center of circle are relative to ground level
Angle keep constant, i.e., unmanned vehicle in Fig. 3 and Fig. 4 keeps constant relative to the azimuth a of observation position, and arrow
The sensing of head also keeps constant relative to the angle of ground level.If without compass functional, can not accomplish that angle a is kept
It is constant, but can ensure when display screen east-west direction horizontal positioned (as shown in Figure 3), arrow can reflect that nobody flies with the center of circle
The actual relative position of row device and remote terminal.
When the non-aqueous placing flat of the display screen of remote terminal, terminal attitude measurement module 103 can pass through magnetometer 1031
The azimuth of the screen datum line of display screen is obtained with accelerometer 1032, here the azimuth of the screen datum line of display screen
Refer to screen datum line in the projection of ground level and the angle of direct north.Magnetometer can be such as used to be calculated with accelerometer
Go out the posture Rbg of display screen relative to the earth, asking poor by position and the current screen position of unmanned vehicle obtains vectorial Pg,
Display screen display vector direction be：The x of vectorial (Pb=Rbg*Pg), y-coordinate, so as to obtain the screen datum line of display screen
Ground level projection relative to the azimuth that direct north rotates.Then the positive optical axis by the figure of display relative to display screen
Azimuth is reversely rotated, is kept using the relative position and course angle for show unmanned vehicle on display screen using ground level as ginseng
Examine, with the horizontal positioned direction of the display screen of remote terminal or to be disposed vertically direction unrelated.
It should be noted that the inapplicable special circumstances when display screen is disposed vertically of this programme.
As the present embodiment another preferred scheme, message processing module 20 is additionally operable to judge the screen of display screen
Whether azimuthal difference of the azimuth of datum line with unmanned vehicle relative to observation position is less than predetermined threshold value, and aobvious
The difference of the elevation angle of the elevation angle of display screen with unmanned vehicle relative to observation position again smaller than predetermined threshold value, if it is,
Then produced on screen and capture target prompting information.Wherein, the screen datum line of display screen refers to that one side with display screen is put down
Capable reference line, can be the line at following midpoint on display screen, bottom and upper segment refers to that it is forward direction to work as the figure of display, and shields
Curtain datum line be for the positive to be up (as shown in Figures 3 and 4).The azimuth of the screen datum line of display screen refers to screen
Datum line is in the projection of ground level and the angle (B angles as shown in Figure 5) of direct north.The elevation angle of display screen refers to display screen
The angle (rb angles as shown in Figure 9) of the positive optical axis and ground level, unmanned vehicle refers to nothing relative to the azimuth of observation position
People's aircraft and the line of observation position are in the projection of ground level and the angle of ground level direct north (such as Fig. 3, Fig. 4 and Fig. 5
A angles), the elevation angle of unmanned vehicle refers to that unmanned vehicle and the line of observation position and the angle of ground level are (as shown in Figure 8
Ra angles), predetermined threshold value is 10 degree.
For example, there is relational expression：Vv2=[v2x v2y v2z] ^T=Rgb* ([0 0 1] ^T)
Wherein, v2=vv2 the first two coordinate, v1=[x2-x1, y2-y1] ^T, v1 represents that vector is holded up to come, and v2 is
The z-axis of display screen takes the first two number to represent in x in the expression of world coordinate system, y direction projections, and x1, y1 is the seat of display screen
Mark, x2, y2 is the coordinate of unmanned vehicle, and T is transposition, and Rgb is display screen attitude matrix, when v2 and v1 differential seat angle is less than certain
Threshold value, represents display screen to the position of unmanned vehicle.
Referring to Fig. 6, when the display screen horizontal positioned of remote terminal, manipulator can hold display screen in the horizontal direction
Unmanned vehicle is found in upper and then arrow rotation, when display screen follow arrow rotate to display screen screen datum line azimuth
With unmanned vehicle relative to observation position azimuth it is consistent when (a angles), now, arrow should positioned at display screen just on
Side.Referring to Fig. 7, as manipulator at the azimuth of the screen datum line of Fig. 6 display screens and unmanned vehicle relative to observation position
On the basis of the azimuth put is consistent, elevation angle and unmanned vehicle phase that display screen rotates to display screen in vertical direction are held
For observe position elevation angle it is consistent when, now the positive optical axis of the display screen of non-aqueous placing flat is oriented to the position of unmanned vehicle
Put, produced on screen and capture target prompting information, can point out to catch by the color change of arrow disappearance or arrow
To target, the flight experience of manipulator is improved.
And in actual applications, display screen the positive optical axis level off to alignment unmanned vehicle in the case of (differential seat angle is small
In predetermined threshold value), it is also believed that having pointed to unmanned vehicle.Can so it understand, using the center of circle in Fig. 7 as one
The summit of vertical cone, the line in the center of circle to unmanned vehicle is the central axis of vertical cone, and predetermined threshold value is the top of vertical cone
Angle, so, sky in vertical cone is fallen when manipulator lifts the position that display screen follows the direction of arrow to rotate to unmanned vehicle
Between in the range of, then can point out display screen just to having arrived unmanned vehicle, now, the azimuth of the screen datum line of display screen with
Unmanned vehicle reaches unanimity relative to the azimuth of observation position, and the elevation angle and unmanned vehicle of display screen are relative to observation
The elevation angle of position reaches unanimity, and the positive optical axis of display screen tends to be oriented to the position of unmanned vehicle, is also produced on screen
Target prompting information is captured, the prompt message can be the color change of arrow disappearance or arrow, improve manipulator's
Flight experience.
The direction of the arrow shown on screen is followed to rotate searching nothing referring to Fig. 7, manipulator can hold remote terminal
People's aircraft, when display screen follow arrow rotate to display screen screen datum line azimuth and unmanned vehicle relative to sight
The azimuth for examining position is consistent, and display screen elevation angle it is consistent relative to the elevation angle of observation position with unmanned vehicle when,
Now the positive optical axis of the display screen of non-aqueous placing flat is oriented to the position of unmanned vehicle, and generation captures target and carried on screen
Show information.And in actual applications, display screen the positive optical axis level off to alignment unmanned vehicle in the case of (differential seat angle is less than
Predetermined threshold value), it is also believed that having pointed to unmanned vehicle.Produced on screen and capture target prompting information, carried
The high flight experience of manipulator.
In addition, remote terminal can also include voice module, for voice message unmanned vehicle positional information and fly
Row attitude information, including unmanned vehicle have many in how much degree in which direction of manipulator, unmanned vehicle apart from manipulator
Few rice, the elevation angle of unmanned vehicle are how many etc., for example, voice broadcast " north-east 30 of the unmanned vehicle at you
Degree, please come back 50 degree i.e. it can be seen that unmanned vehicle ", so as to preferably improve the flight experience of manipulator.
By the unmanned vehicle system of the present embodiment, manipulator is according to the relative position information and boat for showing screen display
To angle, the position where unmanned vehicle just can be easily recognized, eyes need not leave the display screen of remote terminal just
The flight path of manipulation unmanned vehicle freely can be carried out to unmanned vehicle, it is to avoid blind to fly and fly to lose, improve behaviour
The flight experience of control person.
It is a kind of unmanned vehicle accessory system structural representation that the present invention implements to provide, the system bag as shown in Figure 2
Include：Sensed including any one remote terminal in above-mentioned technical proposal and the state measurement being arranged on unmanned vehicle
Device, wherein：
State measurement sensor, for measuring the positional information and state of flight information of unmanned vehicle, and nobody is flown
The positional information and state of flight information transmission of row device are to remote terminal；
Remote terminal, is additionally operable to state of flight information and unmanned vehicle according to unmanned vehicle relative to observation position
Positional information, control unmanned vehicle flight.
Preferably, when observing the position that position is manipulator, the positional information of manipulator by be attached to manipulator or
Sensor on the object entrained by manipulator is obtained；When it is the position that unmanned vehicle takes off to observe position, nobody flies
The position that row device takes off is obtained by state measurement sensor.
It should be noted that, all technical schemes of remote terminal above are equally all suitable in the present embodiment, here
No longer repeat.
It is a kind of flight householder method of unmanned vehicle provided in an embodiment of the present invention, this method bag as shown in Figure 10
Include：
S1002, acquisition observation positional information, the positional information of unmanned vehicle and state of flight information；
Specifically, the positional information of unmanned vehicle and state of flight information pass through the shape that is arranged on unmanned vehicle
State measurement sensor is obtained；The position that observation position is taken off for the position of manipulator or unmanned vehicle, when observation position is behaviour
During the position of control person, the positional information of manipulator is by being attached to manipulator or the sensor on the object entrained by manipulator
Obtain；When it is the position that unmanned vehicle takes off to observe position, the position that unmanned vehicle takes off is by being arranged on aircraft
On state measurement sensor obtain.
S1004, the positional information according to unmanned vehicle and observation positional information obtain unmanned vehicle relative to observation
The positional information of position；
S1006, the state of flight information for exporting unmanned vehicle and the positional information relative to observation position；
Specifically, unmanned vehicle relative to observation position positional information include unmanned vehicle with observation position it
Between distance and unmanned vehicle relative to observation position angle；The course of state of flight information including unmanned vehicle and/
Or the elevation information relative to ground of unmanned vehicle.
Preferably, the positional information by the state of flight information and unmanned vehicle of unmanned vehicle relative to observation position
Output includes：By unmanned vehicle with observing angle, the nothing of the distance between position and unmanned vehicle relative to observation position
The course of people's aircraft is shown on a display screen by the form of figure.
S1008, the position letter of state of flight information and unmanned vehicle relative to observation position according to unmanned vehicle
Breath, controls the flight of unmanned vehicle.
As a kind of preferred scheme of the embodiment of the present invention, also include in step S1006：Obtain the screen base of display screen
The azimuth of directrix, and the positive optical axis reverse rotation azimuth by figure with respect to display screen.To cause the nothing shown on display screen
The relative position and course angle of people's aircraft are kept using ground as reference, the horizontal positioned direction or vertical with the display screen of remote control
Straight placement direction is unrelated (constant referring to Fig. 3 and Fig. 4 a angles).Wherein, the screen datum line of display screen refers to and display screen
Parallel reference line, can be the line at following midpoint on display screen on one side, and bottom and upper segment refers to the figure when display for just
To, and screen datum line be for the positive to be up (as shown in Figures 3 and 4).The azimuth of the screen datum line of display screen
Refer to screen datum line in the projection of ground level and the angle (B angles as shown in Figure 5) of direct north.The positive optical axis of display screen is
Refer to perpendicular to display screen axle, it is also assumed that be people soon display screen when, be aobvious to be observed perpendicular to the angle of display screen
(as shown in Figure 7) of display screen.
As a kind of preferred scheme of the embodiment of the present invention, also include in step S1006：Judge the screen base of display screen
Whether the azimuth of directrix is less than predetermined threshold value, and display with unmanned vehicle relative to azimuthal difference of observation position
The difference of elevation angle of the elevation angle of screen with unmanned vehicle relative to observation position is less than predetermined threshold value, if it is,
Produced on screen and capture target prompting information.That is the direction of arrow is followed to rotate searching when manipulator holds display screen
During unmanned vehicle, when the positive optical axis of display screen levels off to sensing unmanned vehicle (when i.e. differential seat angle is less than predetermined threshold),
Produced on screen and capture target prompting information.The prompt message can be the color change of arrow disappearance or arrow, from
And improve the flight experience (Fig. 7 please be participate in) of manipulator.
In addition, remote terminal can also include voice module, for voice message unmanned vehicle positional information and fly
Row attitude information, including unmanned vehicle have many in how much degree in which direction of manipulator, unmanned vehicle apart from manipulator
Few rice, the elevation angle of unmanned vehicle are how many etc., for example, voice broadcast " north-east 30 of the unmanned vehicle at you
Degree, please come back 50 degree i.e. it can be seen that unmanned vehicle ", so as to preferably improve the flight experience of manipulator.
It should be noted that the various technical schemes of the remote terminal of above-mentioned unmanned vehicle, in this unmanned vehicle
Equally applicable in flight householder method, identical detailed content is no longer repeated.
The remote terminal of unmanned vehicle provided in an embodiment of the present invention, assisting in flying system and method, by grasping nothing
The flight position and state of flight of people's aircraft, helps manipulator to control the flight path of unmanned vehicle, it is to avoid it is blind fly and
Fly to lose, while improving the flight experience of manipulator.
Above by reference to the preferred embodiments of the present invention have been illustrated, not thereby limit to the interest field of the present invention.This
Art personnel do not depart from the scope of the present invention and essence, can have a variety of flexible programs to realize the present invention, for example as one
The feature of individual embodiment can be used for another embodiment and obtain another embodiment.All institutes within the technical concept with the present invention
Any modifications, equivalent substitutions and improvements made, all should be within the interest field of the present invention.
Claims (15)
1. a kind of remote terminal of unmanned vehicle, it is characterised in that the remote terminal includes data obtaining module, information processing
Module and message output module, wherein：
Described information acquisition module, the positional information for obtaining observation position obtains the positional information of unmanned vehicle and flown
Row status information；
Described information processing module, the positional information for the positional information according to the unmanned vehicle and observation position is obtained
Positional information of the unmanned vehicle relative to observation position；
Described information output module, state of flight information and nothing for receiving and being shown by display screen the unmanned vehicle
Positional information of people's aircraft relative to observation position；
Described information processing module, is additionally operable to judge azimuth and the unmanned vehicle of screen datum line of display screen relative to sight
Whether the azimuthal difference for examining position is less than predetermined threshold value, and display screen elevation angle and unmanned vehicle relative to observation
The difference of the elevation angle of position is less than predetermined threshold value, and target prompting information is captured if it is, being produced on screen.
2. the remote terminal of unmanned vehicle according to claim 1, it is characterised in that the observation position is manipulator
Position or the position taken off of unmanned vehicle.
3. the remote terminal of unmanned vehicle according to claim 2, it is characterised in that the remote terminal also includes position
Measurement module, for measuring the positional information of the remote terminal as the positional information of manipulator, and sends described information to
Processing module.
4. the remote terminal of the unmanned vehicle according to claim 1 or 2 or 3, it is characterised in that described information exports mould
Block specifically for：Pass through figure by the state of flight information of the unmanned vehicle and relative to the positional information of observation position
Form is shown on a display screen.
5. the remote terminal of unmanned vehicle according to claim 4, it is characterised in that the unmanned vehicle relative to
Observing the positional information of position includes unmanned vehicle with observing the distance between position and unmanned vehicle relative to observation position
The angle put；The state of flight information includes the course of unmanned vehicle and/or the height relative to ground of unmanned vehicle
Spend information.
6. the remote terminal of unmanned vehicle according to claim 5, it is characterised in that described information acquisition module is also wrapped
Terminal attitude measurement module is included, wherein：
The terminal attitude measurement module, the azimuth of the screen datum line for detecting the display screen；
Described information processing module, for the positive optical axis of the relatively described display screen of the figure to be reversely rotated into the azimuth.
7. the remote terminal of unmanned vehicle according to claim 1, the predetermined threshold value is 10 degree.
8. a kind of flight assisting system of unmanned vehicle, it is characterised in that the system is weighed including claim 1-7 any one
Profit requires described remote terminal and the state measurement sensor being arranged on unmanned vehicle, wherein：
The state measurement sensor, positional information and state of flight information for measuring unmanned vehicle, and by the nothing
The positional information and state of flight information transmission of people's aircraft give the remote terminal；
The remote terminal, is additionally operable to state of flight information and unmanned vehicle according to the unmanned vehicle relative to observation
The positional information of position, controls the flight of unmanned vehicle.
9. the flight assisting system of unmanned vehicle according to claim 8, it is characterised in that
When observing the position that position is manipulator, the positional information of manipulator is by being attached to manipulator or being taken in manipulator
Sensor on the object of band is obtained；
When it is the position that unmanned vehicle takes off to observe position, the position that unmanned vehicle takes off is passed by the state measurement
Sensor is obtained.
10. the flight householder method of a kind of unmanned vehicle, it is characterised in that this method includes：
Obtain the state of flight information of observation positional information, the positional information of unmanned vehicle and unmanned vehicle；
Unmanned vehicle is obtained relative to observation according to the positional information of the positional information of the unmanned vehicle and observation position
The positional information of position；And it is by the state of flight information of the unmanned vehicle and defeated relative to the positional information of observation position
Go out；
Position letter according to the state of flight information and unmanned vehicle of the unmanned vehicle of the output relative to observation position
Breath, controls the flight of unmanned vehicle；
This method also includes：Judge the side of azimuth and unmanned vehicle relative to observation position of the screen datum line of display screen
Whether the difference of parallactic angle is less than predetermined threshold value, and the height of elevation angle and unmanned vehicle relative to observation position of display screen
The difference at angle is less than predetermined threshold value, and target prompting information is captured if it is, being produced on screen.
11. the flight householder method of unmanned vehicle according to claim 10, it is characterised in that the unmanned vehicle
Positional information and state of flight information obtained by the state measurement sensor that is arranged on unmanned vehicle；The observation position
The position that the position or unmanned vehicle for being set to manipulator are taken off, when observing the position that position is manipulator, the position of manipulator
Confidence breath is obtained by being attached to manipulator or the sensor on the object entrained by manipulator；When observation position flies for nobody
During the position that row device takes off, the position that unmanned vehicle takes off is obtained by being arranged on carry-on state measurement sensor.
12. the flight householder method of the unmanned vehicle according to claim 10 or 11, it is characterised in that by it is described nobody
Positional information output of the state of flight information and unmanned vehicle of aircraft relative to observation position includes：By it is described nobody fly
The state of flight of row device and relative to observation position positional information shown on a display screen by the form of figure.
13. the flight householder method of unmanned vehicle according to claim 12, it is characterised in that the unmanned vehicle
Relative to observation position positional information include unmanned vehicle and observation the distance between position and unmanned vehicle relative to
Observe the angle of position；The course of the state of flight information including unmanned vehicle and/or unmanned vehicle relative to ground
The elevation information in face.
14. the flight householder method of unmanned vehicle according to claim 13, it is characterised in that this method also includes：
The azimuth of the screen datum line of the display screen is obtained, and the positive optical axis of the relatively described display screen of the figure is reverse
Rotate the azimuth.
15. the flight householder method of unmanned vehicle according to claim 10, the predetermined threshold value is 10 degree.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710457599.5A CN107256030B (en) | 2013-07-05 | 2013-07-05 | Remote control terminal, flight assistance system and method of unmanned aerial vehicle |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310282462.2A CN103345826B (en) | 2013-07-05 | 2013-07-05 | Remote terminal, flight assisting system and the method for unmanned vehicle |
| CN201710457599.5A CN107256030B (en) | 2013-07-05 | 2013-07-05 | Remote control terminal, flight assistance system and method of unmanned aerial vehicle |
Related Parent Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310282462.2A Division CN103345826B (en) | 2013-07-05 | 2013-07-05 | Remote terminal, flight assisting system and the method for unmanned vehicle |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107256030A true CN107256030A (en) | 2017-10-17 |
| CN107256030B CN107256030B (en) | 2021-01-12 |
Family
ID=49280618
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710457599.5A Active CN107256030B (en) | 2013-07-05 | 2013-07-05 | Remote control terminal, flight assistance system and method of unmanned aerial vehicle |
| CN201310282462.2A Active CN103345826B (en) | 2013-07-05 | 2013-07-05 | Remote terminal, flight assisting system and the method for unmanned vehicle |
Family Applications After (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310282462.2A Active CN103345826B (en) | 2013-07-05 | 2013-07-05 | Remote terminal, flight assisting system and the method for unmanned vehicle |
Country Status (1)
| Country | Link |
|---|---|
| CN (2) | CN107256030B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019144288A1 (en) * | 2018-01-23 | 2019-08-01 | SZ DJI Technology Co., Ltd. | Devices and systems utilizing single chip for control of device movement |
Families Citing this family (24)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103950546B (en) * | 2014-04-21 | 2016-02-24 | 深圳市大疆创新科技有限公司 | Unmanned plane and state of flight auxiliary reminding method thereof |
| CN105083120B (en) * | 2014-04-30 | 2018-12-21 | 比亚迪股份有限公司 | The detection system and flight instruments of environment surrounding automobile |
| EP3164768A1 (en) | 2014-09-30 | 2017-05-10 | SZ DJI Technology Co., Ltd. | Systems and methods for flight simulation |
| CN113628500A (en) * | 2014-09-30 | 2021-11-09 | 深圳市大疆创新科技有限公司 | System and method for supporting analog mobility |
| WO2016065513A1 (en) * | 2014-10-27 | 2016-05-06 | 深圳市大疆创新科技有限公司 | Method and apparatus for prompting position of air vehicle |
| CN104656482A (en) * | 2015-02-03 | 2015-05-27 | 昆山优力电能运动科技有限公司 | Terminal remote control device |
| CN105992980A (en) | 2015-05-18 | 2016-10-05 | 深圳市大疆创新科技有限公司 | Unmanned aerial vehicle control method and device based on headless mode |
| US10598497B2 (en) * | 2015-08-18 | 2020-03-24 | X-Control System Co., Ltd. | Method and device for generating geographic coordinates |
| CN105204520B (en) * | 2015-09-17 | 2018-07-06 | 深圳市十方联智科技有限公司 | A kind of control method of unmanned vehicle |
| CN106647794B (en) * | 2015-10-28 | 2020-01-10 | 小米科技有限责任公司 | Flight control method and device |
| CN105388906A (en) * | 2015-11-30 | 2016-03-09 | 无锡觅睿恪科技有限公司 | Height detection remote control for unmanned aerial vehicle (UAV) |
| CN107318268B (en) | 2016-03-01 | 2020-07-17 | 深圳市大疆创新科技有限公司 | Flight control method, device, control terminal, flight system and processor |
| CN107444639B (en) * | 2016-05-30 | 2024-03-08 | 松下电器（美国）知识产权公司 | Unmanned aerial vehicle, control method, and non-transitory recording medium |
| CN106027996A (en) * | 2016-07-12 | 2016-10-12 | 南京思钛普电子有限公司 | Unmanned vehicles controlling means, unmanned vehicles and use its control system |
| WO2018045654A1 (en) * | 2016-09-09 | 2018-03-15 | 深圳市大疆创新科技有限公司 | Method and system for displaying state of mobile device and control device |
| CN106324639B (en) * | 2016-09-13 | 2019-06-21 | 上海顺砾智能科技有限公司 | A kind of method that unmanned plane independently makes a return voyage |
| CN108170156A (en) * | 2016-12-07 | 2018-06-15 | 广州亿航智能技术有限公司 | Flight state output method and device based on intelligent terminal |
| CN106843266B (en) * | 2016-12-30 | 2024-04-05 | 歌尔科技有限公司 | Remote controller, orientation control system and azimuth indication method of aircraft |
| CN108780323A (en) * | 2017-01-20 | 2018-11-09 | 深圳市大疆创新科技有限公司 | The control method and unmanned plane of unmanned plane |
| CN114564036A (en) * | 2017-12-26 | 2022-05-31 | 深圳市大疆创新科技有限公司 | Flight trajectory original path rehearsal method and aircraft |
| WO2019127303A1 (en) * | 2017-12-29 | 2019-07-04 | 深圳市大疆创新科技有限公司 | Method for prompting and determining orientation of unmanned aerial vehicle, and control terminal |
| DE102018123411A1 (en) * | 2018-09-24 | 2020-03-26 | Autel Robotics Europe Gmbh | Target observation method, associated device and system |
| CN109754587A (en) * | 2018-12-26 | 2019-05-14 | 天长市志铖电子科技有限公司 | The method and remote controler of a kind of pair of unmanned plane and remote controler pairing |
| CN109741637B (en) * | 2019-01-08 | 2021-12-14 | 沈阳无距科技有限公司 | Unmanned aerial vehicle supervision method and device, storage medium and electronic equipment |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH08164896A (en) * | 1994-12-15 | 1996-06-25 | Mitsubishi Heavy Ind Ltd | Visibility display in operating unmanned aircraft |
| JPH09304526A (en) * | 1996-05-15 | 1997-11-28 | Nec Corp | Three-dimensional information display method for terminal control |
| JP2002308195A (en) * | 2001-04-16 | 2002-10-23 | Tech Res & Dev Inst Of Japan Def Agency | Other craft position display method and device in aircraft |
| CN1515914A (en) * | 2001-12-29 | 2004-07-28 | 北京航空航天大学 | Operation method of antenna tracking device for unmanned hverplane |
| EP1645505A2 (en) * | 2002-10-08 | 2006-04-12 | Fuji Jukogyo Kabushiki Kaisha | Unmanned helicopter, takeoff method of unmanned helicopter, and landing method of unmanned helicopter |
| CN101266078A (en) * | 2008-04-29 | 2008-09-17 | 河海大学 | Heliostat tracing controlling apparatus and its control method |
| EP1307797B2 (en) * | 2000-05-17 | 2009-04-08 | The Boeing Company | Intuitive vehicle and machine control |
| WO2009087543A2 (en) * | 2008-01-08 | 2009-07-16 | Rafael Advanced Defense Systems Ltd. | System and method for navigating a remote control vehicle past obstacles |
| CN101809993A (en) * | 2007-07-29 | 2010-08-18 | 奈米光子有限公司 | methods of obtaining panoramic images using rotationally symmetric wide-angle lenses and devices thereof |
| CN102148965A (en) * | 2011-05-09 | 2011-08-10 | 上海芯启电子科技有限公司 | Video monitoring system for multi-target tracking close-up shooting |
| CN102289230A (en) * | 2010-06-18 | 2011-12-21 | Nes&Tec有限公司 | Flight altitude control system of pilotless aircraft |
| US20120173053A1 (en) * | 2011-01-04 | 2012-07-05 | Kabushiki Kaisha Topcon | Flight Control System For Flying Object |
| CN202694151U (en) * | 2011-12-16 | 2013-01-23 | 新时代集团国防科技研究中心 | Control terminal device for unmanned aircraft |
| US20130085629A1 (en) * | 2011-09-30 | 2013-04-04 | Aurora Flight Sciences Corporation | Hardware-Based Weight And Range Limitation System, Apparatus And Method |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102298799B (en) * | 2010-06-25 | 2014-03-26 | 鸿富锦精密工业（深圳）有限公司 | Hand-held device and method for controlling unmanned flying vehicle by utilizing same |
| KR101263441B1 (en) * | 2011-06-29 | 2013-05-10 | 주식회사 네스앤텍 | Method and system of flight control for unmanned aerial vehicle |
| CN102495634A (en) * | 2011-12-07 | 2012-06-13 | 中国南方航空工业(集团)有限公司 | Method and device for controlling unmanned aerial vehicle and operating device for unmanned aerial vehicle |
| CN202758243U (en) * | 2012-09-06 | 2013-02-27 | 北京工业大学 | Unmanned aerial vehicle flight control system |
| CN102890283A (en) * | 2012-11-06 | 2013-01-23 | 白中泽 | Comprehensive location method and comprehensive location system of point-to-point location and GPS (Global Positioning System) satellite location of unmanned aerial vehicle |
-
2013
- 2013-07-05 CN CN201710457599.5A patent/CN107256030B/en active Active
- 2013-07-05 CN CN201310282462.2A patent/CN103345826B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH08164896A (en) * | 1994-12-15 | 1996-06-25 | Mitsubishi Heavy Ind Ltd | Visibility display in operating unmanned aircraft |
| JPH09304526A (en) * | 1996-05-15 | 1997-11-28 | Nec Corp | Three-dimensional information display method for terminal control |
| EP1307797B2 (en) * | 2000-05-17 | 2009-04-08 | The Boeing Company | Intuitive vehicle and machine control |
| JP2002308195A (en) * | 2001-04-16 | 2002-10-23 | Tech Res & Dev Inst Of Japan Def Agency | Other craft position display method and device in aircraft |
| CN1515914A (en) * | 2001-12-29 | 2004-07-28 | 北京航空航天大学 | Operation method of antenna tracking device for unmanned hverplane |
| EP1645505A2 (en) * | 2002-10-08 | 2006-04-12 | Fuji Jukogyo Kabushiki Kaisha | Unmanned helicopter, takeoff method of unmanned helicopter, and landing method of unmanned helicopter |
| CN101809993A (en) * | 2007-07-29 | 2010-08-18 | 奈米光子有限公司 | methods of obtaining panoramic images using rotationally symmetric wide-angle lenses and devices thereof |
| WO2009087543A2 (en) * | 2008-01-08 | 2009-07-16 | Rafael Advanced Defense Systems Ltd. | System and method for navigating a remote control vehicle past obstacles |
| CN101266078A (en) * | 2008-04-29 | 2008-09-17 | 河海大学 | Heliostat tracing controlling apparatus and its control method |
| CN102289230A (en) * | 2010-06-18 | 2011-12-21 | Nes&Tec有限公司 | Flight altitude control system of pilotless aircraft |
| US20120173053A1 (en) * | 2011-01-04 | 2012-07-05 | Kabushiki Kaisha Topcon | Flight Control System For Flying Object |
| CN102148965A (en) * | 2011-05-09 | 2011-08-10 | 上海芯启电子科技有限公司 | Video monitoring system for multi-target tracking close-up shooting |
| US20130085629A1 (en) * | 2011-09-30 | 2013-04-04 | Aurora Flight Sciences Corporation | Hardware-Based Weight And Range Limitation System, Apparatus And Method |
| CN202694151U (en) * | 2011-12-16 | 2013-01-23 | 新时代集团国防科技研究中心 | Control terminal device for unmanned aircraft |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019144288A1 (en) * | 2018-01-23 | 2019-08-01 | SZ DJI Technology Co., Ltd. | Devices and systems utilizing single chip for control of device movement |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103345826B (en) | 2017-07-18 |
| CN103345826A (en) | 2013-10-09 |
| CN107256030B (en) | 2021-01-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103345826B (en) | Remote terminal, flight assisting system and the method for unmanned vehicle | |
| CN103344250B (en) | The flight householder method and device of unmanned vehicle | |
| CN103365298B (en) | The flight assisting system and method for unmanned vehicle | |
| US10648809B2 (en) | Adaptive compass calibration based on local field conditions | |
| US20200241389A1 (en) | Flying camera with string assembly for localization and interaction | |
| CN106708066B (en) | View-based access control model/inertial navigation unmanned plane independent landing method | |
| CN103285599B (en) | A kind of method by remote control means navigator's target drone directly perceived | |
| CN108351653B (en) | System and method for UAV flight control | |
| CN110347171B (en) | Aircraft control method and aircraft | |
| CN109219785A (en) | Simple multisensor calibration | |
| CN110312913A (en) | Method and system for adaptive holder | |
| CN111966114A (en) | Drop-off location planning for delivery vehicles | |
| JP2020508615A (en) | Multi-gimbal assembly | |
| CN106094863A (en) | The system of unmanned plane is driven for immersion | |
| CN107615211A (en) | Merged using the sensor of inertial sensor and imaging sensor | |
| CN107850901A (en) | Merged using the sensor of inertial sensor and imaging sensor | |
| US20120223181A1 (en) | Lighter-Than-Air Systems, Methods, and Kits for Obtaining Aerial Images | |
| CN107850436A (en) | Merged using the sensor of inertial sensor and imaging sensor | |
| CN107850899A (en) | Merged using the sensor of inertial sensor and imaging sensor | |
| CN106774378A (en) | A kind of UAV Flight Control and localization method | |
| CN108885451A (en) | For controlling the method and system of loose impediment | |
| WO2019204405A1 (en) | Perforated capsule hook for stable high speed retract | |
| CN107466384A (en) | A kind of method and device for following the trail of target | |
| JP2019032234A (en) | Display device | |
| CN107340780A (en) | Control method, the device and system of multi-rotor aerocraft |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |