CN107329470A - A kind of control method for the robot that paddles, device and the robot that paddles - Google Patents
A kind of control method for the robot that paddles, device and the robot that paddles Download PDFInfo
- Publication number
- CN107329470A CN107329470A CN201710423940.5A CN201710423940A CN107329470A CN 107329470 A CN107329470 A CN 107329470A CN 201710423940 A CN201710423940 A CN 201710423940A CN 107329470 A CN107329470 A CN 107329470A
- Authority
- CN
- China
- Prior art keywords
- luminaire
- movement locus
- robot
- determined
- paddles
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 64
- 230000033001 locomotion Effects 0.000 claims abstract description 162
- 230000000875 corresponding effect Effects 0.000 claims abstract description 63
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 claims abstract description 44
- 238000003860 storage Methods 0.000 claims abstract description 23
- 238000004891 communication Methods 0.000 claims abstract description 14
- 238000004590 computer program Methods 0.000 claims description 19
- 230000015654 memory Effects 0.000 claims description 19
- 230000001052 transient effect Effects 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 claims description 2
- 230000005055 memory storage Effects 0.000 claims description 2
- 230000006399 behavior Effects 0.000 claims 1
- 230000009189 diving Effects 0.000 description 13
- 230000006870 function Effects 0.000 description 11
- 238000010586 diagram Methods 0.000 description 10
- 238000012545 processing Methods 0.000 description 9
- 230000009471 action Effects 0.000 description 7
- 230000008569 process Effects 0.000 description 5
- 230000004048 modification Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 241000406668 Loxodonta cyclotis Species 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000004807 localization Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000010295 mobile communication Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 239000003795 chemical substances by application Substances 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 230000001276 controlling effect Effects 0.000 description 1
- 238000000151 deposition Methods 0.000 description 1
- 238000007667 floating Methods 0.000 description 1
- 238000001764 infiltration Methods 0.000 description 1
- 230000008595 infiltration Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000035515 penetration Effects 0.000 description 1
- 230000011514 reflex Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
- G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
- G05D1/0253—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means extracting relative motion information from a plurality of images taken successively, e.g. visual odometry, optical flow
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
- G05D1/0214—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory in accordance with safety or protection criteria, e.g. avoiding hazardous areas
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0276—Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle
- G05D1/0285—Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle using signals transmitted via a public communication network, e.g. GSM network
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Electromagnetism (AREA)
- Manipulator (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention discloses a kind of control method for the robot that paddles, device and the robot that paddles, methods described includes：The multiple image for being located at the luminaire that measurand under water is carried is obtained in preset time；And according to the multiple image, determine the movement locus of the luminaire；According to the movement locus, the movement locus type and the direction of motion of the luminaire are determined；And the corresponding relation of the movement locus type, the direction of motion and operational order according to storage, determine the corresponding operational order of movement locus type, the direction of motion of the luminaire；And according to the operational order determined, perform corresponding action.The method provided using the present invention, is not only realized the direct communication of scuba diver and the robot that paddles, so that the robot that paddles is able to carry out the various instructions that scuba diver sends, greatly meets the enjoyment of people's sub-aqua sport.
Description
Technical field
The present invention relates to technical field of robot control in water, more particularly to a kind of control method for the robot that paddles, dress
Put and paddle robot.
Background technology
Utilization is continually developed with the raising and marine resources of people's level of consumption, to dive under water as the fortune under water of representative
The dynamic leisure as more and more people, temper selection, and gradually grow up robot sub-aqua sport of paddling, take pictures,
The functions such as shooting, are capable of the enjoyment of significant increase people's sub-aqua sport.
Communicated in order to achieve the above object, it is necessary to paddle and can be realized between robot and scuba diver, and in the prior art
It is to be connected using wire cable with control cabinet on the water surface with the method for robot communication of paddling generally to realize people, passes through wire cable
The communication between control cabinet is realized, so as to realize the indirect communication of people and the robot that paddles, but the logical of wire cable is used
Letter method, it is adaptable to which people carries out indirect communication with the robot that paddles under water, if being applied to scuba diver and machine of paddling
The communication of people, then scuba diver need to carry with wire cable, this mode requires that wire cable is moved with scuba diver, so not
Only it is unfavorable for scuba diver and does various actions of taking pictures, be less useful for scuba diver and move about, substantially reduces scuba diver's experience；In addition, wired
Cable is connected by water joint with the robot that paddles, with the passage of time, because water joint infiltration to wire cable with relating to
Communication between water robot threatens, and the communication in turn resulting in people between the robot that paddles is not smooth.
Therefore, in the case where not using wire cable, how by controlling the robot that paddles so that robot energy of paddling
Enough real-time tracing users simultaneously perform the various action commands that diving user sends, and greatly meeting the enjoyment of people's sub-aqua sport is
One of technical problem urgently to be resolved hurrily.
The content of the invention
The embodiment of the present invention provides a kind of control method for the robot that paddles, device and the robot that paddles, not adopt
In the case of wire cable, the various action commands that diving user sends are performed, the pleasure of people's sub-aqua sport is greatly met
Interest.
The embodiment of the present invention provides a kind of control method for the robot that paddles, including：
The multiple image for being located at the luminaire that measurand under water is carried is obtained in preset time；And
According to the multiple image, the movement locus of the luminaire is determined；
According to the movement locus, the movement locus type and the direction of motion of the luminaire are determined；And
According to the corresponding relation of the movement locus type, the direction of motion and operational order of storage, the luminaire is determined
Movement locus type, the corresponding operational order of the direction of motion；And
According to the operational order determined, corresponding action is performed.
The embodiment of the present invention provides a kind of control device for the robot that paddles, including：
Collecting unit, the multiframe figure for the luminaire that measurand under water is carried is located at for being obtained in preset time
Picture；
First determining unit, for according to the multiple image, determining the movement locus of the luminaire；
Second determining unit, for according to the movement locus, determining the movement locus type and fortune of the luminaire
Dynamic direction；
3rd determining unit, for the corresponding relation of the movement locus type according to storage, the direction of motion and operational order,
Determine the corresponding operational order of movement locus type, the direction of motion of the luminaire；
Execution unit, for according to the operational order determined, performing corresponding action.
The embodiment of the present invention provides a kind of nonvolatile computer storage media, and be stored with computer executable instructions, institute
State computer executable instructions and be used to performing the application any of the above-described and paddle the control method of robot.
The embodiment of the present invention provides one kind and paddled robot, including：At least one processor；And
The memory being connected with least one described processor communication；Wherein, the memory storage have can by it is described extremely
The instruction of a few computing device, the instruction is by least one described computing device, so that at least one described processing
Device is able to carry out the application any of the above-described and paddled the control method of robot.
The embodiment of the present invention provides a kind of computer program product, and the computer program product is non-transient including being stored in
Computer program on computer-readable recording medium, the computer program includes programmed instruction, wherein, when described program refers to
Order is when being computer-executed, and the computer is performed the application any of the above-described and paddles the control method of robot.
Beneficial effects of the present invention：
Control method, device and the robot that paddles of the robot provided in an embodiment of the present invention that paddles, in preset time
Obtain and be located at after the multiple image for the luminaire that measurand under water is carried, according to the multiple image, determine the hair
The movement locus of light device；Then according to the movement locus, movement locus type and the motion side of the luminaire are determined
To；And the corresponding relation of the movement locus type, the direction of motion and operational order according to storage, determine the luminaire
The corresponding operational order of movement locus type, the direction of motion；And according to the operational order determined, perform corresponding action.By
This is realized under conditions of without using wire cable, the direct communication of people and the robot that paddles, and by the robot that paddles
Control so that the movement locus that the robot that paddles can be drawn out according to measurand using the luminaire, perform phase
The action answered, not only meets the enjoyment of people's sub-aqua sport, and the robot that paddles can also be commanded to perform some detection operations, expands
The use scope for the robot that paddles is opened up.
Other features and advantages of the present invention will be illustrated in the following description, also, partly becomes from specification
Obtain it is clear that or being understood by implementing the present invention.The purpose of the present invention and other advantages can be by the explanations write
Specifically noted structure is realized and obtained in book, claims and accompanying drawing.
Brief description of the drawings
Accompanying drawing described herein is used for providing a further understanding of the present invention, constitutes the part of the present invention, this hair
Bright schematic description and description is used to explain the present invention, does not constitute inappropriate limitation of the present invention.In the accompanying drawings：
Fig. 1 a are the schematic flow sheet of the control method for the robot that paddles that the embodiment of the present invention one is provided；
Fig. 1 b be the embodiment of the present invention one in determine the luminaire movement locus method schematic flow sheet；
The signal of default movement locus in the control method for the robot that paddles that Fig. 2 provides for the embodiment of the present invention one
Figure；
Equipment is carried out to measurand in the control method for the robot that paddles that Fig. 3 a provide for the embodiment of the present invention two
The schematic flow sheet of method；
Fig. 3 b are the schematic diagram of determination glow frequency in the method that the embodiment of the present invention two is provided；
The schematic flow sheet being tracked in the method that Fig. 4 provides for the embodiment of the present invention two to measurand；
Fig. 5 is the structural representation of the control device for the robot that paddles that the embodiment of the present invention three is provided；
Fig. 6 is the hardware architecture diagram for the robot that paddles that the embodiment of the present invention five is provided.
Embodiment
Control method, device and the robot that paddles of the robot provided in an embodiment of the present invention that paddles, not use
In the case of wire cable, the various action commands that diving user sends are performed, the enjoyment of people's sub-aqua sport is greatly met.
It should be noted that the control method of the robot provided in an embodiment of the present invention that paddles can apply to the user that dives under water
Paddled the scene that robot taken pictures by control, substantially process is：A luminaire is carried under diving user before water, such as
Signal lamp or reflex reflector, if the luminaire is signal lamp, diving user open signal lamp, and will be described under water
Signal lamp is moved according to the corresponding track of photographing instruction, and the corresponding movement locus of such as photographing instruction is U-shaped track, then dives
Water user drives the signal lamp to be moved along U-shaped track.Then paddle robot multi collect and obtain be located under water
Diving user carry signal lamp multiple image, then according to multi collect and obtain multiple image, determine the letter
The movement locus (U-shaped movement locus) of signal lamp；Further according to the movement locus, and the movement locus prestored refers to operation
The corresponding relation of order, determines the corresponding operational order of movement locus (action of taking pictures) of the luminaire；And according to determining
Operational order (action of taking pictures), corresponding action (being taken pictures to diving user) is performed, it is possible thereby to pass through signal lamp control
Make the robot that paddles so that the robot that paddles takes pictures to diving user, improves the experience of diving user.
In addition, the executive agent of the control method of the robot provided in an embodiment of the present invention that paddles can be machine of paddling
People, and the robot that paddles is provided with camera device, the measurand being located under water by camera device multi collect and acquisition is carried
Luminaire multiple image；After multiple image is obtained, multiple image is entered by the image processor in the robot that paddles
Row processing, and the movement locus of the luminaire is determined with reference to the master controller for the robot that paddles, then further according to the fortune
Dynamic rail mark, and the movement locus and the corresponding relation of operational order prestored, determine the movement locus of the luminaire
Corresponding operational order；The operational order determined is sent to corresponding execution mould by master controller after operational order is determined
Block, performs the corresponding action of the operational order.
The preferred embodiments of the present invention are illustrated below in conjunction with Figure of description, it will be appreciated that described herein
Preferred embodiment is merely to illustrate and explain the present invention, and is not intended to limit the present invention, and in the case where not conflicting, this hair
The feature in embodiment and embodiment in bright can be mutually combined.
Embodiment one
As shown in Figure 1a, be the embodiment of the present invention one provide the robot that paddles control method schematic flow sheet, can
To comprise the following steps：
S11, the multiple image that the luminaire for being located at measurand carrying under water is obtained in preset time.
When it is implemented, can be according to the image for gathering the luminaire in preset time with fixed frequency, such as pre-
If it is luminous that the measurand being located under water using the camera multi collect and acquisition for robot built-in of paddling in the time is carried
The multiple image of equipment, such as preset time are 1 second, and every collection in 0.02 second once, then can be gathered 50 times in 1 second, and point
50 two field pictures of the luminaire are not obtained.
Specifically, shape is opened, it is necessary to be in positioned at the luminaire that measurand under water is carried when performing the present invention
State, then measurand perform corresponding motion using the luminaire, based on this, could realize that the control robot that paddles is held
The purpose of row corresponding actions.
It is preferred that the luminaire can be, but not limited to include signal lamp and light deflection device；If the luminaire
For signal lamp, then need to carry the measurand manual turning-on signal lamp of the signal lamp；If the luminaire is light deflection device
When, then before multi collect and acquisition are located at the multiple image for the luminaire that measurand under water is carried, in addition to：
Detecting lamp is opened in advance, wherein the detecting lamp is used for when being pointed to measurand under water and being detected, it is tactile
The light deflection device for sending out measurand described lights.
When it is implemented, the light deflection device can be the equipment with reflectorized material, when detecting lamp is irradiated to reflective set
Standby upper, reflectorized material thereon can produce the light of high brightness under the irradiation of detecting lamp.So that robot collection of paddling is described anti-
The multiple image of light device.
It is preferred that paddling robot before step S11 is performed, it is necessary to begin look for working as by the camera device of its own
Whether front position has diving user, if it is not, can be rotated according to predetermined angle since current location, waits to rotate
After predetermined angle, searched again, until finding diving user；Or looked into while can also be using rotation
Look for, this process searched in rotation is relatively accurate, can also search diving user using other method certainly.Looking into
Find after diving user, then perform the control method for the robot that paddles that the present invention is provided.
S12, according to the multiple image, determine the movement locus of the luminaire.
When it is implemented, after multiple image is obtained, for each two field picture, recording the position of luminaire, then will
The position of luminaire is connected in multiple image, determines the movement locus of the luminaire.
It is preferred that the movement locus of the luminaire can be determined according to the method shown in Fig. 1 b, it can include following
Step：
S121, according to the multiple image, using the coordinate system pre-established, determine described in each two field picture light set
Standby coordinate value.
When it is implemented, the rectangular coordinate system UOV on pixel, image can be set up using the image upper left corner as round dot
The transverse and longitudinal coordinate of middle any point represents the pixel value of the point, it is possible thereby to for each two field picture, obtain the luminaire
Pixel value (u, v), the coordinate value of as described luminaire.It should be noted that the luminaire can not be one again
Spot light, so the luminaire in the every two field picture collected is likely to be a circle, now we can obtain every frame
The pixel value of the home position of the circle of this in image, is used as the coordinate value of the luminaire.
S122, the luminaire according to each two field picture determined coordinate value, determine the luminaire
Movement locus.
When it is implemented, determining described in each two field picture after the coordinate value of luminaire, map that to same
In coordinate system, and the movement locus of the luminaire is drawn out in the coordinate system.
S13, according to the movement locus, determine the movement locus type and the direction of motion of the luminaire.
When it is implemented, after the movement locus that step S12 determines the luminaire, you can true according to movement locus
The movement locus type and the direction of motion of the luminaire are made, the luminaire can be determined from multiple image
Original position, several centre positions and final position, thus will obtain the movement locus of the luminaire, so after its line
The direction of motion and movement locus type of the luminaire are determined according to the movement locus of acquisition afterwards.
It is preferred that what movement locus type provided in an embodiment of the present invention represented is the corresponding figure of movement locus, such as institute
The movement locus for stating luminaire is a circle, then movement locus type is circle.The direction of motion is expressed as from start bit
It is fortune to put the direction that starting position coordinates point to final position coordinate in the direction of final position, such as step S121~S122
Dynamic direction.
It is preferred that the corresponding movement locus of movement locus a in Fig. 2 in the movement locus schematic diagram with reference to shown in Fig. 2, Fig. 2
Type is straight line, the direction of motion be level to the right；The corresponding movement locus types of movement locus b are straight line, the direction of motion in Fig. 2
For level to the left；Movement locus type corresponding movement locus c is straight line, and the direction of motion is vertically upward；D pairs of movement locus
The movement locus type answered is straight line, and the direction of motion is vertically downward；Movement locus type corresponding movement locus e is circle
Shape, the direction of motion is counterclockwise；Movement locus type corresponding movement locus f is circle, and the direction of motion is clockwise；Need
Illustrate, the embodiment of the present invention can also increase movement locus newly, such as U-shaped movement locus, triangular motion track, the present invention
To this without limiting.
S14, the corresponding relation according to the movement locus type, the direction of motion and operational order of storage, are determined described luminous
The corresponding operational order of movement locus type, the direction of motion of equipment.
After the movement locus type and the direction of motion of the luminaire are determined according to step S13, you can according to pre-
The corresponding relation of the movement locus type, the direction of motion and the operational order that first store, determines the motion rail of the luminaire
The corresponding operational order of mark type, the direction of motion, wherein the movement locus type, the direction of motion and the operational order that prestore
Shown in corresponding relation reference table 1：
Table 1
| Movement locus type | The direction of motion | Operational order |
| Straight line | Level is to the right | Advancement commands |
| Straight line | Level is to the left | Retreat instruction |
| Straight line | Vertically upward | Turn left to instruct |
| Straight line | Vertically downward | Turn right and instruct |
| It is circular | Counterclockwise | Float and instruct |
| It is circular | Clockwise | Dive is instructed |
| It is U-shaped | Upward opening | Photographing instruction |
| …… | …… | …… |
When it is implemented, if the movement locus types determined of step S13 are straight line, and the direction of motion be level to
The right side, then can determine and expect robot advance pre-determined distance of paddling positioned at measurand under water；If it is determined that the hair
The movement locus type of light device is circle, and the direction of motion is clockwise, then can determine the measurand positioned under water
Expect robot dive instruction etc. of paddling.
It should be noted that the movement locus type, the direction of motion and the operational order that are stored in the embodiment of the present invention one
Corresponding relation is made an appointment, for example, staff presets movement locus type, the direction of motion and operational order
Corresponding relation, is then enter into the robot that paddles.In addition, staff also needs to inform measurand movement locus class
The corresponding relation of type, the direction of motion and operational order, thus measurand can control to paddle robot according to the corresponding relation
Perform the work that measurand needs.
It should be noted that movement locus of embodiment of the present invention type, the direction of motion can also correspond to other operational orders,
The present invention to this without limit, in addition, the U-shaped track such as set, the corresponding operational order of upward opening is photographing instructions.
When it is implemented, advance provided in an embodiment of the present invention or retreat, float or the operational order such as dive, its advance or
The distance of retrogressing, floating or dive is pre-set, i.e., advance according to the distance set or retreat or float or dive
Deng.Equally, it is provided in an embodiment of the present invention left-hand rotation or turn right angle be also what is pre-set, i.e., often move once, it is necessary to
Turned left or turned right according to angle set in advance.Wherein, the angle or the size of distance pre-set can be according to reality
Situation determines that the present invention is to this without limiting.
If it is preferred that the luminaire without moving, that is, it is a point to shoot obtained movement locus, that is, is moved
Type of gesture be a little, without motion direction, then can represent keep current operation or position it is constant.
It is preferred that in order to improve determine operational order precision, positioned at measurand under water drive luminaire by
When being moved according to desired guiding trajectory, it can repeat twice.
The operational order that S15, basis are determined, performs corresponding action.
When it is implemented, if the operational order determined in step S14 is advancement commands, the robot that paddles is according to pre-
If apart from forward movement.The luminaire carried by measurand, realizes the communication of measurand and the robot that paddles, and in fact
Now to the control for the robot that paddles so that the robot that paddles performs corresponding action.
It is preferred that can be performed by the image processor in the robot that paddles and master controller in the embodiment of the present invention one
Step S11~S15 process, described image processor can be DSP (at Digital Signal Process, data signal
Manage device), the master controller can be ARM microprocessor, such as STM32F427.
The control method for the robot that paddles that the embodiment of the present invention one is provided, multi collect and position is obtained in preset time
In after the multiple image for the luminaire that measurand under water is carried, the coordinate pre-established is utilized according to the multiple image
System, determines the coordinate value of luminaire described in each two field picture；The luminaire according to each two field picture determined
Coordinate value, determine the movement locus of the luminaire.After the movement locus of the luminaire is determined, according to described
Movement locus, determines the movement locus type and the direction of motion of the luminaire；And according to the movement locus prestored
The corresponding relation of type, the direction of motion and operational order, determines movement locus type, the reversing of motion correspondence of the luminaire
Operational order；And according to the operational order determined, perform corresponding action.Thus achieve without using wire cable
Under the conditions of, the direct communication of people and the robot that paddles, and pass through the control to the robot that paddles so that the robot that paddles being capable of root
The movement locus drawn out according to measurand using the luminaire, is performed corresponding action, not only meets people under water
The enjoyment of motion, can also command the robot that paddles to perform some detection operations, extend the use scope for the robot that paddles.
Embodiment two
For a better understanding of the present invention, the embodiment of the present invention two is performing step S13's on the basis of embodiment one
Before, the flow shown in Fig. 3 a is can also carry out, be may comprise steps of：
S31, according to the multiple image, determine the characteristic information of the luminaire.
It is preferred that the characteristic information between the luminaire includes following at least one：Glow frequency, the color of light and institute
State the shape of luminaire light-emitting window.
Specifically, the glow color can be red, green or yellow etc., but typically be compared when selecting using penetration power
Long light is used, i.e., used from the long luminaire of the wavelength of the light sent.The shape of the light sent can for it is circular,
Square or triangle etc..The present invention is to this without limiting.
When it is implemented, due to positioned at measurand more than one under water, therefore, in order to at least one measurand
Make a distinction, can be with glow frequency in method provided in an embodiment of the present invention, the shape of glow color and the light sent, or
Any combinations of these three features of person distinguishes luminaire.For example, two luminaires have different glow frequencies, or
With different glow colors, or shape with different light etc..In order to further improve the accurate of identification luminaire
Property, it can require that luminaire has fixed glow frequency and glow color the two features.When being identified, only this
Two signature verifications could be identified the identification to measurand and be passed through by rear.
It is preferred that the robot that paddles when determining the characteristic information of the luminaire, lights according to the multiple image
Color and the shape of light sent can easier be confirmed to come, but it is determined that during glow frequency, it is possible to use Nai Kuisi
Special sampling thheorem is determined, when being more than 2 times of highest frequency using frequency, and sampled signal can retain all of original signal
Information.Image frame per second as sample frequency, is carried out coding judgement, as shown in Figure 3 b, Rule of judgment is by the present invention to signal lamp：
Chang Liang duration is more than TLFor long signal, 1 is encoded to；Duration is less than or equal to TLFor short message or, it is encoded to 0.Thus lead to
Cross the glow frequency that different coding determines luminaire.
It is preferred that the luminaire that the embodiment of the present invention is used can also be breath light, its characteristic information is that breath light is bright
The slope of angle value change.
The characteristic information that S32, basis are determined, the identity to measurand is identified.
When it is implemented, staff again by luminaire light measurand when, the identity of measurand can be recorded, such as
The information such as ID card No., are illustrated by taking identification card number as an example, then staff can record ID card No. and luminaire
Corresponding relation, and the characteristic information of the luminaire recorded in advance；In addition, exemplified by be easy to record, due to luminous
Equipment has unit type, and the unit type of the luminaire can be directly recorded during record, or staff can be advance
Unique number is carried out to all luminaires, luminaire can also be represented with this, ID card No. is thereby establish, lighting sets
Corresponding relation between standby model (numbering) and the characteristic information three of luminaire.
Therefore, robot is paddled when measurand is identified, after the characteristic information of luminaire is determined,
Can according to the corresponding relation between the characteristic information three of ID card No., luminaire model (numbering) and luminaire,
The identity of measurand is determined, positioning is tracked to the measurand.
It is preferred that the embodiment of the present invention is provided when being tracked positioning to measurand, can be according to the side shown in Fig. 4
Method is implemented, and may comprise steps of：
S41, determine with positioned at the distance between measurand under water.
When it is implemented, the robot that paddles can utilize based on binocular vision system determine its with positioned at tested pair under water
The relative position relation of elephant, the relative position relation includes left and right, up and down and distant relationships, and especially by paddling, robot is set
The camera calibration put determines the inside and outside parameter of video camera, generally uses Zhang Zhengyou standardizations.Determining the figure of measurand
As after coordinate value, the corresponding world coordinates of image coordinate is determined based on binocular vision system.
After relative position relation is determined, it is determined that with positioned at the distance between measurand under water.
Whether the distance that S42, judgement are determined is in the range of pre-determined distance, if it is not, then performing step S43；It is no
Then, then step S44 is performed.
When it is implemented, the distance determined is judged whether in the range of pre-determined distance, if representing machine of paddling
Device people can understand the image for the luminaire for collecting measurand and its carrying, then keep current location constant；If no
In the range of pre-determined distance, in order to ensure the image clearly collected, it is necessary to be tracked to measurand, its movement side
To should ensure that it is mobile after with the distance between measurand within a preset range.
It is preferred that in order to further track measurand, when detecting measurand along a direction advance certain distance,
The robot that then paddles need to track measurand and accordingly advance certain distance.
S43, the measurand is tracked.
S44, holding current location are constant.
In order to be further understood that the tracking of the offer of the embodiment of the present invention two, methods described can be applied under water
There are the application scenarios of multiple measurands, for example, there is 3 measurands under water, respectively A, B and C, then the robot that paddles can be with
This 3 measurands are identified respectively for the method provided according to the embodiment of the present invention two, then determine and this 3 quilts respectively
The relative position of object is surveyed, when this 3 measurand movements, then the distance with measurand is redefined, to realize to quilt
Survey the track and localization of object.
The control method for the robot that paddles that the embodiment of the present invention two is provided, on the basis of embodiment one, utilizes collection
The multiple image arrived, determines the characteristic information of the luminaire；And according to the characteristic information determined, to tested pair
The identity of elephant is identified, and thus achieves the identification to measurand；Further, it is also possible to determine with being located at tested pair under water
The distance between as；If it is determined that the distance is not in the range of pre-determined distance, then the measurand is tracked, it is real
Now to the track and localization of measurand, the use scope for the robot that paddles is extended.
Embodiment three
Based on same inventive concept, a kind of control device for the robot that paddles is additionally provided in the embodiment of the present invention, due to
The principle that said apparatus solves problem is similar to the control method for the robot that paddles, therefore the implementation side of may refer to of said apparatus
The implementation of method, repeats part and repeats no more.
As shown in figure 5, the structural representation of the control device of the robot that paddles provided for the embodiment of the present invention three, bag
Include：Obtaining unit 51, the first determining unit 52, the second determining unit 53, the 3rd determining unit 54 and execution unit 55, wherein：
Obtaining unit 51, the multiframe for the luminaire that measurand under water is carried is located at for being obtained in preset time
Image；
First determining unit 52, for according to the multiple image, determining the movement locus of the luminaire；
Second determining unit 53, for according to the movement locus, determine the luminaire movement locus type and
The direction of motion；
3rd determining unit 54, for the movement locus type according to storage, direction of motion pass corresponding with operational order
System, determines the corresponding operational order of movement locus type, the direction of motion of the luminaire；
Execution unit 55, for according to the operational order determined, performing corresponding action.
When it is implemented, described device, in addition to：4th determining unit and recognition unit, wherein：
4th determining unit, in the 3rd determining unit 54 according to the movement locus type of storage, motion
Direction and the corresponding relation of operational order, determine that the corresponding operation of movement locus type, the direction of motion of the luminaire refers to
Before order, according to the multiple image, the characteristic information of the luminaire is determined；
The recognition unit, for the characteristic information determined according to the 4th determining unit, to the body of measurand
Part is identified.
It is preferred that the characteristic information includes following at least one：Glow frequency, the color of light and the luminaire go out
The shape of optical port.
When it is implemented, first determining unit 52, is specifically included：First determination subelement and second determines that son is single
Member, wherein：
First determination subelement, for according to the multiple image, using the coordinate system pre-established, is determined each
The coordinate value of luminaire described in two field picture；
Second determination subelement, for the coordinate value of the luminaire according to each two field picture determined,
Determine the movement locus of the luminaire.
It is preferred that described device, in addition to the 5th determining unit and tracking cell, wherein：
5th determining unit, for determining and positioned at the distance between measurand under water；
The tracking cell, for if it is determined that the distance is not in the range of pre-determined distance, then to described tested pair
As being tracked.
Preferably, the luminaire at least includes following one kind：Signal lamp and light deflection device；And described device, also
Including：
Opening unit, if be light deflection device for the luminaire, in the multi collect of obtaining unit 51
And obtain before being located at the multiple image for the luminaire that measurand under water is carried, detecting lamp is opened in advance, wherein described
The light deflection device that detecting lamp is used to trigger the measurand when being pointed to measurand under water and being detected is luminous.
For convenience of description, above each several part is divided by function describes respectively for each module (or unit).Certainly, exist
Implement the function of each module (or unit) can be realized in same or multiple softwares or hardware during the present invention.For example, this
The control device for the robot that paddles that inventive embodiments three are provided can be arranged in the robot that paddles, and by paddling, robot is completed
The IMAQ of the luminaire carried to measurand and the corresponding action of execution.
Example IV
The embodiment of the present application four provides a kind of nonvolatile computer storage media, the computer-readable storage medium storage
There are computer executable instructions, the computer executable instructions can perform the robot that paddles in above-mentioned any means embodiment
Control method.
Embodiment five
Fig. 6 be the embodiment of the present application five provide implementation paddle robot control method the robot that paddles hardware knot
Structure schematic diagram, as shown in fig. 6, this is paddled, robot includes：
In one or more processors 610 and memory 620, Fig. 6 by taking a processor 610 as an example.
Performing the robot that paddles of the control method for the robot that paddles can also include：Input unit 630 and output device
640。
Processor 610, memory 620, input unit 630 and output device 640 can pass through bus or other modes
In connection, Fig. 6 exemplified by being connected by bus.
Memory 620 is as a kind of non-volatile computer readable storage medium storing program for executing, available for storage non-volatile software journey
The control method pair of the robot that paddles in sequence, non-volatile computer executable program and module, such as the embodiment of the present application
Programmed instruction/module/the unit answered is (for example, obtaining unit 51, the first determining unit 52, the second determining unit shown in accompanying drawing 5
53rd, the 3rd determining unit 54 and execution unit 55).Processor 610 is stored in non-volatile soft in memory 620 by operation
Part program, instruction and module/unit, so that at execute server or the various function application and data of intelligent terminal
Reason, that is, realize that above method embodiment is paddled the control method of robot.
Memory 620 can include storing program area and storage data field, wherein, storing program area can store operation system
Application program required for system, at least one function；Storage data field can be stored to be made according to the control device for the robot that paddles
With data created etc..In addition, memory 620 can include high-speed random access memory, it can also include non-volatile
Memory, for example, at least one disk memory, flush memory device or other non-volatile solid state memory parts.In some realities
Apply in example, memory 620 is optional including the memory remotely located relative to processor 610, and these remote memories can lead to
Network connection is crossed to the control device for the robot that paddles.The example of above-mentioned network include but is not limited to internet, intranet,
LAN, mobile radio communication and combinations thereof.
Input unit 630 can receive the numeral or character information of input, and produce the control device with the robot that paddles
User set and function control it is relevant key signals input.Output device 640 may include the display devices such as display screen.
One or more of modules are stored in the memory 620, when by one or more of processors
During 610 execution, the control method of the robot that paddles in above-mentioned any means embodiment is performed.
The said goods can perform the method that the embodiment of the present application is provided, and possesses the corresponding functional module of execution method and has
Beneficial effect.Not ins and outs of detailed description in the present embodiment, reference can be made to the method that the embodiment of the present application is provided.
The electronic equipment of the embodiment of the present application exists in a variety of forms, includes but is not limited to:
(1) mobile communication equipment：The characteristics of this kind equipment is that possess mobile communication function, and to provide speech, data
Communicate as main target.This Terminal Type includes:Smart mobile phone (such as iPhone), multimedia handset, feature mobile phone, and it is low
Hold mobile phone etc..
(2) super mobile personal computer equipment：This kind equipment belongs to the category of personal computer, there is calculating and processing work(
Can, typically also possess mobile Internet access characteristic.This Terminal Type includes:PDA, MID and UMPC equipment etc., such as iPad.
(3) portable entertainment device：This kind equipment can show and play content of multimedia.The kind equipment includes:Audio,
Video player (such as iPod), handheld device, e-book, and intelligent toy and portable car-mounted navigation equipment.
(4) server：The equipment for providing the service of calculating, the composition of server is total including processor, hard disk, internal memory, system
Line etc., server is similar with general computer architecture, but is due to need to provide highly reliable service, therefore in processing energy
Require higher in terms of power, stability, reliability, security, scalability, manageability.
(5) other electronic installations with data interaction function.
Embodiment six
The embodiment of the present application six provides a kind of computer program product, wherein, the computer program product includes depositing
The computer program on non-transient computer readable storage medium storing program for executing is stored up, the computer program includes programmed instruction, wherein, when
When described program instruction is computer-executed, the computer is set to perform any one of the application above method embodiment machine of paddling
The control method of device people.
It should be understood by those skilled in the art that, embodiments of the invention can be provided as method, system or computer program
Product.Therefore, the present invention can be using the reality in terms of complete hardware embodiment, complete software embodiment or combination software and hardware
Apply the form of example.Moreover, the present invention can be used in one or more computers for wherein including computer usable program code
The computer program production that usable storage medium is implemented on (including but is not limited to magnetic disk storage, CD-ROM, optical memory etc.)
The form of product.
The present invention is the flow with reference to method according to embodiments of the present invention, equipment (system) and computer program product
Figure and/or block diagram are described.It should be understood that can be by every first-class in computer program instructions implementation process figure and/or block diagram
Journey and/or the flow in square frame and flow chart and/or block diagram and/or the combination of square frame.These computer programs can be provided
The processor of all-purpose computer, special-purpose computer, Embedded Processor or other programmable data processing devices is instructed to produce
A raw machine so that produced by the instruction of computer or the computing device of other programmable data processing devices for real
The device for the function of being specified in present one flow of flow chart or one square frame of multiple flows and/or block diagram or multiple square frames.
These computer program instructions, which may be alternatively stored in, can guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works so that the instruction being stored in the computer-readable memory, which is produced, to be included referring to
Make the manufacture of device, the command device realize in one flow of flow chart or multiple flows and/or one square frame of block diagram or
The function of being specified in multiple square frames.
These computer program instructions can be also loaded into computer or other programmable data processing devices so that in meter
Series of operation steps is performed on calculation machine or other programmable devices to produce computer implemented processing, thus in computer or
The instruction performed on other programmable devices is provided for realizing in one flow of flow chart or multiple flows and/or block diagram one
The step of function of being specified in individual square frame or multiple square frames.
, but those skilled in the art once know basic creation although preferred embodiments of the present invention have been described
Property concept, then can make other change and modification to these embodiments.So, appended claims are intended to be construed to include excellent
Select embodiment and fall into having altered and changing for the scope of the invention.
Obviously, those skilled in the art can carry out the essence of various changes and modification without departing from the present invention to the present invention
God and scope.So, if these modifications and variations of the present invention belong to the scope of the claims in the present invention and its equivalent technologies
Within, then the present invention is also intended to comprising including these changes and modification.
Claims (10)
1. a kind of control method for the robot that paddles, it is characterised in that including：
The multiple image for being located at the luminaire that measurand under water is carried is obtained in preset time；And
According to the multiple image, the movement locus of the luminaire is determined；
According to the movement locus, the movement locus type and the direction of motion of the luminaire are determined；And
According to the corresponding relation of the movement locus type, the direction of motion and operational order of storage, the fortune of the luminaire is determined
Dynamic type of gesture, the corresponding operational order of the direction of motion；And
According to the operational order determined, corresponding action is performed.
2. the method as described in claim 1, it is characterised in that in movement locus type, the direction of motion and the behaviour according to storage
Make before the corresponding relation instructed, the corresponding operational order of movement locus type, the direction of motion for determining the luminaire, go back
Including：
According to the multiple image, the characteristic information of the luminaire is determined；And
According to the characteristic information determined, the identity to measurand is identified.
3. method as claimed in claim 2, it is characterised in that the characteristic information includes following at least one：Glow frequency,
The shape of the color of light and the luminaire light-emitting window.
4. the method as described in claim 1, it is characterised in that according to the multiple image, determine the fortune of the luminaire
Dynamic rail mark, is specifically included：
According to the multiple image, using the coordinate system pre-established, the coordinate of luminaire described in each two field picture is determined
Value；
The coordinate value of luminaire, determines the movement locus of the luminaire according to each two field picture determined.
5. the method as described in Claims 1 to 4 any claim, it is characterised in that also include：
It is determined that with positioned at the distance between measurand under water；
If it is determined that the distance is not in the range of pre-determined distance, then the measurand is tracked.
6. the method as described in Claims 1 to 4 any claim, it is characterised in that the luminaire at least include with
Lower one kind：Signal lamp and light deflection device；And if when the luminaire is light deflection device, then in multi collect and obtain position
Before the multiple image for the luminaire that measurand under water is carried, in addition to：
Detecting lamp is opened in advance, wherein the detecting lamp is used for when being pointed to measurand under water and being detected, triggering institute
The light deflection device for stating measurand lights.
7. a kind of control device for the robot that paddles, it is characterised in that including：
Obtaining unit, the multiple image for the luminaire that measurand under water is carried is located at for being obtained in preset time；
First determining unit, for according to the multiple image, determining the movement locus of the luminaire；
Second determining unit, for according to the movement locus, determining movement locus type and the motion side of the luminaire
To；
3rd determining unit, for the corresponding relation of the movement locus type according to storage, the direction of motion and operational order, it is determined that
The corresponding operational order of movement locus type, the direction of motion of the luminaire；
Execution unit, for according to the operational order determined, performing corresponding action.
8. a kind of nonvolatile computer storage media, be stored with computer executable instructions, it is characterised in that the computer
Executable instruction is used to perform the method as described in claim 1 to 6 any claim.
The robot 9. one kind is paddled, including：
At least one processor；And
The memory being connected with least one described processor communication；Characterized in that,
The memory storage has can be by the instruction of at least one computing device, and the instruction is by described at least one
Manage device to perform, so that at least one described processor is able to carry out the method as described in claim 1 to 6 any claim.
10. a kind of computer program product, it is characterised in that the computer program product includes being stored in non-transient computer
Computer program on readable storage medium storing program for executing, the computer program includes programmed instruction, wherein, when described program instruction is counted
When calculation machine is performed, the computer is set to perform the method as described in claim 1 to 6 any claim.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710423940.5A CN107329470B (en) | 2017-06-07 | 2017-06-07 | Control method and device of wading robot and wading robot |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710423940.5A CN107329470B (en) | 2017-06-07 | 2017-06-07 | Control method and device of wading robot and wading robot |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107329470A true CN107329470A (en) | 2017-11-07 |
| CN107329470B CN107329470B (en) | 2021-06-29 |
Family
ID=60194547
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710423940.5A Expired - Fee Related CN107329470B (en) | 2017-06-07 | 2017-06-07 | Control method and device of wading robot and wading robot |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107329470B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108062091A (en) * | 2017-11-14 | 2018-05-22 | 北京臻迪科技股份有限公司 | Snorkel power compensation method, apparatus and the floating mobile body of floating mobile body |
| CN113285765A (en) * | 2021-07-20 | 2021-08-20 | 深之蓝海洋科技股份有限公司 | Underwater robot communication method, electronic equipment and underwater robot |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN202765273U (en) * | 2012-09-14 | 2013-03-06 | 董昊 | Remote operated vehicle (ROV) for underwater photography |
| US20140078050A1 (en) * | 2011-11-28 | 2014-03-20 | Tencent Technology (Shenzhen) Company Limited | Method and system for triggering and controlling human-computer interaction operating instructions |
| CN103777746A (en) * | 2012-10-23 | 2014-05-07 | 腾讯科技（深圳）有限公司 | Human-machine interactive method, terminal and system |
| CN104002942A (en) * | 2014-06-09 | 2014-08-27 | 北京理工大学 | Micro autonomous submersible |
| CN104267643A (en) * | 2014-09-27 | 2015-01-07 | 江苏华宏实业集团有限公司 | Target positioning recognition system of underwater robot |
| CN104317302A (en) * | 2014-09-28 | 2015-01-28 | 江苏华宏实业集团有限公司 | Intellectualized underwater robot system |
| CN106314733A (en) * | 2016-10-26 | 2017-01-11 | 朱苏学 | Unmanned underwater photographing robot |
| CN106774355A (en) * | 2016-12-14 | 2017-05-31 | 上海理工大学 | Unmanned remotely controlled submersible vehicle system based on moving camera |
-
2017
- 2017-06-07 CN CN201710423940.5A patent/CN107329470B/en not_active Expired - Fee Related
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140078050A1 (en) * | 2011-11-28 | 2014-03-20 | Tencent Technology (Shenzhen) Company Limited | Method and system for triggering and controlling human-computer interaction operating instructions |
| CN202765273U (en) * | 2012-09-14 | 2013-03-06 | 董昊 | Remote operated vehicle (ROV) for underwater photography |
| CN103777746A (en) * | 2012-10-23 | 2014-05-07 | 腾讯科技（深圳）有限公司 | Human-machine interactive method, terminal and system |
| CN104002942A (en) * | 2014-06-09 | 2014-08-27 | 北京理工大学 | Micro autonomous submersible |
| CN104267643A (en) * | 2014-09-27 | 2015-01-07 | 江苏华宏实业集团有限公司 | Target positioning recognition system of underwater robot |
| CN104317302A (en) * | 2014-09-28 | 2015-01-28 | 江苏华宏实业集团有限公司 | Intellectualized underwater robot system |
| CN106314733A (en) * | 2016-10-26 | 2017-01-11 | 朱苏学 | Unmanned underwater photographing robot |
| CN106774355A (en) * | 2016-12-14 | 2017-05-31 | 上海理工大学 | Unmanned remotely controlled submersible vehicle system based on moving camera |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108062091A (en) * | 2017-11-14 | 2018-05-22 | 北京臻迪科技股份有限公司 | Snorkel power compensation method, apparatus and the floating mobile body of floating mobile body |
| CN113285765A (en) * | 2021-07-20 | 2021-08-20 | 深之蓝海洋科技股份有限公司 | Underwater robot communication method, electronic equipment and underwater robot |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107329470B (en) | 2021-06-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107045844B (en) | A kind of landscape guide method based on augmented reality | |
| CN110132274A (en) | A kind of indoor orientation method, device, computer equipment and storage medium | |
| CN104335152B (en) | Navigation instruction is provided when equipment is in locking mode | |
| JP7168694B2 (en) | 3D special effect generation method, device and electronic device with human face | |
| CN109682392A (en) | Vision navigation method and system based on deeply study | |
| CN110262730A (en) | Edit methods, device, equipment and the storage medium of game virtual resource | |
| CN110163976A (en) | A kind of method, apparatus, terminal device and the storage medium of virtual scene conversion | |
| CN108702448A (en) | Unmanned plane image-pickup method and unmanned plane | |
| CN107615310A (en) | Message processing device | |
| CN105979133A (en) | Tracking shooting method, mobile terminal and system | |
| CN103279187B (en) | A kind of method and intelligent terminal for being used to build more scene virtual panoramic spaces | |
| CN105210117A (en) | Augmented reality (AR) capture & play | |
| CN109947886A (en) | Image processing method, device, electronic equipment and storage medium | |
| CN105549725A (en) | Three-dimensional scene interaction display device and method | |
| CN110202573A (en) | Full-automatic hand and eye calibrating, working face scaling method and device | |
| US20160117811A1 (en) | Method for generating a target trajectory of a camera embarked on a drone and corresponding system | |
| CN107329470A (en) | A kind of control method for the robot that paddles, device and the robot that paddles | |
| WO2018025511A1 (en) | Information processing device, method, and computer program | |
| CN110926334A (en) | Measuring method, measuring device, electronic device and storage medium | |
| CN109754421A (en) | A kind of vision calibration method, device and robot controller | |
| CN107656991A (en) | A kind of remote guide method, apparatus and system | |
| CN110140100B (en) | Three-dimensional augmented reality object user interface functionality | |
| CN106267843A (en) | A kind of Educational toy external member and ocean mould recognition methods thereof | |
| CN104184941A (en) | Monitoring system | |
| CN107562200A (en) | A kind of VR menu control methods and system based on outdoor scene stage property |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20210629 |