US20180232415A1 - Mobile device enabled robotic system - Google Patents
Mobile device enabled robotic system Download PDFInfo
- Publication number
- US20180232415A1 US20180232415A1 US15/951,750 US201815951750A US2018232415A1 US 20180232415 A1 US20180232415 A1 US 20180232415A1 US 201815951750 A US201815951750 A US 201815951750A US 2018232415 A1 US2018232415 A1 US 2018232415A1
- Authority
- US
- United States
- Prior art keywords
- mobile
- mobile device
- vehicle
- robot
- robotic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
- 238000004891 communication Methods 0.000 claims abstract description 34
- 230000006855 networking Effects 0.000 claims description 33
- 230000003993 interaction Effects 0.000 claims description 23
- 238000000034 method Methods 0.000 claims description 21
- 241000282412 Homo Species 0.000 claims description 12
- 230000004438 eyesight Effects 0.000 abstract description 6
- 238000010586 diagram Methods 0.000 description 8
- 238000013507 mapping Methods 0.000 description 5
- 230000001413 cellular effect Effects 0.000 description 4
- 230000004807 localization Effects 0.000 description 3
- 230000003997 social interaction Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 230000006399 behavior Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 230000004297 night vision Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000006641 stabilisation Effects 0.000 description 2
- 238000011105 stabilization Methods 0.000 description 2
- 241001465754 Metazoa Species 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000033001 locomotion Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 230000000087 stabilizing effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G06F17/30442—
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04B—TRANSMISSION
- H04B1/00—Details of transmission systems, not covered by a single one of groups H04B3/00 - H04B13/00; Details of transmission systems not characterised by the medium used for transmission
- H04B1/38—Transceivers, i.e. devices in which transmitter and receiver form a structural unit and in which at least one part is used for functions of transmitting and receiving
- H04B1/3827—Portable transceivers
- H04B1/3833—Hand-held transceivers
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/0011—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots associated with a remote control arrangement
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/217—Database tuning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2272—Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24564—Applying rules; Deductive queries
- G06F16/24565—Triggers; Constraints
-
- G06F17/30306—
-
- G06F17/30321—
-
- G06F17/30336—
-
- G06F17/30339—
-
- G06F17/3051—
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L51/00—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
- H04L51/52—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail for supporting social networking services
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M1/00—Substation equipment, e.g. for use by subscribers
- H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
- H04M1/724—User interfaces specially adapted for cordless or mobile telephones
- H04M1/72403—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
- H04M1/72409—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality by interfacing with external accessories
- H04M1/72415—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality by interfacing with external accessories for remote control of appliances
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
- H04W4/12—Messaging; Mailboxes; Announcements
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
- B64U10/00—Type of UAV
- B64U10/10—Rotorcrafts
- B64U10/13—Flying platforms
- B64U10/14—Flying platforms with four distinct rotor axes, e.g. quadcopters
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
- B64U2101/00—UAVs specially adapted for particular uses or applications
- B64U2101/30—UAVs specially adapted for particular uses or applications for imaging, photography or videography
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
- B64U2201/00—UAVs characterised by their flight controls
- B64U2201/20—Remote controls
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M1/00—Substation equipment, e.g. for use by subscribers
- H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
- H04M1/724—User interfaces specially adapted for cordless or mobile telephones
- H04M1/72403—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
- H04M1/72409—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality by interfacing with external accessories
- H04M1/72412—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality by interfacing with external accessories using two-way short-range wireless interfaces
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M2250/00—Details of telephonic subscriber devices
- H04M2250/10—Details of telephonic subscriber devices including a GPS signal receiver
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M2250/00—Details of telephonic subscriber devices
- H04M2250/12—Details of telephonic subscriber devices including a sensor for measuring a physical value, e.g. temperature or motion
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M2250/00—Details of telephonic subscriber devices
- H04M2250/52—Details of telephonic subscriber devices including functional features of a camera
Definitions
- the present disclosure generally relates to unmanned aerial vehicles (UAVs), and in particular to UAVs with sensors and mobile device/smartphone onboard as the center for computation, sensor data acquisition and communication.
- UAVs unmanned aerial vehicles
- UAVs unmanned aerial vehicles
- a true disruptive wave of innovation will be facilitated if mobile robotics system (including UAVs) can fully take advantage of what the mobile industry has to offer.
- the low cost of computation, hardware, and connectivity offered by the mobile industry will be truly beneficial for mobile robotic industry (UAVs included).
- the supporting resources and infrastructures, such as cellular infrastructures and GPS satellites are already available for UAVs to tap into. Therefore, there is an unmet need for the hardware and software integration of mobile device and UAVs and commercialization of the achievable functionalities.
- a system which includes a mobile device coupled to a mobile vehicle.
- the mobile device can be communicatively coupled to the mobile vehicle, and in yet another embodiment, the mobile device can be physically coupled to the mobile vehicle.
- the mobile device includes at least one sensor, at least one processor, and at least one communication means.
- the at least one communication means is capable of communicating and networking among robotic mobile vehicles and between robotic mobile vehicles and humans, using social networking and/or messaging services and accessing, storing, and retrieving information from the Internet.
- the mobile vehicle includes a chassis, at least one actuator, and a power source.
- the mobile device can further include at least one actuator and can further include a power source.
- the mobile device can also be configured to gain power from the power source of the mobile vehicle.
- the mobile device can also be configured to gain power from the power source from the mobile device itself.
- the mobile vehicle can be a robot, an unmanned aerial vehicle, a humanoid an unmanned land vehicle, an unmanned underwater vehicle, and/or an unmanned watercraft.
- the mobile device can be a mobile phone, a smartphone-type device, and/or a tablet.
- a system which includes at least one robot that is configured to communicate with at least one target.
- the system also includes at least one communication means, wherein the communication means is capable of communicating and networking among robotic mobile vehicles and between robotic mobile vehicles and humans, using human social networking and/or messaging services and accessing, storing, and retrieving information from the Internet and/or dedicated robot social networking and/or messaging services, which include intelligent software applications run on a mobile device.
- the mobile device is attached to a vehicle, and the mobile device is capable of decision making, autonomous navigation and social interactions among robots and between robot and human, which can be accomplished onboard and/or through cloud computing.
- the target can include a human and/or a robot.
- the robot can be an autonomous robot and/or a home appliance and/or an autonomous vehicle.
- a method of communication and networking among robotic mobile vehicles and between robotic mobile vehicles and humans using human social networking or dedicated robot social networking methods includes running an intelligent software application on a mobile device, wherein the mobile device is attached to a vehicle.
- the mobile device is capable of decision making, autonomous navigation and interactions among robots and between robot and human.
- the human social networking can include at least one human interacting with at least one robot to achieve a natural and intelligent interaction.
- the dedicated robot networking includes at least one robot interacting with at least one other robot to achieve networking interaction.
- a method of communication and networking among robotic mobile vehicles and between robotic mobile vehicles and humans using messaging services or dedicated robotic messaging services comprising running an intelligent software application on a mobile device, wherein the mobile device is attached to a vehicle, and wherein the mobile device is capable of decision making, autonomous navigation and interactions among robots and between robot and human.
- the messaging services can include at least one human interacting with at least one robot to achieve a natural and intelligent interaction.
- the dedicated robotic messaging services can include at least one robot interacting with at least one other robot to achieve networking interaction.
- FIG. 1 is a diagram showing an embodiment of the hardware architecture disclosed herein.
- FIG. 2 is a system diagram of one embodiment of the herein described system.
- FIG. 3 is a system diagram of an embodiment of the herein described mobile device.
- FIG. 4 is a system diagram of an embodiment of the herein described mobile vehicle.
- FIG. 5 is a system diagram of an embodiment of the herein described system.
- a low-cost unmanned aerial vehicle (UAV) equipped with sensor modules and mobile devices (for instance, a smartphone or tablet) running intelligent software for autonomous navigation, onboard computer vision, communication, and robotic social networks, is disclosed herein.
- the sensor modules include but are not limited to cameras, radar, LIDAR, ultrasonic range sensors, night vision cameras, stereovision systems, laser range finder, pressure sensors, altimeter to measure various environment information such as obstacle detection, distance to the obstacle, recognition of other objects in front and around the vehicle, point cloud map/3D mapping of surroundings, therefore provide possible sensing for functions of obstacle avoidance, boundary following, collision avoidance, and autonomous navigation.
- the mobile device/smartphone serves as the center of computation, sensor data acquisition, storage and analysis, connectivity through cellular network, RF communication (such as Bluetooth and Wi-Fi).
- the camera of the mobile device can work as the primary or one of the image acquisition sensors.
- the software running on the mobile device includes but not limited to navigation planning algorithms, image processing/computer vision algorithms, sensor data analysis, trajectory planning algorithms, connectivity to the cloud, and GUI.
- the software architecture includes three layers: a high level layer hybrid automata switching between behaviors such as robotic social networking, artificial intelligent planning (including for instance, reading in map and GPS and generating intermediary waypoints); a middle level layer hybrid automata generating trajectory and avoiding obstacle between waypoints; a low level layer trajectory tracking and flight stabilization feedback controller.
- the hardware architecture includes three parts: the unmanned aerial vehicle (with/without onboard low level stabilizing controller), variety of sensor modules attached to the unmanned aerial vehicle, and a mobile device including cellar connectivity, camera, GPS, CPU, storage, RF communication (Wi-Fi, Bluetooth), attached to the unmanned aerial vehicle.
- social networking refers to platforms and services designed for building online communities and communication and interaction capabilities. Most social networks are web-based and provide ways for users to interact and communicate.
- messaging services means textual and/or pictorial and/or visual communication between users.
- Each message has an originator and at least one recipient.
- the originator can be a human and/or a social network, and/or a robot.
- the users and the at least one recipient can similarly be humans and/or social networks, and/or robots.
- a robot refers to a reprogrammable, multifunction manipulator designed to move materials, which can include, parts, tools, humans, living creatures, and/or specialized devices. This can be accomplished through variable programmed motions for the performance of a variety of tasks.
- a mobile device such as a smartphone
- the robot such as UAV
- wireless (or wired) sensor modules The sensors are communicatively coupled to the mobile device(s) and/or UAV's electronics. The sensor readings are collected and analyzed using algorithms, which compute useful information for the navigation system. The mobile device sends out commands to drive the UAV.
- a smartphone is presented as an embodiment, a mobile device as used herein can be a smartphone-type device, a tablet, and similar devices that exhibit communication, sensing and computing power similar to a smartphone-type device.
- the hardware architecture is disclosed herein.
- a plurality of sensor modules 103 ( FIG. 1 ) is placed in predetermined locations on the vehicle.
- the mobile device 102 (including for example, a smartphone) is physically attached onto the body of the UAV.
- the mobile device serves as the center of computation with its CPU, memory and data storage. It also provides sensor data acquisition for its own sensors (such as GPS, IMU, temperature and Camera) and outside sensor modules 103 .
- Sensor modules 103 can include but are not limited to cameras, radar, LIDAR, ultrasonic range sensors, night vision cameras, stereovision systems, laser range finder, pressure sensors, and altimeters etc.
- the navigation system 115 as a software bundle running on the mobile phone takes in the sensor readings and generates command to drive UAV 101 .
- the phone camera 117 records the video of the surroundings and feeds the data to either data storage 118 or upload through cellular network 113 to online storage or online website, or to the computer vision algorithms 116 .
- the computer vision algorithms 116 provide analysis of the visional data and generate information to be used by navigation system 115 .
- a graphical user interface (GUI) 119 is also provided through the phone's touch screen for possible interaction with operator or people.
- the robot social network 120 provides the arena of communication among group of UAVs or between human and UAVs.
- FIG. 3 a system diagram for an embodiment of the mobile device 102 is shown. Still referring to FIG. 3 , sensor modules 103 are coupled to a processor 151 , which is further coupled to a communication means 161 .
- the mobile device 102 can also optionally include at least one actuator 181 .
- a power source 171 supplies power to the mobile device 102 .
- the mobile device 102 can be any one of or a combination of a smartphone, a tablet, or a personal digital assistant.
- the power source 171 can be any one of or a combination of a battery, solar power, or even mechanical power.
- the mobile vehicle 202 includes a sensor module 104 , which is coupled to a chassis 191 .
- the mobile vehicle 202 can also include at least one actuator 172 .
- the mobile vehicle 202 can also optionally include a processor 182 .
- the mobile device 202 also includes a power source 152 .
- the mobile vehicle 202 can be any one of or a combination of a robot, a humanoid, a car, a UAV, or even an underwater vehicle.
- the mobile device 102 and the mobile vehicle 202 can be physically and/or communicatively coupled.
- the power source 152 can be any one of or a combination of a battery, solar power, or even mechanical power.
- the software architecture (not shown) for the navigation system 115 can include three layers: a high level layer hybrid automata switching between behaviors such as task planning, artificial intelligent planning (including for example, reading in map and GPS and generating intermediary waypoints); a middle level layer hybrid automata generating trajectory and avoiding obstacle between waypoints; a low level layer trajectory tracking and flight stabilization feedback controller.
- the mobile device 102 for sensing, in one embodiment, for widely available computation power, the mobile device 102 , such as a smartphone-type device or a tablet device, is the center for higher-level image processing and/or computer vision algorithms and/or sensor fusion algorithms.
- robust ultrasonic range sensors may be used to achieve obstacle and collision avoidance.
- LIDAR is used to make high-resolution maps.
- the map of for the navigation system 116 is downloaded from existing Map services such as Google Map® and TomTom® for real-time navigation and traffic information.
- the map is constructed along the way with onboard sensor and mapping algorithms.
- Bluetooth and/or Wi-Fi and/or cellular are used for localization for the position or relative position of the UAVs. Thus formation control of UAVs is realized.
- SLAM simultaneous localization and mapping
- SLAM is used to acquire the mapping and system location without using GPS and map services.
- the robot social network is used, where either existing social network such as Facebook®, Twitter® and any other human online social networking methods are adopted.
- a human user can send command or to-do-list for the robot via online messaging, status posting, and/or hash tagging or any other existing methods that are used by human on the online social networks.
- the robot can do social updates using similar existing methods to messaging or update the current status via words, photos and/or videos.
- the communications between robot and human are thought the existing methods of text message and phone call between mobile phones.
- the robot social network is used, where either existing social network such as Facebook®, Twitter® and any other human online social networking methods are adopted.
- robots can send command or to-do-list for the other robot via online messaging, status posting, and/or hash tagging or any other existing methods that are used by human on the online social networks.
- the robot can do social updates using similar existing methods to messaging or update the current status via words, photos and/or videos.
- the communications among robots are through the existing methods of text message and phone call between mobile phones.
- the communications among robots are using Bluetooth networks.
- the communications among robots are using Wi-Fi networks.
- a system which includes a mobile device coupled to a mobile vehicle.
- the mobile device can be communicatively coupled to the mobile vehicle, and in yet another embodiment, the mobile device can be physically coupled to the mobile vehicle.
- the mobile device includes at least one sensor, at least one processor, and at least one communication means.
- the at least one processor is configured to make operational decisions through local or cloud-based computing.
- the at least one communication means is capable of communicating and networking among robotic mobile vehicles and between robotic mobile vehicles and humans, using social networking and/or messaging services and accessing, storing, and retrieving information from the Internet.
- the mobile vehicle includes a chassis, at least one actuator, and a power source.
- the mobile device can further include at least one actuator and can further include a power source.
- the mobile device can also be configured to gain power from the power source of the mobile vehicle.
- the mobile device can also be configured to gain power from the power source from the mobile device itself.
- the mobile vehicle can be a robot, an unmanned aerial vehicle, a humanoid an unmanned land vehicle, an unmanned underwater vehicle, and/or an unmanned watercraft.
- the mobile device can be a mobile phone, a smartphone-type device, and/or a tablet.
- a system which includes at least one robot that is configured to communicate with at least one target.
- the system also includes at least one communication means, wherein the communication means is capable of communicating and networking among robotic mobile vehicles and between robotic mobile vehicles and humans, using human social networking and/or messaging services and accessing, storing, and retrieving information from the Internet and/or dedicated robot social networking and/or messaging services, which include intelligent software applications run on a mobile device.
- the mobile device is attached to a vehicle, and the mobile device is capable of decision making, autonomous navigation and social interactions among robots and between robot and human, which can be accomplished onboard and/or through cloud computing.
- the target can include a human and/or a robot.
- the robot can be an autonomous robot and/or a home appliance and/or an autonomous vehicle.
- a method of communication and networking among robotic mobile vehicles and between robotic mobile vehicles and humans using human social networking or dedicated robot social networking methods includes running an intelligent software application on a mobile device, wherein the mobile device is attached to a vehicle.
- the mobile device is capable of decision making, autonomous navigation and interactions among robots and between robot and human.
- the human social networking can include at least one human interacting with at least one robot to achieve a natural and intelligent interaction.
- the dedicated robot networking includes at least one robot interacting with at least one other robot to achieve networking interaction.
- a method of communication and networking among robotic mobile vehicles and between robotic mobile vehicles and humans using messaging services or dedicated robotic messaging services comprising running an intelligent software application on a mobile device, wherein the mobile device is attached to a vehicle, and wherein the mobile device is capable of decision making, autonomous navigation and interactions among robots and between robot and human.
- the messaging services can include at least one human interacting with at least one robot to achieve a natural and intelligent interaction.
- the dedicated robotic messaging services can include at least one robot interacting with at least one other robot to achieve networking interaction.
- ground and aerial vehicles are disclosed as embodiments, these ground vehicles and aerial vehicles are not limited to inanimate objects. Animate users (including animals and humans) may be the vehicles as well.
- mobile robots such use is not intended to be limiting and rather, the robots can include stationary robots as well.
- a human user may need to accomplish household chores, for instance, grocery shopping.
- the user can provide instructions to a mobile device, for example “pick up groceries” and will list the groceries needed.
- the mobile device sends the instructions (which can include using natural language) to a robot (which can be either autonomous or human-driven), such as a car.
- the car can then be alerted that groceries need to be picked up and can act on those instructions.
- Other examples include laundry and house cleaning.
- messaging services are defined to include email and instant messaging, and short messaging services.
- Another example of the herein described systems and methods involves robots interacting with other robots from human instructions.
- a task such as the lawn on a human user's house needs to be mowed
- the instructions can be sent to a robot, which can then interact with other robots using a social media network to optimize accomplishing acting on the instructions to finish the task.
- the robots and the human user can all then be kept updated through a social media network on the progress of the task.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Automation & Control Theory (AREA)
- Human Computer Interaction (AREA)
- Computing Systems (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
- Telephone Function (AREA)
Abstract
An unmanned aerial vehicle (UAV) equipped with sensor modules and mobile devices (including smartphone, tablet) running intelligent software for autonomous navigation, onboard computer vision, communication, and robotic social networks, is disclosed herein.
Description
- The present U.S. patent application is a continuation of U.S. patent application Ser. No. 14/752,620, filed Nov. 25, 2015, which is related to and claims the priority benefit of U.S. Provisional Patent Application Ser. No. 62/085,329, filed Nov. 27, 2014, the contents of which is hereby incorporated by reference in its entirety into this disclosure.
- The present disclosure generally relates to unmanned aerial vehicles (UAVs), and in particular to UAVs with sensors and mobile device/smartphone onboard as the center for computation, sensor data acquisition and communication.
- This section introduces aspects that may help facilitate a better understanding of the disclosure. Accordingly, these statements are to be read in this light and are not to be understood as admissions about what is or is not prior art.
- Recently, owning to the technical advances in robotics, unmanned aerial vehicles (UAVs) are increasingly popular in many application domains. While UAVs/drones are currently successfully deployed for military applications, their adoption and availability in commercial and civilian domains are currently hindered by its high cost and limited functionalities. The last few years have seen successes of commercial applications emerging in aerial video photography, toy/game and monitoring/surveillances. Functionalities of those systems, however, are still limited to remote control by human and simple point-to-point navigation, and the cost is too high as well. In the meantime, the mobile industry is one of the most rapidly-developing industries, which brings down the cost of portable mobile devices with increasing functionalities. A true disruptive wave of innovation will be facilitated if mobile robotics system (including UAVs) can fully take advantage of what the mobile industry has to offer. The low cost of computation, hardware, and connectivity offered by the mobile industry will be truly beneficial for mobile robotic industry (UAVs included). Moreover, the supporting resources and infrastructures, such as cellular infrastructures and GPS satellites are already available for UAVs to tap into. Therefore, there is an unmet need for the hardware and software integration of mobile device and UAVs and commercialization of the achievable functionalities.
- In one aspect, a system is disclosed, which includes a mobile device coupled to a mobile vehicle. The mobile device can be communicatively coupled to the mobile vehicle, and in yet another embodiment, the mobile device can be physically coupled to the mobile vehicle. The mobile device includes at least one sensor, at least one processor, and at least one communication means. The at least one communication means is capable of communicating and networking among robotic mobile vehicles and between robotic mobile vehicles and humans, using social networking and/or messaging services and accessing, storing, and retrieving information from the Internet. The mobile vehicle includes a chassis, at least one actuator, and a power source. The mobile device can further include at least one actuator and can further include a power source. The mobile device can also be configured to gain power from the power source of the mobile vehicle. The mobile device can also be configured to gain power from the power source from the mobile device itself.
- In another aspect, the mobile vehicle can be a robot, an unmanned aerial vehicle, a humanoid an unmanned land vehicle, an unmanned underwater vehicle, and/or an unmanned watercraft. The mobile device can be a mobile phone, a smartphone-type device, and/or a tablet.
- In yet another aspect, a system is presented which includes at least one robot that is configured to communicate with at least one target. The system also includes at least one communication means, wherein the communication means is capable of communicating and networking among robotic mobile vehicles and between robotic mobile vehicles and humans, using human social networking and/or messaging services and accessing, storing, and retrieving information from the Internet and/or dedicated robot social networking and/or messaging services, which include intelligent software applications run on a mobile device. The mobile device is attached to a vehicle, and the mobile device is capable of decision making, autonomous navigation and social interactions among robots and between robot and human, which can be accomplished onboard and/or through cloud computing. The target can include a human and/or a robot. The robot can be an autonomous robot and/or a home appliance and/or an autonomous vehicle.
- In yet another aspect, a method of communication and networking among robotic mobile vehicles and between robotic mobile vehicles and humans using human social networking or dedicated robot social networking methods is presented. The method includes running an intelligent software application on a mobile device, wherein the mobile device is attached to a vehicle. The mobile device is capable of decision making, autonomous navigation and interactions among robots and between robot and human. The human social networking can include at least one human interacting with at least one robot to achieve a natural and intelligent interaction. The dedicated robot networking includes at least one robot interacting with at least one other robot to achieve networking interaction.
- In yet another aspect, a method of communication and networking among robotic mobile vehicles and between robotic mobile vehicles and humans using messaging services or dedicated robotic messaging services, comprising running an intelligent software application on a mobile device, wherein the mobile device is attached to a vehicle, and wherein the mobile device is capable of decision making, autonomous navigation and interactions among robots and between robot and human. The messaging services can include at least one human interacting with at least one robot to achieve a natural and intelligent interaction. The dedicated robotic messaging services can include at least one robot interacting with at least one other robot to achieve networking interaction.
-
FIG. 1 is a diagram showing an embodiment of the hardware architecture disclosed herein. -
FIG. 2 is a system diagram of one embodiment of the herein described system. -
FIG. 3 is a system diagram of an embodiment of the herein described mobile device. -
FIG. 4 is a system diagram of an embodiment of the herein described mobile vehicle. -
FIG. 5 is a system diagram of an embodiment of the herein described system. - For the purposes of promoting an understanding of the principles of the present disclosure, reference will now be made to the embodiments illustrated in the drawings, and specific language will be used to describe the same. It will nevertheless be understood that no limitation of the scope of this disclosure is thereby intended.
- A low-cost unmanned aerial vehicle (UAV) equipped with sensor modules and mobile devices (for instance, a smartphone or tablet) running intelligent software for autonomous navigation, onboard computer vision, communication, and robotic social networks, is disclosed herein. The sensor modules include but are not limited to cameras, radar, LIDAR, ultrasonic range sensors, night vision cameras, stereovision systems, laser range finder, pressure sensors, altimeter to measure various environment information such as obstacle detection, distance to the obstacle, recognition of other objects in front and around the vehicle, point cloud map/3D mapping of surroundings, therefore provide possible sensing for functions of obstacle avoidance, boundary following, collision avoidance, and autonomous navigation. The mobile device/smartphone serves as the center of computation, sensor data acquisition, storage and analysis, connectivity through cellular network, RF communication (such as Bluetooth and Wi-Fi). The camera of the mobile device can work as the primary or one of the image acquisition sensors. The software running on the mobile device includes but not limited to navigation planning algorithms, image processing/computer vision algorithms, sensor data analysis, trajectory planning algorithms, connectivity to the cloud, and GUI. The software architecture includes three layers: a high level layer hybrid automata switching between behaviors such as robotic social networking, artificial intelligent planning (including for instance, reading in map and GPS and generating intermediary waypoints); a middle level layer hybrid automata generating trajectory and avoiding obstacle between waypoints; a low level layer trajectory tracking and flight stabilization feedback controller. The hardware architecture includes three parts: the unmanned aerial vehicle (with/without onboard low level stabilizing controller), variety of sensor modules attached to the unmanned aerial vehicle, and a mobile device including cellar connectivity, camera, GPS, CPU, storage, RF communication (Wi-Fi, Bluetooth), attached to the unmanned aerial vehicle. For purposes of the present disclosure, social networking refers to platforms and services designed for building online communities and communication and interaction capabilities. Most social networks are web-based and provide ways for users to interact and communicate. In addition, for the purposes of the present disclosure, messaging services means textual and/or pictorial and/or visual communication between users. Each message has an originator and at least one recipient. The originator can be a human and/or a social network, and/or a robot. The users and the at least one recipient can similarly be humans and/or social networks, and/or robots. Further, for purposes of the present disclosure, a robot refers to a reprogrammable, multifunction manipulator designed to move materials, which can include, parts, tools, humans, living creatures, and/or specialized devices. This can be accomplished through variable programmed motions for the performance of a variety of tasks.
- Hardware of the novel system featuring the mobile device(s) (such as a smartphone), the robot (such as UAV), and wireless (or wired) sensor modules, is presented here. The sensors are communicatively coupled to the mobile device(s) and/or UAV's electronics. The sensor readings are collected and analyzed using algorithms, which compute useful information for the navigation system. The mobile device sends out commands to drive the UAV. It should be appreciated that although a smartphone is presented as an embodiment, a mobile device as used herein can be a smartphone-type device, a tablet, and similar devices that exhibit communication, sensing and computing power similar to a smartphone-type device.
- The hardware architecture is disclosed herein. In one embodiment, referring to
FIG. 1 , a plurality of sensor modules 103 (FIG. 1 ) is placed in predetermined locations on the vehicle. The mobile device 102 (including for example, a smartphone) is physically attached onto the body of the UAV. - Referring to
FIG. 2 for the system diagram, in one embodiment, the mobile device (including for example, a smartphone) serves as the center of computation with its CPU, memory and data storage. It also provides sensor data acquisition for its own sensors (such as GPS, IMU, temperature and Camera) andoutside sensor modules 103.Sensor modules 103 can include but are not limited to cameras, radar, LIDAR, ultrasonic range sensors, night vision cameras, stereovision systems, laser range finder, pressure sensors, and altimeters etc. Thenavigation system 115 as a software bundle running on the mobile phone takes in the sensor readings and generates command to driveUAV 101. Thephone camera 117 records the video of the surroundings and feeds the data to eitherdata storage 118 or upload throughcellular network 113 to online storage or online website, or to thecomputer vision algorithms 116. Thecomputer vision algorithms 116 provide analysis of the visional data and generate information to be used bynavigation system 115. A graphical user interface (GUI) 119 is also provided through the phone's touch screen for possible interaction with operator or people. The robotsocial network 120 provides the arena of communication among group of UAVs or between human and UAVs. - Referring to
FIG. 3 , a system diagram for an embodiment of themobile device 102 is shown. Still referring toFIG. 3 ,sensor modules 103 are coupled to aprocessor 151, which is further coupled to a communication means 161. Themobile device 102 can also optionally include at least oneactuator 181. Apower source 171 supplies power to themobile device 102. Themobile device 102 can be any one of or a combination of a smartphone, a tablet, or a personal digital assistant. In some aspects thepower source 171 can be any one of or a combination of a battery, solar power, or even mechanical power. - Referring to
FIG. 4 , a system diagram for an embodiment of themobile vehicle 202 is shown. Still referring toFIG. 4 , themobile vehicle 202 includes asensor module 104, which is coupled to achassis 191. Themobile vehicle 202 can also include at least oneactuator 172. Themobile vehicle 202 can also optionally include aprocessor 182. Themobile device 202 also includes apower source 152. Themobile vehicle 202 can be any one of or a combination of a robot, a humanoid, a car, a UAV, or even an underwater vehicle. Referring toFIG. 5 , in one embodiment, themobile device 102 and themobile vehicle 202 can be physically and/or communicatively coupled. In some aspects thepower source 152 can be any one of or a combination of a battery, solar power, or even mechanical power. - Referring to
FIG. 2 , the software architecture (not shown) for thenavigation system 115 can include three layers: a high level layer hybrid automata switching between behaviors such as task planning, artificial intelligent planning (including for example, reading in map and GPS and generating intermediary waypoints); a middle level layer hybrid automata generating trajectory and avoiding obstacle between waypoints; a low level layer trajectory tracking and flight stabilization feedback controller. - For sensing, in one embodiment, for widely available computation power, the
mobile device 102, such as a smartphone-type device or a tablet device, is the center for higher-level image processing and/or computer vision algorithms and/or sensor fusion algorithms. In yet another embodiment, robust ultrasonic range sensors may be used to achieve obstacle and collision avoidance. In yet another embodiment, LIDAR is used to make high-resolution maps. - For system mapping and localization, in one embodiment, the map of for the
navigation system 116 is downloaded from existing Map services such as Google Map® and TomTom® for real-time navigation and traffic information. In yet another embodiment, the map is constructed along the way with onboard sensor and mapping algorithms. In yet another embodiment, Bluetooth and/or Wi-Fi and/or cellular are used for localization for the position or relative position of the UAVs. Thus formation control of UAVs is realized. In yet another embodiment, SLAM (simultaneous localization and mapping) is used to acquire the mapping and system location without using GPS and map services. - For communication between human and robots, in one embodiment, the robot social network is used, where either existing social network such as Facebook®, Twitter® and any other human online social networking methods are adopted. Thus in this embodiment, a human user can send command or to-do-list for the robot via online messaging, status posting, and/or hash tagging or any other existing methods that are used by human on the online social networks. The robot can do social updates using similar existing methods to messaging or update the current status via words, photos and/or videos. In another embodiment, the communications between robot and human are thought the existing methods of text message and phone call between mobile phones.
- For communication and networking between robots, in one embodiment, the robot social network is used, where either existing social network such as Facebook®, Twitter® and any other human online social networking methods are adopted. Thus in this embodiment, robots can send command or to-do-list for the other robot via online messaging, status posting, and/or hash tagging or any other existing methods that are used by human on the online social networks. The robot can do social updates using similar existing methods to messaging or update the current status via words, photos and/or videos. In another embodiment, the communications among robots are through the existing methods of text message and phone call between mobile phones. In yet another embodiment, the communications among robots are using Bluetooth networks. In yet another embodiment, the communications among robots are using Wi-Fi networks.
- In yet another embodiment, a system is disclosed, which includes a mobile device coupled to a mobile vehicle. The mobile device can be communicatively coupled to the mobile vehicle, and in yet another embodiment, the mobile device can be physically coupled to the mobile vehicle. The mobile device includes at least one sensor, at least one processor, and at least one communication means. The at least one processor is configured to make operational decisions through local or cloud-based computing. The at least one communication means is capable of communicating and networking among robotic mobile vehicles and between robotic mobile vehicles and humans, using social networking and/or messaging services and accessing, storing, and retrieving information from the Internet. The mobile vehicle includes a chassis, at least one actuator, and a power source. The mobile device can further include at least one actuator and can further include a power source. The mobile device can also be configured to gain power from the power source of the mobile vehicle. The mobile device can also be configured to gain power from the power source from the mobile device itself.
- In yet another embodiment, the mobile vehicle can be a robot, an unmanned aerial vehicle, a humanoid an unmanned land vehicle, an unmanned underwater vehicle, and/or an unmanned watercraft. The mobile device can be a mobile phone, a smartphone-type device, and/or a tablet.
- In yet another embodiment, a system is presented which includes at least one robot that is configured to communicate with at least one target. The system also includes at least one communication means, wherein the communication means is capable of communicating and networking among robotic mobile vehicles and between robotic mobile vehicles and humans, using human social networking and/or messaging services and accessing, storing, and retrieving information from the Internet and/or dedicated robot social networking and/or messaging services, which include intelligent software applications run on a mobile device. The mobile device is attached to a vehicle, and the mobile device is capable of decision making, autonomous navigation and social interactions among robots and between robot and human, which can be accomplished onboard and/or through cloud computing. The target can include a human and/or a robot. The robot can be an autonomous robot and/or a home appliance and/or an autonomous vehicle.
- In yet another embodiment, a method of communication and networking among robotic mobile vehicles and between robotic mobile vehicles and humans using human social networking or dedicated robot social networking methods is presented. The method includes running an intelligent software application on a mobile device, wherein the mobile device is attached to a vehicle. The mobile device is capable of decision making, autonomous navigation and interactions among robots and between robot and human. The human social networking can include at least one human interacting with at least one robot to achieve a natural and intelligent interaction. The dedicated robot networking includes at least one robot interacting with at least one other robot to achieve networking interaction.
- In yet another embodiment, a method of communication and networking among robotic mobile vehicles and between robotic mobile vehicles and humans using messaging services or dedicated robotic messaging services, comprising running an intelligent software application on a mobile device, wherein the mobile device is attached to a vehicle, and wherein the mobile device is capable of decision making, autonomous navigation and interactions among robots and between robot and human. The messaging services can include at least one human interacting with at least one robot to achieve a natural and intelligent interaction. The dedicated robotic messaging services can include at least one robot interacting with at least one other robot to achieve networking interaction.
- The herein disclosed systems and methods can be easily integrated into any existing vehicle (including ground vehicles and aerial vehicles) and will thereby permit low-cost transformation of existing mobile robotics system (including UAVs) into an intelligent system with aforementioned advanced functionalities. It should be appreciated that although ground and aerial vehicles are disclosed as embodiments, these ground vehicles and aerial vehicles are not limited to inanimate objects. Animate users (including animals and humans) may be the vehicles as well. In addition, although referenced is made herein to “mobile” robots, such use is not intended to be limiting and rather, the robots can include stationary robots as well.
- As an example of the herein described systems and methods, a human user may need to accomplish household chores, for instance, grocery shopping. Using the herein described system and method, the user can provide instructions to a mobile device, for example “pick up groceries” and will list the groceries needed. The mobile device sends the instructions (which can include using natural language) to a robot (which can be either autonomous or human-driven), such as a car. The car can then be alerted that groceries need to be picked up and can act on those instructions. Other examples include laundry and house cleaning.
- There are at least two ways in which such interactions can occur: through messaging services and/or through social interactions. These messaging services are defined to include email and instant messaging, and short messaging services.
- Another example of the herein described systems and methods involves robots interacting with other robots from human instructions. As an example, if a task, such as the lawn on a human user's house needs to be mowed, the instructions can be sent to a robot, which can then interact with other robots using a social media network to optimize accomplishing acting on the instructions to finish the task. The robots and the human user can all then be kept updated through a social media network on the progress of the task. There are at least two ways in which such interactions can occur: through dedicated robotic social networking and/or messaging services, and/or human social media and messaging service interactions. These messaging services are defined to include email and instant messaging, and short messaging services.
- Those skilled in the art will recognize that numerous modifications can be made to the specific implementations described above. The implementations should not be limited to the particular limitations described. Other implementations may be possible.
Claims (20)
1. A system, comprising:
a mobile device, wherein the mobile device comprises:
at least one communication means, wherein the communication means is capable of communicating among robotic mobile vehicles and between robotic mobile vehicles and humans using a messaging service, wherein the at least one communication means is configured to access, store, and retrieve information from the Internet; and
a mobile vehicle, wherein the mobile vehicle is coupled to the mobile device, and the mobile vehicle comprises:
a chassis;
at least one actuator; and
a power source.
2. The system of claim 1 , wherein the mobile device further comprises at least one actuator.
3. The system of claim 1 , wherein the mobile device further comprises a power source.
4. The system of claim 1 , wherein the mobile device is configured to gain power from the power source of the mobile vehicle.
5. The system of claim 1 , wherein the mobile device is configured to gain power from the power source of the mobile device.
6. The system of claim 1 , wherein the mobile vehicle is a robot.
7. The system of claim 1 , wherein the mobile vehicle is an unmanned aerial vehicle.
8. The system of claim 1 , wherein the mobile vehicle is a humanoid.
9. The system of claim 1 , wherein the mobile vehicle is an unmanned land vehicle.
10. The system of claim 1 , wherein the mobile vehicle is an unmanned underwater vehicle.
11. The system of claim 1 , wherein the mobile vehicle is an unmanned watercraft.
12. The system of claim 1 , wherein the mobile device is a mobile phone.
13. The system of claim 1 , wherein the mobile device is a tablet.
14. The system of claim 1 , wherein the mobile device is communicatively coupled to the mobile vehicle.
15. A system, comprising:
at least one robot, wherein the at least one robot is configured to communicate with at least one target;
at least one communication means, wherein the communication means is capable of communicating among robotic mobile vehicles and between robotic mobile vehicles using a messaging service, wherein the at least one communication means is configured to access, store, and retrieve information from at least one of the Internet, a dedicated robot social networking, or a messaging service, wherein the mobile device is coupled to a vehicle, and wherein the mobile device is capable of decision making, autonomous navigation, interactions among robots, and interactions between a robot and a human; and wherein the vehicle comprises an actuator.
16. The system of claim 15 , wherein the at least one target comprises a human.
17. The system of claim 15 , wherein the at least one target comprises a robot.
18. The system of claim 17 , wherein the robot is an autonomous robot.
19. The system of claim 18 , wherein the autonomous robot is a home appliance and/or an autonomous vehicle.
20. A method of communicating among robotic mobile vehicles and between robotic mobile vehicles using a social network comprising: running an intelligent software application on a mobile device, wherein the mobile device is coupled to a vehicle, and wherein the mobile device is capable of decision making, autonomous navigation, interactions among robots, and interactions between robot and human, wherein the vehicle comprises an actuator.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/951,750 US20200097584A9 (en) | 2014-11-27 | 2018-04-12 | Mobile device enabled robotic system |
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201462085329P | 2014-11-27 | 2014-11-27 | |
US14/952,620 US9954992B2 (en) | 2014-11-27 | 2015-11-25 | Mobile device enabled robotic system |
US15/951,750 US20200097584A9 (en) | 2014-11-27 | 2018-04-12 | Mobile device enabled robotic system |
Related Parent Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/752,620 Continuation US20160378822A1 (en) | 2015-06-26 | 2015-06-26 | Automated recommendation and creation of database index |
US14/952,620 Continuation US9954992B2 (en) | 2014-11-27 | 2015-11-25 | Mobile device enabled robotic system |
Publications (2)
Publication Number | Publication Date |
---|---|
US20180232415A1 true US20180232415A1 (en) | 2018-08-16 |
US20200097584A9 US20200097584A9 (en) | 2020-03-26 |
Family
ID=59276076
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/952,620 Active 2036-10-04 US9954992B2 (en) | 2014-11-27 | 2015-11-25 | Mobile device enabled robotic system |
US15/951,750 Abandoned US20200097584A9 (en) | 2014-11-27 | 2018-04-12 | Mobile device enabled robotic system |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/952,620 Active 2036-10-04 US9954992B2 (en) | 2014-11-27 | 2015-11-25 | Mobile device enabled robotic system |
Country Status (1)
Country | Link |
---|---|
US (2) | US9954992B2 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20190073912A1 (en) * | 2017-09-01 | 2019-03-07 | Qualcomm Incorporated | Collision Management For A Robotic Vehicle |
Families Citing this family (21)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CA2977945A1 (en) | 2015-03-03 | 2016-09-09 | Prenav Inc. | Scanning environments and tracking unmanned aerial vehicles |
US20160335898A1 (en) * | 2015-04-09 | 2016-11-17 | Vulcan, Inc. | Automated drone management system |
WO2017020196A1 (en) * | 2015-07-31 | 2017-02-09 | 深圳市大疆创新科技有限公司 | Detection device, detection system, detection method and portable apparatus |
US11386383B2 (en) * | 2016-02-02 | 2022-07-12 | Mikko Vaananen | Social drone |
CN109416889A (en) * | 2016-06-13 | 2019-03-01 | 深圳市大疆创新科技有限公司 | Unmanned vehicle, delivery system, unmanned vehicle control method and control unmanned vehicle program |
US20180091797A1 (en) * | 2016-09-27 | 2018-03-29 | The Boeing Company | Apparatus and method of compensating for relative motion of at least two aircraft-mounted cameras |
WO2018144396A1 (en) | 2017-02-02 | 2018-08-09 | PreNav, Inc. | Tracking image collection for digital capture of environments, and associated systems and methods |
JP6789189B2 (en) * | 2017-07-27 | 2020-11-25 | 京セラ株式会社 | Aircraft and portable electronics |
JP6703507B2 (en) * | 2017-07-27 | 2020-06-03 | 京セラ株式会社 | Portable electronic device and security system |
WO2019021915A1 (en) * | 2017-07-27 | 2019-01-31 | 京セラ株式会社 | Aircraft, portable electronic instrument, security system, and telephone system |
JP6713960B2 (en) * | 2017-07-27 | 2020-06-24 | 京セラ株式会社 | Portable electronic devices, air vehicles and telephone systems |
ES2902707T3 (en) * | 2017-08-15 | 2022-03-29 | Saronikos Trading & Services Unipessoal Lda | Remotely controlled multirotor aircraft comprising an improved frame |
EP3668788B1 (en) | 2017-08-15 | 2023-10-18 | Saronikos Trading and Services, Unipessoal Lda | Improved multirotor aircraft and interface device |
CN108416268B (en) * | 2018-02-02 | 2021-06-01 | 华侨大学 | Action recognition method based on double-robot visual communication |
US10908609B2 (en) | 2018-04-30 | 2021-02-02 | Toyota Research Institute, Inc. | Apparatus and method for autonomous driving |
CN109358632A (en) * | 2018-12-07 | 2019-02-19 | 浙江大学昆山创新中心 | A kind of AGV control system of view-based access control model navigation |
US11105921B2 (en) * | 2019-02-19 | 2021-08-31 | Honeywell International Inc. | Systems and methods for vehicle navigation |
KR102135314B1 (en) * | 2020-03-09 | 2020-07-20 | 국방과학연구소 | A unmanned air vehicle and method for controlling the unmanned air vehicle |
CN112073304A (en) * | 2020-09-04 | 2020-12-11 | 华人运通(上海)云计算科技有限公司 | Social communication system and method between vehicles |
CN113051304B (en) * | 2021-04-02 | 2022-06-24 | 中国有色金属长沙勘察设计研究院有限公司 | Calculation method for fusion of radar monitoring data and three-dimensional point cloud |
KR20220168375A (en) | 2021-06-16 | 2022-12-23 | 한국전자통신연구원 | Method and appratus for interworking between multiple drone communication |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6856894B1 (en) * | 2003-10-23 | 2005-02-15 | International Business Machines Corporation | Navigating a UAV under remote control and manual control with three dimensional flight depiction |
US20150317924A1 (en) * | 2014-05-02 | 2015-11-05 | John Chowhan Park | Unmanned Aerial System for Creating Aerial Message |
US20150321758A1 (en) * | 2013-08-31 | 2015-11-12 | II Peter Christopher Sarna | UAV deployment and control system |
Family Cites Families (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP5761829B2 (en) * | 2011-09-16 | 2015-08-12 | 国立大学法人東京海洋大学 | Underwater communication system |
US20130238168A1 (en) * | 2012-03-07 | 2013-09-12 | Jerome Reyes | Vehicle control and interface with mobile device |
US8930044B1 (en) * | 2012-12-28 | 2015-01-06 | Google Inc. | Multi-part navigation process by an unmanned aerial vehicle for navigating to a medical situatiion |
US9747901B1 (en) * | 2015-02-27 | 2017-08-29 | Amazon Technologies, Inc. | Speech interaction for unmanned aerial vehicles |
-
2015
- 2015-11-25 US US14/952,620 patent/US9954992B2/en active Active
-
2018
- 2018-04-12 US US15/951,750 patent/US20200097584A9/en not_active Abandoned
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6856894B1 (en) * | 2003-10-23 | 2005-02-15 | International Business Machines Corporation | Navigating a UAV under remote control and manual control with three dimensional flight depiction |
US20150321758A1 (en) * | 2013-08-31 | 2015-11-12 | II Peter Christopher Sarna | UAV deployment and control system |
US20150317924A1 (en) * | 2014-05-02 | 2015-11-05 | John Chowhan Park | Unmanned Aerial System for Creating Aerial Message |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20190073912A1 (en) * | 2017-09-01 | 2019-03-07 | Qualcomm Incorporated | Collision Management For A Robotic Vehicle |
US10783796B2 (en) * | 2017-09-01 | 2020-09-22 | Qualcomm Incorporated | Collision management for a robotic vehicle |
Also Published As
Publication number | Publication date |
---|---|
US20200097584A9 (en) | 2020-03-26 |
US20170201614A1 (en) | 2017-07-13 |
US9954992B2 (en) | 2018-04-24 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9954992B2 (en) | Mobile device enabled robotic system | |
Tian et al. | Search and rescue under the forest canopy using multiple UAVs | |
CN109478068B (en) | Method, apparatus and storage medium for dynamically controlling a vehicle | |
US11353892B2 (en) | Drop-off location planning for delivery vehicle | |
US11429101B2 (en) | Adaptive autonomy system architecture | |
US20200034620A1 (en) | Self-reliant autonomous mobile platform | |
US20170102699A1 (en) | Drone control through imagery | |
US11858148B2 (en) | Robot and method for controlling the same | |
US20230058405A1 (en) | Unmanned aerial vehicle (uav) swarm control | |
JP6495557B2 (en) | Drone control system, method and program | |
Kandath et al. | Autonomous navigation and sensorless obstacle avoidance for UGV with environment information from UAV | |
US20220214692A1 (en) | VIsion-Based Robot Navigation By Coupling Deep Reinforcement Learning And A Path Planning Algorithm | |
JP6861434B2 (en) | Aircraft guidance methods, guidance devices, and guidance systems | |
US20210187749A1 (en) | Mobile robot and method for operating the same | |
US10375359B1 (en) | Visually intelligent camera device with peripheral control outputs | |
WO2020008754A1 (en) | Information processing device, optimum time estimation method, self-position estimation method, and recording medium in which program is recorded | |
KR102368734B1 (en) | Drone and drone control methods | |
WO2021049227A1 (en) | Information processing system, information processing device, and information processing program | |
Lu et al. | Mission planning of iOS application for a quadrotor UAV | |
CN111052028B (en) | System and method for automatic surface and sky detection | |
Zhang et al. | An integrated unmanned aerial vehicle system for vision based control | |
Gangapurwala et al. | A Novel Approach to Docking System for Autonomous Unmanned Aerial Vehicles | |
Martínez-Carranza et al. | Towards autonomous flight of low-cost MAVs by using a probabilistic visual odometry approach | |
US20240020876A1 (en) | Machine-Learned Monocular Depth Estimation and Semantic Segmentation for 6-DOF Absolute Localization of a Delivery Drone | |
Venchova et al. | Evolutionary Steps of the Embedded Computing Services: From Fixed to Mobile Streaming |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |