WO2021079778A1 - 移動体、情報処理方法、プログラム及び情報処理システム - Google Patents

移動体、情報処理方法、プログラム及び情報処理システム Download PDF

Info

Publication number
WO2021079778A1
WO2021079778A1 PCT/JP2020/038505 JP2020038505W WO2021079778A1 WO 2021079778 A1 WO2021079778 A1 WO 2021079778A1 JP 2020038505 W JP2020038505 W JP 2020038505W WO 2021079778 A1 WO2021079778 A1 WO 2021079778A1
Authority
WO
WIPO (PCT)
Prior art keywords
moving body
obstacle
information processing
control unit
unit
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2020/038505
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
駿 李
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Corp
Original Assignee
Sony Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Corp filed Critical Sony Corp
Priority to JP2021554291A priority Critical patent/JPWO2021079778A1/ja
Priority to US17/769,840 priority patent/US20220413518A1/en
Publication of WO2021079778A1 publication Critical patent/WO2021079778A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/10—Simultaneous control of position or course in three dimensions
    • G05D1/101—Simultaneous control of position or course in three dimensions specially adapted for aircraft
    • G05D1/106—Change initiated in response to external conditions, e.g. avoidance of elevated terrain or of no-fly zones
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64C—AEROPLANES; HELICOPTERS
    • B64C13/00—Control systems or transmitting systems for actuating flying-control surfaces, lift-increasing flaps, air brakes, or spoilers
    • B64C13/02—Initiating means
    • B64C13/16—Initiating means actuated automatically, e.g. responsive to gust detectors
    • B64C13/18—Initiating means actuated automatically, e.g. responsive to gust detectors using automatic pilot
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64C—AEROPLANES; HELICOPTERS
    • B64C39/00—Aircraft not otherwise provided for
    • B64C39/02—Aircraft not otherwise provided for characterised by special use
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64C—AEROPLANES; HELICOPTERS
    • B64C39/00—Aircraft not otherwise provided for
    • B64C39/02—Aircraft not otherwise provided for characterised by special use
    • B64C39/024—Aircraft not otherwise provided for characterised by special use of the remote controlled vehicle type, i.e. RPV
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64D—EQUIPMENT FOR FITTING IN OR TO AIRCRAFT; FLIGHT SUITS; PARACHUTES; ARRANGEMENT OR MOUNTING OF POWER PLANTS OR PROPULSION TRANSMISSIONS IN AIRCRAFT
    • B64D45/00—Aircraft indicators or protectors not otherwise provided for
    • B64D45/04—Landing aids; Safety measures to prevent collision with earth's surface
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64D—EQUIPMENT FOR FITTING IN OR TO AIRCRAFT; FLIGHT SUITS; PARACHUTES; ARRANGEMENT OR MOUNTING OF POWER PLANTS OR PROPULSION TRANSMISSIONS IN AIRCRAFT
    • B64D47/00—Equipment not otherwise provided for
    • B64D47/08—Arrangements of cameras
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G5/00—Traffic control systems for aircraft
    • G08G5/80—Anti-collision systems
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U10/00—Type of UAV
    • B64U10/10—Rotorcrafts
    • B64U10/13—Flying platforms
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U2101/00—UAVs specially adapted for particular uses or applications
    • B64U2101/30—UAVs specially adapted for particular uses or applications for imaging, photography or videography
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U2201/00—UAVs characterised by their flight controls
    • B64U2201/10—UAVs characterised by their flight controls autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]

Definitions

  • This technology relates to mobiles, information processing methods, programs and information processing systems.
  • Patent Document 1 a technique for stably avoiding obstacles is adopted.
  • drone aircraft need to detect obstacles in order to prevent their own failure.
  • the obstacle is a transparent object such as a window glass or a mirror, or a small object that is difficult to recognize
  • the drone aircraft detects such an obstacle by ordinary obstacle detection using a ToF or a stereo camera. It is difficult to do so and may collide with obstacles during flight.
  • the moving body has a control unit.
  • the control unit controls the behavior of the moving body based on the estimation result of estimating whether or not there is an obstacle that hinders the movement of the moving body from the image captured by the imaging unit.
  • the control unit may decelerate or stop the moving body when it is estimated that the obstacle is present.
  • control unit may execute a process of closely examining whether or not the obstacle actually exists.
  • the control unit may move the moving body to a position where the detection unit can detect the distance.
  • the control unit may cause the detection unit to measure the distance a plurality of times while moving the moving body.
  • control unit may land or stop the moving body or generate a movement route of the moving body to avoid the obstacle.
  • the control unit applies the image to the learner generated by applying the learning data to the machine learning algorithm, and based on the estimation result of estimating whether or not the obstacle exists.
  • the behavior of the moving body may be controlled.
  • the moving body may be a flying body.
  • the obstacle may be a transparent or translucent object.
  • the information processing system includes an information processing device and a mobile body.
  • the information processing device estimates from the image captured by the imaging unit whether or not there is an obstacle that hinders the movement of the moving body.
  • the mobile body controls the behavior of the mobile body based on the estimation result.
  • the information processing device may estimate whether or not the obstacle exists by applying the image to the learning device generated by applying the learning data to the machine learning algorithm.
  • the information processing device may be a server.
  • the information processing method is The behavior of the moving body is controlled based on the estimation result of estimating whether or not there is an obstacle that hinders the movement of the moving body from the image captured by the imaging unit.
  • the program according to one form of the present technology causes the mobile body to perform the following steps.
  • FIG. 1 is a diagram showing a configuration example of the information processing system 1 of the present embodiment
  • FIG. 2 is a block diagram showing a configuration example of the information processing system 1.
  • the information processing system 1 includes a drone machine 10 and an information processing device 20.
  • the drone machine 10 and the information processing device 20 are connected to each other so as to be able to communicate with each other via the network N.
  • the network N may be the Internet, a mobile communication network, a local area network, or the like, or may be a network in which these plurality of types of networks are combined.
  • the drone body 10 includes a control unit 11, a storage unit 12, a sensor 13, a camera 14, and a communication unit 15.
  • the drone body 10 is an example of a "mobile body” within the scope of claims.
  • the control unit 11 controls the entire operation of the drone machine 10 or a part thereof according to the program stored in the storage unit 12.
  • the storage unit 12 stores sensor data output from the sensor 13, image data acquired from the camera 14, and the like.
  • the control unit 11 functionally includes an acquisition unit 16, an operation control unit 17, and an obstacle presence inspection unit 18.
  • the acquisition unit 16 acquires the sensor data detected by the sensor 13 and the image data captured by the camera 14.
  • the motion control unit 17 controls the speed of the drone body 10 and the sensor 13 based on the judgment of the obstacle presence estimation unit 211 or the obstacle presence inspection unit 18.
  • the obstacle existence inspection unit 18 determines whether or not the obstacle really exists after the drone body 10 is controlled by the operation control unit 214.
  • the obstacle in the present embodiment is, for example, a transparent or translucent transparent object such as a window glass, or an unrecognizable object such as a fine net or an electric wire, and the same applies to the following description.
  • the sensor 13 includes a distance measuring sensor such as a sonar, a radar, or a rider, a GPS sensor that measures the position information of the drone body 10, and the like.
  • the sensor 13 is an example of a "detection unit" in the claims.
  • the communication unit 15 communicates with the information processing device 20 via the network N.
  • the communication unit 15 functions as a communication interface for the drone body 10.
  • the camera 14 actually uses various members such as an image sensor such as a CMOS (Complementary Metal Oxide Semiconductor) or a CCD (Charge Coupled Device) and a lens for controlling the image formation of a subject image on the image sensor. It is a device that captures a space and generates an captured image.
  • CMOS Complementary Metal Oxide Semiconductor
  • CCD Charge Coupled Device
  • the camera 14 may capture a still image or may capture a moving image.
  • the camera 14 is connected to the main body of the drone body 10 via a drive mechanism (not shown) such as a 3-axis gimbal.
  • the information processing device 20 includes a communication unit 23, a control unit 21, and a storage unit 22.
  • the information processing device 20 is typically a server device, but is not limited to this, and may be any other computer such as a PC.
  • the information processing device 20 may be a control control device that gives instructions to the drone body 10 and executes flight control to guide the drone aircraft 10.
  • the communication unit 23 communicates with the drone machine 10 via the network N.
  • the communication unit 23 functions as a communication interface of the information processing device 20.
  • the control unit 21 controls the entire operation of the information processing device 20 or a part thereof according to the program stored in the storage unit 22.
  • the control unit 21 corresponds to the "control unit" in the claims.
  • the control unit 21 functionally has an obstacle presence estimation unit 211.
  • the obstacle existence estimation unit 211 determines whether or not there is an obstacle that hinders the flight of the drone aircraft 10 in the real space where the drone aircraft 10 exists.
  • FIG. 3 is a block diagram showing a hardware configuration example of the drone machine 10 and the information processing device 20.
  • the drone machine 10 and the information processing device 20 may be realized by the information processing device 100 shown in FIG.
  • the information processing device 100 has a CPU (Central Processing unit) 101, a ROM (Read Only Memory) 102, and a RAM (Random Access Memory) 103.
  • the control units 11 and 21 may be the CPU 101.
  • the information processing device 100 may have a host bus 104, a bridge 105, an external bus 106, an interface 107, an input device 108, an output device 109, a storage device 110, a drive 111, a connection port 112, and a communication device 113. ..
  • the information processing device 100 may have a configuration including an image pickup device 114 and a sensor 115. Further, the information processing apparatus 100 replaces or together with the CPU 101 for processing such as a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit). It may have a circuit.
  • DSP Digital Signal Processor
  • ASIC Application Specific Integrated Circuit
  • FPGA Field-Programmable Gate Array
  • GPU Graphics Processing Unit
  • the CPU 101 functions as an arithmetic processing device and a control device, and controls all or a part of the operation in the information processing device 100 according to various programs recorded in the ROM 102, the RAM 103, the storage device 110, or the removable recording medium 30.
  • the storage units 12 and 22 may be the ROM 102, the RAM 103, the storage device 110, or the removable recording medium 30.
  • the ROM 102 stores programs and calculation parameters used by the CPU 101.
  • the RAM 103 primarily stores a program used in the execution of the CPU 101, parameters that are appropriately changed in the execution, and the like.
  • the CPU 101, ROM 102, and RAM 103 are connected to each other by a host bus 104 composed of an internal bus such as a CPU bus. Further, the host bus 104 is connected to an external bus 106 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 105.
  • a PCI Peripheral Component Interconnect / Interface
  • the input device 108 is a device operated by a user, such as a mouse, keyboard, touch panel, buttons, switches, and levers.
  • the input device 108 may be, for example, a remote control device using infrared rays or other radio waves, or an externally connected device 40 such as a mobile phone corresponding to the operation of the information processing device 100.
  • the input device 108 includes an input control circuit that generates an input signal based on the information input by the user and outputs the input signal to the CPU 101. By operating the input device 108, the user inputs various data to the information processing device 100 and instructs the processing operation.
  • the output device 109 is composed of a device capable of notifying the user of the acquired information using sensations such as sight, hearing, and touch.
  • the output device 109 may be, for example, a display device such as an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display, an audio output device such as a speaker or a headphone, or a vibrator.
  • a display device such as an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display
  • an audio output device such as a speaker or a headphone, or a vibrator.
  • the output device 109 outputs the result obtained by the processing of the information processing device 100 as a video such as a text or an image, a voice such as voice or sound, or a vibration.
  • the storage device 110 is a data storage device configured as an example of the storage unit of the information processing device 100.
  • the storage device 110 is composed of, for example, a magnetic storage device such as an HDD (Hard Disk Drive), a semiconductor storage device, an optical storage device, a magneto-optical storage device, or the like.
  • the storage device 110 stores, for example, a program executed by the CPU 101, various data, various data acquired from the outside, and the like.
  • the drive 111 is a reader / writer for a removable recording medium 30 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and is built in or externally attached to the information processing device 100.
  • the drive 111 reads the information recorded on the mounted removable recording medium 30 and outputs the information to the RAM 103. Further, the drive 111 writes a record on the removable recording medium 30 mounted on the drive 111.
  • connection port 112 is a port for connecting the device to the information processing device 100.
  • the connection port 112 may be, for example, a USB (Universal Serial Bus) port, an IEEE1394 port, a SCSI (Small Computer System Interface) port, or the like.
  • connection port 112 may be an RS-232C port, an optical audio terminal, an HDMI (registered trademark) (High-Definition Multimedia Interface) port, or the like.
  • the communication device 113 is, for example, a communication interface composed of a communication device for connecting to the network N.
  • the communication device 113 may be, for example, a communication card for LAN (Local Area Network), Bluetooth (registered trademark), Wi-Fi, WUSB (Wireless USB), LTE (Long Term Evolution), or the like.
  • the communication device 113 may be a router for optical communication, a router for ADSL (Asymmetric Digital Subscriber Line), a modem for various communications, or the like.
  • the communication device 113 transmits and receives signals and the like to and from the Internet and other communication devices using a predetermined protocol such as TCP / IP.
  • the network N connected to the communication device 113 is a network connected by radio, and may include, for example, the Internet, infrared communication, radio wave communication, short-range wireless communication, satellite communication, and the like.
  • the communication units 15 and 23 may be the communication device 113.
  • the image pickup device 114 is a device that captures an image of the real space and generates an captured image.
  • the camera 14 corresponds to the image pickup apparatus 114.
  • the sensor 115 is, for example, various sensors such as an acceleration sensor, an angular velocity sensor, a geomagnetic sensor, an illuminance sensor, a temperature sensor, a pressure sensor, or a sound sensor (microphone).
  • various sensors such as an acceleration sensor, an angular velocity sensor, a geomagnetic sensor, an illuminance sensor, a temperature sensor, a pressure sensor, or a sound sensor (microphone).
  • the sensor 115 acquires information on the state of the information processing device 100 itself, such as the posture of the housing of the information processing device 100, and information on the surrounding environment of the information processing device 100, such as brightness and noise around the information processing device 100. To do. Further, the sensor 115 may include a GPS receiver that receives a GPS (Global Positioning System) signal and measures the latitude, longitude, and altitude of the device. The sensor 13 corresponds to the sensor 115.
  • GPS Global Positioning System
  • Each of the above-mentioned components may be configured by using a general-purpose member, or may be composed of a member specialized in the function of each component. Such a configuration can be appropriately changed depending on the technical level at the time of implementation.
  • FIG. 4 is a flowchart showing a typical operation flow of the information processing system 1. Hereinafter, the operation of the information processing system 1 will be described with reference to FIG. 4 as appropriate.
  • the obstacle presence estimation unit 211 corresponds between the image data captured by the camera 14 at a predetermined frame rate (for example, several tens of fps) and the action result of the drone machine 10 when the image data is captured. Acquire the attached data (hereinafter, learning data) from the user. At this time, the user transmits the learning data as an error report to the information processing apparatus 20 via, for example, an arbitrary device (not shown), and the learning data is stored in the storage unit 22.
  • a predetermined frame rate for example, several tens of fps
  • the above-mentioned action result refers to, for example, an action in which the drone machine 10 collides with an obstacle existing in the real space and crashes, or the drone machine 10 suddenly stops in front of the obstacle.
  • FIG. 5 is a flowchart showing the details of step S102. Hereinafter, step S102 will be described with reference to FIG. 5 as appropriate.
  • the information processing device 20 of the present embodiment is an information processing device that uses a so-called specialized AI (Artificial Intelligence) that substitutes for the intellectual work of the user.
  • FIG. 6 is a diagram simply showing a processing procedure of a general specialized AI.
  • Specialized AI is a mechanism that, as a large framework, obtains a result by applying arbitrary input data to a trained model constructed by incorporating learning data into an algorithm that functions as a learning program. ..
  • the obstacle presence estimation unit 211 reads out the learning data stored in the storage unit 22 (step S1021). This learning data corresponds to the "learning data" in FIG.
  • the obstacle presence estimation unit 211 generates a learning device by applying the learning data read from the storage unit 22 to a preset algorithm.
  • the above-mentioned algorithm corresponds to the "algorithm" in FIG. 6, and is, for example, a machine learning algorithm.
  • the type of machine learning algorithm is not particularly limited, and is, for example, RNN (Recurrent Neural Network), CNN (Convolutional Neural Network), GAN (Generative Adversarial Network), or MLP (MLP).
  • An algorithm using a neural network such as Multilayer Perceptron may be used, and other supervised learning methods (boostering method, SVM (Support Vector Machine) method, SVR method (Support Vector Regression) method, etc.), It may be an arbitrary algorithm that executes an unsupervised learning method, a semi-supervised learning method, an enhanced learning method, or the like.
  • MLP and CNN which is an extension thereof, are typically adopted as an algorithm used for constructing a learning device.
  • MLP is a kind of neural network, and it is known that any nonlinear function can be approximated by a three-layer neural network if there are an infinite number of neurons in the hidden layer H, and it is customarily a three-layer neural network. There are many. Therefore, in the present embodiment, the case where the MLP is a three-layer neural network will be described as an example.
  • the obstacle presence estimation unit 211 acquires the connection weight of the three-layer neural network stored in the storage unit 22 (step S1022), and applies this connection weight to the sigmoid function to generate a learner. Specifically, if the input stimulus to the i-th neuron Ii in the input layer I is xi and the connection weight between Ii and the j-th neuron in the hidden layer H is ⁇ Iji, the output zj of the hidden layer H is, for example. , It is represented by the following formula (1).
  • sigmoid is a sigmoid function and is expressed by the following equation (2).
  • a 1, it is a standard sigmoid function.
  • the output signal yk of the kth neuron in the output layer O is represented by, for example, the following equation (3).
  • the sigmoid function of the output layer O is omitted.
  • the notation for each element using ⁇ in the equations (1) and (3) is expressed more concisely by applying the sigmoid function for each dimension.
  • W I, W H is an internal parameter of a three-layer neural network (weights).
  • the obstacle presence estimation unit 211 executes a process of updating the learning device until the output error is minimized (step S1023). Specifically, the obstacle presence estimation unit 211 is obtained by using the image data for constructing the learning data and the action result as an input signal and a teacher signal (teacher data), respectively, and applying the input signal to the equation (4). error between the output signal and the teacher signal updates the internal parameters W I, W H until convergence. The obstacle presence estimation unit 211 outputs the internal parameters WI (min) and WH (min) that minimize the error to the storage unit 22 (step S1024).
  • the obstacle presence estimation unit 211 reads out the internal parameters WI (min) and WH (min) stored in the storage unit 22, and applies these to the equation (4) so that the learning device 221 is stored in the storage unit 22. To build.
  • the learner 221 corresponds to the "trained model" of FIG.
  • FIG. 7 is a flowchart showing the details of step S103. Hereinafter, step S103 will be described with reference to FIG. 7 as appropriate.
  • the obstacle presence estimation unit 211 acquires an image captured at a predetermined frame rate (for example, several tens of fps) from the camera 14 (step S1031). This image corresponds to the "input data" in FIG.
  • the obstacle presence estimation unit 211 estimates whether or not an obstacle exists in the image by applying the learner 221 to the image acquired in the previous step S1031 (step S1032). This estimation result is output to the operation control unit 17.
  • the estimation result corresponds to the "result" of FIG.
  • the motion control unit 17 decelerates the drone aircraft 10 in flight.
  • an instruction to stop is output, and the operation control unit 17 executes this instruction (step S1034).
  • the drone aircraft 10 stops or decelerates regardless of whether or not an obstacle actually exists, so that a collision between the obstacle and the drone aircraft 10 is avoided.
  • step S1035 the obstacle presence inspection unit 18 examines whether or not the obstacle presumed to exist in the previous step S1032 really exists.
  • step S1035 some application examples of step S1035 will be described.
  • FIG. 8 is a conceptual diagram showing the drone aircraft 10 and the obstacle together
  • FIG. 9 is a graph plotting the existence probability of the obstacle according to the distance between the drone aircraft 10 and the obstacle.
  • the obstacle existence inspection unit 18 outputs an instruction to move the drone aircraft 10 to a position D away from the obstacle by a predetermined distance D to the operation control unit 17.
  • the operation control unit 17 executes this instruction.
  • the predetermined distance D is a distance at which the sensor 13 can measure the distance between the drone body 10 and the obstacle.
  • the obstacle presence inspection unit 18 outputs an instruction to measure the distance between the drone machine 10 and the obstacle a plurality of times while bringing the drone machine 10 close to the obstacle, and the sensor 13 outputs this instruction to the sensor 13. Execute the instruction.
  • FIG. 9 it can be seen that the smaller the distance between the drone body 10 and the obstacle, the higher the probability of existence of the obstacle.
  • FIG. 10 is a conceptual diagram showing the drone aircraft 10 and obstacles together
  • FIG. 11 is a graph in which the existence probabilities of obstacles are plotted according to the moving distance of the drone aircraft 10.
  • the obstacle existence inspection unit 18 outputs an instruction to move the drone aircraft 10 to a position D away from the obstacle by a predetermined distance D to the operation control unit 17.
  • the operation control unit 17 executes this instruction.
  • the obstacle presence inspection unit 18 outputs an instruction to move the drone aircraft 10 to the motion control unit 17 while maintaining the predetermined distance D after approaching the drone aircraft 10 to the predetermined distance D, and outputs the instruction to move the drone aircraft 10 to the motion control unit 17. 17 executes this instruction.
  • the obstacle presence inspection unit 18 outputs an instruction to measure the distance between the drone body 10 and the obstacle a plurality of times to the sensor 13, and the sensor 13 executes this instruction.
  • FIG. 11 it can be seen that the existence probability of obstacles increases as the moving distance of the drone aircraft 10 increases.
  • the obstacle presence estimation unit 211 estimates that there is no obstacle in the image captured in the previous step S1031 (NO in step S1033)
  • the obstacle presence estimation unit 211 outputs an instruction to continue the flight to the motion control unit 17.
  • the motion control unit 17 executes this instruction.
  • the motion control unit 17 determines that the obstacle actually exists in the previous step S1035, specifically, the obstacle is measured a plurality of times in the distance between the drone machine 10 and the obstacle.
  • the existence probability of the drone exceeds the predetermined thresholds L1 and L3, the drone aircraft 10 is landed or stopped.
  • a movement path for avoiding obstacles is generated, and flight control according to this movement path is executed (step S1036). As a result, the collision between the drone body 10 and the obstacle is surely avoided.
  • the motion control unit 17 specifically measures the distance between the drone body 10 and the obstacle a plurality of times.
  • the existence probability of the drone is less than the predetermined threshold values L2 and L4, the control imposed on the drone aircraft 10 in the previous step S1034 is released.
  • the drone machine 10 when it is estimated that an obstacle exists by applying the image captured by the camera 14 to the learning device 221, the drone machine 10 is decelerated or stopped, but the present invention is not limited to this.
  • the drone machine 10 may be decelerated or stopped by recognizing an obstacle in the image according to a predetermined algorithm for recognizing an object.
  • the drone aircraft 10 when examining the presence or absence of an obstacle in the previous step S1035, the drone aircraft 10 is brought close to the obstacle and the distance between the drone aircraft 10 and the obstacle is detected a plurality of times.
  • the drone body 10 may be rotated with or instead of such an operation so that the sensor 13 or the camera 14 faces the obstacle.
  • the present technology may be applied to a moving body other than a flying body (for example, a robot), and its use is not particularly limited.
  • a flying body for example, a robot
  • airframes include aircraft, unmanned aerial vehicles, unmanned helicopters, and the like.
  • the description has been made on the premise that the drone body 10 flies outdoors, but the present invention is not limited to this, and the present technology may be applied to, for example, a moving body moving indoors.
  • the moving body including a control unit that controls the behavior of the moving body based on an estimation result of estimating whether or not there is an obstacle that hinders the movement of the moving body from an image captured by the imaging unit.
  • the control unit is a moving body that decelerates or stops the moving body when it is estimated that the obstacle is present.
  • the control unit is a mobile body that executes a process of examining whether or not the obstacle actually exists when it is estimated that the obstacle exists.
  • the above-mentioned moving body is a moving body which is a flying body.
  • the obstacle is a moving object that is a transparent or translucent object.
  • An information processing device that estimates whether or not there is an obstacle that hinders the movement of the moving object from the image captured by the image pickup unit.
  • the information processing device is an information processing system that estimates whether or not an obstacle exists by applying the image to a learning device generated by applying learning data to a machine learning algorithm.
  • the information processing device is an information processing system that is a server.
  • An information processing method that controls the behavior of the moving body based on an estimation result that estimates whether or not there is an obstacle that hinders the movement of the moving body from the image captured by the imaging unit.
  • a program that causes the moving body to perform a step of controlling the behavior of the moving body based on an estimation result that estimates whether or not there is an obstacle that hinders the movement of the moving body from the image captured by the imaging unit.

Landscapes

  • Engineering & Computer Science (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Automation & Control Theory (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
  • Image Analysis (AREA)
PCT/JP2020/038505 2019-10-24 2020-10-12 移動体、情報処理方法、プログラム及び情報処理システム Ceased WO2021079778A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2021554291A JPWO2021079778A1 (cg-RX-API-DMAC7.html) 2019-10-24 2020-10-12
US17/769,840 US20220413518A1 (en) 2019-10-24 2020-10-12 Movable object, information processing method, program, and information processing system

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2019193263 2019-10-24
JP2019-193263 2019-10-24

Publications (1)

Publication Number Publication Date
WO2021079778A1 true WO2021079778A1 (ja) 2021-04-29

Family

ID=75619310

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/038505 Ceased WO2021079778A1 (ja) 2019-10-24 2020-10-12 移動体、情報処理方法、プログラム及び情報処理システム

Country Status (3)

Country Link
US (1) US20220413518A1 (cg-RX-API-DMAC7.html)
JP (1) JPWO2021079778A1 (cg-RX-API-DMAC7.html)
WO (1) WO2021079778A1 (cg-RX-API-DMAC7.html)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115946884A (zh) * 2022-09-28 2023-04-11 浙大城市学院 一种检测被动式房屋碳排放的装置
US12168533B1 (en) * 2023-01-18 2024-12-17 Sifly Aviation, Inc. Autonomous unmanned aircraft system

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018144846A1 (en) * 2017-02-02 2018-08-09 Brain Corporation Systems and methods for assisting a robotic apparatus
JP2018133749A (ja) * 2017-02-16 2018-08-23 オリンパス株式会社 被操縦体、移動装置、撮像装置、移動制御方法、移動補助方法、移動制御プログラムおよび移動補助プログラム
JP2019053501A (ja) * 2017-09-14 2019-04-04 Kddi株式会社 飛行装置、管理装置、飛行管理方法及びプログラム

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8108092B2 (en) * 2006-07-14 2012-01-31 Irobot Corporation Autonomous behaviors for a remote vehicle
EP3391339A2 (en) * 2015-12-18 2018-10-24 Iris Automation, Inc. Real-time visual situational awareness system
DE102017203838B4 (de) * 2017-03-08 2022-03-17 Audi Ag Verfahren und System zur Umfelderfassung
US10937325B2 (en) * 2018-12-27 2021-03-02 Intel Corporation Collision avoidance system, depth imaging system, vehicle, obstacle map generator, and methods thereof
US11237572B2 (en) * 2018-12-27 2022-02-01 Intel Corporation Collision avoidance system, depth imaging system, vehicle, map generator and methods thereof
US11030766B2 (en) * 2019-03-25 2021-06-08 Dishcraft Robotics, Inc. Automated manipulation of transparent vessels

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018144846A1 (en) * 2017-02-02 2018-08-09 Brain Corporation Systems and methods for assisting a robotic apparatus
JP2018133749A (ja) * 2017-02-16 2018-08-23 オリンパス株式会社 被操縦体、移動装置、撮像装置、移動制御方法、移動補助方法、移動制御プログラムおよび移動補助プログラム
JP2019053501A (ja) * 2017-09-14 2019-04-04 Kddi株式会社 飛行装置、管理装置、飛行管理方法及びプログラム

Also Published As

Publication number Publication date
JPWO2021079778A1 (cg-RX-API-DMAC7.html) 2021-04-29
US20220413518A1 (en) 2022-12-29

Similar Documents

Publication Publication Date Title
Sadhu et al. On-board deep-learning-based unmanned aerial vehicle fault cause detection and identification
US20240289930A1 (en) Deep learning-based real-time detection and correction of compromised sensors in autonomous machines
CN107925821B (zh) 监控
US11126833B2 (en) Artificial intelligence apparatus for recognizing user from image data and method for the same
KR102696200B1 (ko) 음성 인식 서비스를 제공하는 인공 지능 장치 및 그의 동작 방법
US10339913B2 (en) Context-based cancellation and amplification of acoustical signals in acoustical environments
EP4066243B1 (en) Sound event detection learning
JP7823680B2 (ja) 情報処理装置、制御方法、及びプログラム
WO2021095351A1 (ja) 監視装置、監視方法、及びプログラム
CN114270285B (zh) 移动体、信息处理装置、信息处理方法和程序产品
KR102280803B1 (ko) 로봇 및 그의 구동 방법
KR102936206B1 (ko) 무인기 추적 방법 및 장치
US11372473B2 (en) Information processing apparatus and information processing method
US20220413518A1 (en) Movable object, information processing method, program, and information processing system
US20220309343A1 (en) Always-on local action controller for low power, battery-operated autonomous intelligent devices
JP6705886B2 (ja) 監視
US12055396B2 (en) Information processing apparatus, moving body, information processing system, information processing method, and program
JP2018523231A5 (cg-RX-API-DMAC7.html)
KR102816865B1 (ko) 일체형 cctv 카메라 시스템
JP7452543B2 (ja) 情報処理装置、情報処理方法、プログラム及び情報処理システム
CN119450018A (zh) 投影方法和装置
US11997445B2 (en) Systems and methods for live conversation using hearing devices
JP7468515B2 (ja) 情報処理装置、情報処理方法及びプログラム
KR102247911B1 (ko) 인공 지능 로봇 및 인공 지능 서버
WO2026034826A1 (en) Command detection for continuous conversation with digital assistants using auto encoders and joint layers

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20879720

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021554291

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20879720

Country of ref document: EP

Kind code of ref document: A1