WO2018228258A1 - Mobile electronic device and method therein - Google Patents

Mobile electronic device and method therein Download PDF

Info

Publication number
WO2018228258A1
WO2018228258A1 PCT/CN2018/090176 CN2018090176W WO2018228258A1 WO 2018228258 A1 WO2018228258 A1 WO 2018228258A1 CN 2018090176 W CN2018090176 W CN 2018090176W WO 2018228258 A1 WO2018228258 A1 WO 2018228258A1
Authority
WO
WIPO (PCT)
Prior art keywords
electronic device
mobile electronic
image
obstacle
task
Prior art date
Application number
PCT/CN2018/090176
Other languages
French (fr)
Chinese (zh)
Inventor
潘景良
陈灼
李腾
陈嘉宏
高鲁
Original Assignee
炬大科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 炬大科技有限公司 filed Critical 炬大科技有限公司
Publication of WO2018228258A1 publication Critical patent/WO2018228258A1/en

Links

Images

Classifications

    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05DSYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
    • G05D1/02Control of position or course in two dimensions
    • G05D1/021Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0212Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
    • G05D1/0217Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory in accordance with energy consumption, time reduction or distance reduction criteria

Definitions

  • the first mobile electronic device further includes a charging post, wherein the charging post comprises the image processor and the positioning module.
  • the first mobile electronic device may further include a sensor that transmits second obstacle information around the first mobile electronic device to the motion module, the motion module further configured to Adjusting a motion orientation of the first mobile electronic device to avoid the second obstacle.
  • the senor comprises an ultrasonic sensor and/or a laser sensor.
  • a method in a first mobile electronic device comprising: Communicatingly connecting to the wireless signal transceiver of at least one camera installed indoors, acquiring an image of a task area including the first mobile electronic device collected by the at least one camera; communicably connecting And the image processor to the wireless signal transceiver identifying feature information of the first mobile electronic device in an image of the task area, and establishing a coordinate system for the image, and for the task area Each point is assigned a corresponding coordinate value; by comparing the feature information of the first mobile electronic device and the coordinate origin on the established coordinate system by the positioning module communicably connected to the image processor Identifying a location and an outline of the first mobile electronic device, and mapping a map of the task area; communicably connecting to The positioning module of the mobile electronic device transmitting the map to the second mobile electronic device, and the second mobile electronic device receiving a task location
  • FIG. 1 shows a schematic diagram of a system in which a first mobile electronic device and a second mobile electronic device are located, in accordance with one embodiment of the present invention.
  • FIG. 2 shows a block diagram of a processor in a first mobile electronic device in accordance with one embodiment of the present invention.
  • 3A-3D show schematic diagrams of a first mobile electronic device in accordance with one embodiment of the present invention.
  • the first mobile electronic device 100 includes, but is not limited to, a cleaning robot, an industrial automation robot, a service robot, a disaster relief robot, an underwater robot, a space robot, an autonomous vehicle, and the like.
  • the camera can be carried by, for example, a drone flying overhead.
  • the first mobile electronic device 100 does not itself include a camera.
  • the mission area represents the venue where the first mobile electronic device 100 performs the task. For example, when the first mobile electronic device 100 is a cleaning robot, the mission area represents an area where the cleaning robot needs to be cleaned. For another example, when the first mobile electronic device 100 is a disaster relief robot, the mission area indicates a place where the disaster relief robot needs to be rescued.
  • the first mobile electronic device 100 includes a wireless signal transceiver 102, an image processor 1040, a positioning module 1042, and a motion module 106.
  • the first mobile electronic device 100 further includes an encoder and an Inertial Measurement Unit (IMU), which are carried by the first mobile electronic device 100, that is, the robot 100 itself.
  • the first mobile electronic device 100 may further include a memory 110, a data processor 1046, which may be on the robot 100 or inside the charging post, optionally inside the charging post. The two functions of the memory 110 and the data processor 1046 are to store and process image information, data information, and the like.
  • Memory 110 is communicably coupled to processor 104.
  • Data processor 1046 is located in processor 104. As shown in FIG. 2, data processor 1046 is coupled to image processor 1040 and path planning module 1044.
  • Image processor 1040 is communicably coupled to wireless signal transceiver 102 and is configured to identify feature information of first mobile electronic device 100 in an image of the mission area. In addition, the image processor 1040 also establishes a two-dimensional coordinate system for the planar image, and assigns corresponding coordinate values to each point in the room.
  • the positioning module 1042 is communicably coupled to the image processor 1040, configured to compare the identified feature information of the first mobile electronic device 100 with a coordinate origin on the established coordinate system to identify the location and contour of the first mobile electronic device 100. And draw a map of the mission area.
  • the indoor plan view taken in the camera 140 is passed to the image processor 1040 in the first mobile electronic device 100 for processing analysis.
  • the function of image processor 1040 is to identify features of first mobile electronic device 100 in the image.
  • the graphics processor 1040 may identify the features by using a Scale Invariant Feature Transform (SIFT) algorithm or a Speeded Up Robust Features (SURF) algorithm.
  • SIFT Scale Invariant Feature Transform
  • SURF Speeded Up Robust Features
  • the image processor 1040 first identifies the key points of the object of the reference image stored in the memory 110, extracts the SIFT features, and then compares the SIFT features of the respective key points in the memory 110 with the SIFT features of the newly acquired image, and then based on the K nearest neighbor
  • the matching feature of the algorithm K-Nearest Neighbor KNN
  • the SURF algorithm is based on an approximate 2D Haar wavelet response and uses an integral image for image convolution using a Hessian matrix-based measure for the detector. And use a distribution-based descriptor.
  • image processor 1040 and positioning module 1042 may both be included in processor 104. Processor 104 will be further described below in conjunction with FIG.
  • the position and contour of the 100 is achieved for the development of indoor maps.
  • the user's second mobile electronic device 120 such as the handset 120, is coupled to the wireless signal transceiver 102 of the first mobile electronic device 100 via WIFI.
  • the APP of the mobile phone 120 reads the prepared indoor map, and the user sets the task position on the map by using the mobile APP, that is, a single or multiple positions to be cleaned (corresponding to X, Y coordinate information), and the first mobile electronic device 100
  • the positioning module 1042 records the set position information.
  • the first mobile electronic device 100 searches for the first mobile electronic device 100, such as the image of the cleaning robot 100, by using the ceiling camera 140 and its internal image processing module 1040, and cooperates with the positioning module 1042 to coordinate information in the real-time map of the robot 100. , Y) for positioning.
  • the image of the mission area further includes at least one first obstacle.
  • the image processor 1040 is further configured to identify feature information of the at least one first obstacle in the image of the mission area. As discussed above, image processor 1040 can perform image recognition using SIFT or SURF algorithms.
  • the positioning module 1042 is further configured to compare the feature information of the at least one first obstacle with a coordinate origin to identify the position and contour of the at least one obstacle.
  • the obstacle avoidance module 1048 and the motion module 106 are further configured to determine a task location selected by the user using the second mobile electronic device 120, such as the handset 120, the location of the located first mobile electronic device 100, and the identified at least one first The position and contour of the obstacle, the path is planned, and motion is performed according to the path.
  • the path planning module 1044 in the first mobile electronic device 100 plans an optimal path that can avoid the obstacle, and transmits the path information to the robot 100 through the wireless network.
  • the robot 100 starts from this path to the cleaning location to start the task.
  • the first mobile electronic device 100 may further include a sensor 108 that transmits second obstacle information around the first mobile electronic device 100 to the motion module 106.
  • the motion module 106 is also configured to adjust the motion orientation of the first mobile electronic device 100 to avoid the second obstacle. It can be understood that because the height of the installation is different, the camera 140 mounted on the ceiling captures the top view information, and the sensor mounted on the first mobile electronic device 100 acquires the head-up information, so the camera 140 mounted on the ceiling captures the image.
  • the obstacle information may be different from the obstacle of the sensor in the robot because there may be shadowing.
  • the motion module 106 integrates the first obstacle information captured by the camera 140 mounted on the ceiling and the second obstacle information sensed by the sensor to perform an obstacle avoidance planning path.
  • the sensor comprises an ultrasonic sensor and/or a laser sensor.
  • the first obstacle and the second obstacle may be the same or different. Therefore, the camera 140 and the sensor 108 can assist each other. For example, if there is shielding, the robot 100 needs to perform obstacle avoidance by its own laser sensor, ultrasonic sensor, or the like in the shaded portion.
  • the camera 140 is mounted vertically on the ceiling of the mission area.
  • the camera 140 can be, for example, a wide-angle camera, such as a fisheye camera. Therefore, when the image processor 1040 uses the photo taken by the camera 140, only the height information needs to be considered, so that the captured image is scaled, and the target in the image can be obtained (for example, the first mobile electronic device 100, for example, The actual distance between the robot and the first obstacle).
  • the height information may be acquired by an altimeter or a distance measuring device, such as a laser range finder or an infrared range finder. The height information may be measured by the first mobile electronic device 100 or may be pre-stored in the first mobile electronic device 100. in.
  • FIGS. 3A-3D show schematic diagrams of a first mobile electronic device 100 in accordance with one embodiment of the present invention.
  • the first mobile electronic device 100 will be described as an example of the robot 300.
  • the robot 300 is mainly composed of a main body 310, a wireless transceiver 320, a microprocessor 330, a sensor 340, and a charging post 350.
  • the user can implement the fixed point function in a certain area through the wireless transceiver 320, the microprocessor 330 and the charging post 350, and assist the main body 310 through the sensor 340 to perform the specified function operation.
  • the wireless transceiver 320 can transmit communication signals such as Bluetooth, WIFI, ZigBee, infrared, and ultrasonic waves through mobile electronic devices such as mobile phones and computers, and can also perform position information interaction with the microprocessor 330 and the charging pile 350.
  • the function sends the location information to the microprocessor 330 in an instructional manner.
  • the microprocessor 330 is disposed on the host body 310, can receive an instruction sent by the wireless transceiver 320, and performs program processing on the received instruction, and transmits the processed result to the control unit of the host body 310 to execute the corresponding task.
  • the microprocessor 330 can also transmit communication signals such as Bluetooth, WIFI, ZigBee, infrared, ultrasonic, etc. to the charging pile 350 to realize an automatic back-stack charging function.
  • the sensor 340 can detect the surrounding environment in real time, and realize the functions of intelligent obstacle avoidance, communication signal detection and transmission.
  • the charging post 350 can receive the signal transmitted by the wireless transceiver 320, and can also receive the communication signal sent by the microprocessor 330 to implement the location information interaction function, thereby guiding the microprocessor 330 to control the host body 310 to implement the back-charge charging. The function.
  • FIG. 4 shows a flow chart of a method in a first mobile electronic device in accordance with one embodiment of the present invention.
  • the method 400 includes acquiring, in block 410, an image of a mission area, including the first mobile electronic device 100, captured by the camera 140 by a wireless signal transceiver 102 communicably coupled to a camera 140 mounted indoors;
  • the feature information of the first mobile electronic device 100 in the image of the mission area is identified in block 420 by the image processor 1040 communicatively coupled to the wireless signal transceiver 102; in block 430, the image is communicably coupled to the image
  • the positioning module 1042 of the processor 1040 identifies the location and contour of the first mobile electronic device 100 by establishing a coordinate system and comparing the identified feature information of the first mobile electronic device 100 with the coordinate origin, and mapping the task area.
  • the method 400 further comprising (not shown): identifying, by the image processor 1040, at least one of the images of the mission area Feature information of an obstacle; comparing the feature information of the at least one first obstacle with the coordinate origin by the positioning module 1042, identifying the position and contour of the at least one obstacle; and the first position of the positioning according to the task position by the motion module 106 Moving the position of the electronic device 100 and the identified position and contour of the at least one first obstacle, planning a path, and moving according to the path.
  • the method 400 further includes (not shown): moving through the motion module 106; acquiring an image of the mission area including the first mobile electronic device 106 in real time by the camera;
  • the device 1040 and the positioning module 1042 dynamically determine feature information, location, and contours of the first mobile electronic device 100.
  • method 400 further includes (not shown) assisting the camera in acquiring the first mobile electronic device by an encoder and an inertial measurement module communicably coupled to the processor The position and posture.
  • the camera is mounted vertically on the ceiling of the mission area.
  • the first mobile electronic device 100 further includes a charging post 160, wherein the charging post 160 includes an image processor 1040 and a positioning module 1042.
  • the method 400 further includes (not shown) sending the second obstacle information around the first mobile electronic device 100 through the sensor 108. To the motion module 106; and adjusting the motion orientation of the first mobile electronic device 100 by the motion module 106 to avoid the second obstacle.
  • the senor comprises an ultrasonic sensor and/or a laser sensor.

Landscapes

  • Engineering & Computer Science (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Automation & Control Theory (AREA)
  • Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)

Abstract

A first mobile electronic device comprises a wireless signal transceiver, an image processor, a positioning module, and a motion module, wherein the wireless signal transceiver acquires an image of a task area collected by a camera, the task area comprising the first mobile electronic device; the image processor identifies feature information of the first mobile electronic device in the image of the task area, and establishes a coordinate system for the image; the positioning module identifies the position and contour of the first mobile electronic device by comparing the identified feature information of the first mobile electronic device with the coordinate origin on the coordinate system, and draws a map of the task area; a second mobile electronic device can also be connected in a communication manner in order to transmit the map to the second mobile electronic device, and the second mobile electronic device receives a task position set by the user on the map; and a motion module receives the task position from the second mobile electronic device and plans a path on the basis of the task position and the identified position and contour of the first mobile electronic device, and moves according to the path.

Description

一种移动电子设备以及该移动电子设备中的方法Mobile electronic device and method in the mobile electronic device 技术领域Technical field
本发明涉及电子设备领域。具体而言,本发明涉及智能机器人系统领域。The present invention relates to the field of electronic devices. In particular, the invention relates to the field of intelligent robot systems.
背景技术Background technique
传统的扫地机器人按扫描的地图自主定位和移动或者碰撞反弹变向随机行走,同时清扫地面。因此,传统的扫地机器人因为制图和定位技术不成熟或不精确,在工作过程中无法完全判断地面复杂状况,容易出现失去位置与方向的情况。此外,某些机型由于不具备定位能力,只能通过碰撞反弹的物理原理来变向,甚至会造成家居用品或者机器人自身损坏甚至人身伤害,对用户造成干扰等问题。The traditional sweeping robot randomly moves according to the scanned map autonomously positioned and moved or collided, and sweeps the ground at the same time. Therefore, the traditional sweeping robot cannot fully judge the complex situation of the ground during the work process because of the immature or inaccurate drawing and positioning technology, and it is easy to lose the position and direction. In addition, some models can only change direction by the physical principle of collision rebound because they do not have the positioning ability, and even cause damage to the household goods or the robot itself or even personal injury, causing interference to the user.
发明内容Summary of the invention
本发明的实施例所述的移动电子设备系统,例如机器人系统利用安装在室内的固定摄像头,进行定位和绘制地图,以及通过用户的反馈确定任务区域,进行路径规划,实时定位机机器人,从而机器人按照规划的路径进行行走。由此解决了机器人无法判断地面情况、所处位置和最佳移动路线的问题。以固定的摄像头代替机器人的寻迹传感器;以机器人的重复劳动取代人的劳动,节省机器人智能研发和设备的成本以及投资在扫地吸尘机构上的成本。此外,因为摄像头是固定的,因此也消除了设置在机器人内部的摄像头所带来的累积误差,从而实现准确的定位和地图绘制。The mobile electronic device system according to the embodiment of the present invention, for example, the robot system performs positioning and drawing of a map by using a fixed camera installed indoors, and determines a task area by user feedback, performs path planning, and real-time locates the robot, thereby the robot Follow the planned path. This solves the problem that the robot cannot judge the ground condition, the location and the best moving route. Replace the robot's tracking sensor with a fixed camera; replace the human labor with the repeated labor of the robot, save the cost of robot intelligent research and development and equipment, and invest in the cost of sweeping the dust collecting mechanism. In addition, because the camera is fixed, the accumulated error caused by the camera disposed inside the robot is also eliminated, thereby achieving accurate positioning and map drawing.
实施例中所述的机器人系统具有准确的定位和地图绘制,可以提高机器人的工作效率,同时减轻用户的工作负荷。The robot system described in the embodiment has accurate positioning and map drawing, which can improve the working efficiency of the robot and reduce the workload of the user.
根据一个实施例所述的一种第一移动电子设备包括无线信号收发器、图像处理器、定位模块以及运动模块,其中:所述无线信号收发器可通信地连接到安装在室内的摄像头,配置为获取由所述摄像头所采集的包括所述第一移动电子设备的任务区的图像;所述图像处理器可通信地连接至所述无线信号收发器,配置为识别所述任务区的图像中的所述第一移动电子设备的特征信息;所述定位模块可通信地连接至所述图像处理器,配置为通过建立坐标系并比较所识别的第一移动电子设备的特征信息与坐标原点,识别所述第一移动电子设备的位置与轮廓,并绘制所述任务区的地图;所述定位模块还可通信地连接至第二移动电子设备,配置为向所述第二移动电子设备发送所述地图,且所述第二移动电子设备接收来自用户在所述地图上设定的任务位置;以及所述运动模块可通信地连接至所述定位模块和所述第二移动电子设备,配置为接收来自所述第二移动电子设备的所述任务位置,并根据所述任务位置和所识别的所述第一移动电子设备的位置与轮廓,规划路径,并根据所述路径进行运动。A first mobile electronic device according to an embodiment includes a wireless signal transceiver, an image processor, a positioning module, and a motion module, wherein: the wireless signal transceiver is communicably connected to a camera mounted indoors, configured Acquiring an image of a task area comprising the first mobile electronic device acquired by the camera; the image processor is communicably coupled to the wireless signal transceiver, configured to identify an image of the task area Feature information of the first mobile electronic device; the positioning module is communicably coupled to the image processor, configured to establish a coordinate system and compare the identified feature information of the first mobile electronic device with a coordinate origin, Identifying a location and an outline of the first mobile electronic device and mapping a map of the task area; the positioning module is further communicably coupled to the second mobile electronic device, configured to transmit to the second mobile electronic device a map, and the second mobile electronic device receives a task location set by the user on the map; a mobile module communicatively coupled to the positioning module and the second mobile electronic device, configured to receive the mission location from the second mobile electronic device, and based on the mission location and the identified A mobile electronic device is positioned and contoured, a path is planned, and motion is performed according to the path.
可选地或者可替代地,其中所述任务区的图像还包括至少一个第一障碍物,所述图像处理器还配置为识别所述任务区的图像中的所述至少一个第一障碍物的特征信息;所述定位模块还配置为比较所述至少一个第一障碍物的特征信息与坐标原点,识别所述至少一个障碍物的位置与轮廓;所述运动模块还配置为根据所述任务位置、所定位的所述第一移动电子设备的位置和所识别的至少一个第一障碍物的位置与轮廓,规划路径,并根据所述路径进行运动。Optionally or alternatively, wherein the image of the mission area further comprises at least one first obstacle, the image processor further configured to identify the at least one first obstacle in the image of the mission area Feature information; the positioning module is further configured to compare feature information of the at least one first obstacle with a coordinate origin to identify a position and a contour of the at least one obstacle; the motion module is further configured to be according to the task location Positioning the position of the first mobile electronic device and the identified position and contour of the at least one first obstacle, planning a path, and moving according to the path.
可选地或者可替代地,其中所述运动模块正在运动,且所述摄像头实时地采集包括所述第一移动电子设备的任务区的图像;所述图像处理器和所述定位模块还用于动态地确定所述第一移动电子设备的所述特征信息、位置和轮廓。Optionally or alternatively, wherein the motion module is moving, and the camera acquires an image comprising a task area of the first mobile electronic device in real time; the image processor and the positioning module are further The feature information, location, and profile of the first mobile electronic device are dynamically determined.
可选地或者可替代地,所述摄像头垂直安装在所述任务区的天花板上。Alternatively or in the alternative, the camera is mounted vertically on the ceiling of the mission area.
可选地或者可替代地,所述的第一移动电子设备,还包括充电桩,其中所述充电桩包括所述图像处理器和所述定位模块。Optionally or alternatively, the first mobile electronic device further includes a charging post, wherein the charging post comprises the image processor and the positioning module.
可选地或者可替代地,第一移动电子设备还可包含传感器,所述传感器将所述第一移动电子设备周围的第二障碍物信息发送至所述运动模块,所述运动模块还配置为调整所述第一移动电子设备的运动方位以避开所述第二障碍物。Alternatively or in the alternative, the first mobile electronic device may further include a sensor that transmits second obstacle information around the first mobile electronic device to the motion module, the motion module further configured to Adjusting a motion orientation of the first mobile electronic device to avoid the second obstacle.
可选地或者可替代地,所述的第一移动电子设备,所述传感器包括超声波传感器和/或激光传感器。Alternatively or in the alternative, the first mobile electronic device, the sensor comprises an ultrasonic sensor and/or a laser sensor.
根据本发明的另一个发明,一种在第一移动电子设备中的方法,所述第一移动电子设备包括无线信号收发器、图像处理器、定位模块以及运动模块,所述方法包括:通过可通信地连接到安装在室内的至少一个摄像头的所述无线信号收发器,获取由所述至少一个摄像头所采集的包括所述第一移动电子设备在内的任务区的图像;通过可通信地连接至所述无线信号收发器的所述图像处理器,识别所述任务区的图像中的所述第一移动电子设备的特征信息,以及为所述图像建立坐标系,并对所述任务区内的每一个点赋予相应的坐标值;通过可通信地连接至所述图像处理器的所述定位模块,通过在所建立的坐标系上比较所识别的第一移动电子设备的特征信息与坐标原点,识别所述第一移动电子设备的位置与轮廓,并绘制所述任务区的地图;通过可通信地连接至第二移动电子设备的所述定位模块,向所述第二移动电子设备发送所述地图,且所述第二移动电子设备接收来自用户在所述地图上设定的任务位置;以及通过可通信地连接至所述定位模块和所述第二移动电子设备的所述运动模块,接收来自所述第二移动电子设备的所述任务位置,并根据所述任务位置和所识别的所述第一移动电子设备的位置与轮廓,规划路径,并根据所述路径进行运动。According to another invention of the present invention, a method in a first mobile electronic device, the first mobile electronic device comprising a wireless signal transceiver, an image processor, a positioning module, and a motion module, the method comprising: Communicatingly connecting to the wireless signal transceiver of at least one camera installed indoors, acquiring an image of a task area including the first mobile electronic device collected by the at least one camera; communicably connecting And the image processor to the wireless signal transceiver identifying feature information of the first mobile electronic device in an image of the task area, and establishing a coordinate system for the image, and for the task area Each point is assigned a corresponding coordinate value; by comparing the feature information of the first mobile electronic device and the coordinate origin on the established coordinate system by the positioning module communicably connected to the image processor Identifying a location and an outline of the first mobile electronic device, and mapping a map of the task area; communicably connecting to The positioning module of the mobile electronic device transmitting the map to the second mobile electronic device, and the second mobile electronic device receiving a task location set from the user on the map; The motion module coupled to the positioning module and the second mobile electronic device receives the task location from the second mobile electronic device and based on the task location and the identified first movement The position and contour of the electronic device, the planned path, and movement according to the path.
附图说明DRAWINGS
本发明的更完整的理解通过参照关联附图描述的详细的说明书所获得,在附图中相似的附图标记指代相似的部分。A more complete understanding of the present invention is obtained by reference to the detailed description of the accompanying drawings.
图1示出根据本发明的一个实施例的第一移动电子设备和第二移动电子设备所在系统的示意图。1 shows a schematic diagram of a system in which a first mobile electronic device and a second mobile electronic device are located, in accordance with one embodiment of the present invention.
图2示出了根据本发明的一个实施例的第一移动电子设备中的处理器的框图。2 shows a block diagram of a processor in a first mobile electronic device in accordance with one embodiment of the present invention.
图3A-3D示出了根据本发明的一个实施例的第一移动电子设备的示意图。3A-3D show schematic diagrams of a first mobile electronic device in accordance with one embodiment of the present invention.
图4示出了根据本发明的一个实施例的在第一移动电子设备中的方法流程图。4 shows a flow chart of a method in a first mobile electronic device in accordance with one embodiment of the present invention.
具体实施方式detailed description
实施例一Embodiment 1
图1示出根据本发明的一个实施例的第一移动电子设备100和第二移动电子设备120所在系统的示意图。1 shows a schematic diagram of a system in which a first mobile electronic device 100 and a second mobile electronic device 120 are located, in accordance with one embodiment of the present invention.
参照图1,第一移动电子设备100包括但不限于扫地机器人、工业自动化机器人、服务型机器人、排险救灾机器人、水下机器人、空间机器人、自动驾驶汽车上等。当第一移动电子设备100是在室外操作设备如自动驾驶汽车时,摄像头例如,可以由飞在上空的无人机携带。在一个实施例中,第一移动电子设备100本身不包含摄像头。Referring to FIG. 1, the first mobile electronic device 100 includes, but is not limited to, a cleaning robot, an industrial automation robot, a service robot, a disaster relief robot, an underwater robot, a space robot, an autonomous vehicle, and the like. When the first mobile electronic device 100 is in an outdoor operating device such as an autonomous driving car, the camera can be carried by, for example, a drone flying overhead. In one embodiment, the first mobile electronic device 100 does not itself include a camera.
第二移动电子设备120包括但不限于:手机、平板电脑、笔记本电脑、遥控器等。移动电子设备可选地包含操作界面。在一个可选的实施方式中,移动电子设备是手机,操作界面是手机APP。The second mobile electronic device 120 includes, but is not limited to, a mobile phone, a tablet computer, a notebook computer, a remote controller, and the like. The mobile electronic device optionally includes an operator interface. In an optional implementation, the mobile electronic device is a mobile phone, and the operation interface is a mobile phone APP.
第一移动电子设备100与第二移动电子设备120之间的信号传输方式包括但不限于:蓝牙、WIFI、ZigBee、红外、超声波、UWB等,在本实施例中以信号传输方式是WIFI为例进行描述。The signal transmission manner between the first mobile electronic device 100 and the second mobile electronic device 120 includes, but is not limited to, Bluetooth, WIFI, ZigBee, infrared, ultrasonic, UWB, etc., in this embodiment, the signal transmission mode is WIFI as an example. Describe.
任务区表示第一移动电子设备100执行任务的场地。例如,当第一移动电子设备100为扫地机器人时,任务区表示扫地机器人需要清扫的区域。又例如,当第一移动电子设备100为排险救灾机器人时,任务区表示该排险救灾机器人需要抢险的场所。The mission area represents the venue where the first mobile electronic device 100 performs the task. For example, when the first mobile electronic device 100 is a cleaning robot, the mission area represents an area where the cleaning robot needs to be cleaned. For another example, when the first mobile electronic device 100 is a disaster relief robot, the mission area indicates a place where the disaster relief robot needs to be rescued.
如图1所示,在一个实施例中,第一移动电子设备100包括无线信号收发器102、图像处理器1040、定位模块1042以及运动模块106。此外,第一移动电子设备100还包括编码器和惯性测量单元(Inertial Measurement Unit,IMU),这两部分由第一移动电子设备100,也即机器人100自身携带。此外,第一移动电子设备100还可以包括存储器110、数据处理器1046,该两个实体可以在机器人100身上,也可以在充电桩内部,可选地在充电桩内部。存储器110、数据处理器1046这两部功能是存储和处理图像信息、数据信息等。存储器110可通信地连接至处理器104。数据处理器1046位于处理器104中。如在图2中所示,数据处理器1046连接至图像处理器1040和路径规划模块1044。As shown in FIG. 1, in one embodiment, the first mobile electronic device 100 includes a wireless signal transceiver 102, an image processor 1040, a positioning module 1042, and a motion module 106. In addition, the first mobile electronic device 100 further includes an encoder and an Inertial Measurement Unit (IMU), which are carried by the first mobile electronic device 100, that is, the robot 100 itself. In addition, the first mobile electronic device 100 may further include a memory 110, a data processor 1046, which may be on the robot 100 or inside the charging post, optionally inside the charging post. The two functions of the memory 110 and the data processor 1046 are to store and process image information, data information, and the like. Memory 110 is communicably coupled to processor 104. Data processor 1046 is located in processor 104. As shown in FIG. 2, data processor 1046 is coupled to image processor 1040 and path planning module 1044.
无线信号收发器102可通信地连接到安装在室内的摄像头140。无线信号收发器102配置为获取由摄像头140所采集的包括第一移动电子设备100在内的任务区的图像。例如,使用固定在天花板上的摄像头140对室内平面进行俯视角度拍摄。摄像头140中的无线通信模块与第一移动电子设备100中的无线通信模块102通过WIFI连接。The wireless signal transceiver 102 is communicably coupled to a camera 140 mounted indoors. The wireless signal transceiver 102 is configured to acquire an image of the mission area, including the first mobile electronic device 100, captured by the camera 140. For example, the indoor plane is photographed in a bird's-eye view using a camera 140 fixed to the ceiling. The wireless communication module in the camera 140 is connected to the wireless communication module 102 in the first mobile electronic device 100 via WIFI.
图像处理器1040可通信地连接至无线信号收发器102,配置为识别任务区的图像中的第一移动电子设备100的特征信息。此外,图像处理器1040还为平面图像建立二维坐标系,对室内每一个点赋予相应的坐标值。定位模块1042可通信地连接至图像处理器1040,配置为在所建立的坐标 系上比较所识别的第一移动电子设备100的特征信息与坐标原点,识别第一移动电子设备100的位置与轮廓,并绘制任务区的地图。 Image processor 1040 is communicably coupled to wireless signal transceiver 102 and is configured to identify feature information of first mobile electronic device 100 in an image of the mission area. In addition, the image processor 1040 also establishes a two-dimensional coordinate system for the planar image, and assigns corresponding coordinate values to each point in the room. The positioning module 1042 is communicably coupled to the image processor 1040, configured to compare the identified feature information of the first mobile electronic device 100 with a coordinate origin on the established coordinate system to identify the location and contour of the first mobile electronic device 100. And draw a map of the mission area.
例如,摄像头140中拍摄下的室内平面图传至第一移动电子设备100中的图像处理器1040进行处理分析。图像处理器1040的功能是识别图像中的第一移动电子设备100的特征。可选地,图形处理器1040可以采用基于尺度不变特征变换(Scale Invariant Feature Transform,SIFT)算法或加速稳健特征(Speeded Up Robust Features,SURF)算法识别上述特征。采用SIFT算法,需要在存储器110中存储参考图像。图像处理器1040首先识别存储在存储器110中的参考图像的对象的关键点,提取SIFT特征,然后通过比较存储器110中的各个关键点SIFT特征与新采集的图像的SIFT特征,再基于K最邻近算法(K-Nearest Neighbor KNN)的匹配特征,来识别新图像中的对象。SURF算法是基于近似的2D哈尔小波(Haar wavelet)响应,并利用积分图像(integral images)进行图像卷积,使用了基于Hessian矩阵的测度去构造检测子(Hessian matrix-based measure for the detector),并使用了基于分布的描述子(a distribution-based descriptor)。本领域技术人员可以理解,图像处理器1040和定位模块1042可以都包含在处理器104中。以下将结合图2对处理器104进行进一步的描述。For example, the indoor plan view taken in the camera 140 is passed to the image processor 1040 in the first mobile electronic device 100 for processing analysis. The function of image processor 1040 is to identify features of first mobile electronic device 100 in the image. Optionally, the graphics processor 1040 may identify the features by using a Scale Invariant Feature Transform (SIFT) algorithm or a Speeded Up Robust Features (SURF) algorithm. With the SIFT algorithm, a reference image needs to be stored in the memory 110. The image processor 1040 first identifies the key points of the object of the reference image stored in the memory 110, extracts the SIFT features, and then compares the SIFT features of the respective key points in the memory 110 with the SIFT features of the newly acquired image, and then based on the K nearest neighbor The matching feature of the algorithm (K-Nearest Neighbor KNN) to identify objects in the new image. The SURF algorithm is based on an approximate 2D Haar wavelet response and uses an integral image for image convolution using a Hessian matrix-based measure for the detector. And use a distribution-based descriptor. Those skilled in the art will appreciate that image processor 1040 and positioning module 1042 may both be included in processor 104. Processor 104 will be further described below in conjunction with FIG.
定位模块1042可通信地连接至图像处理器1040,配置为通过建立坐标系并比较所识别的第一移动电子设备100的特征信息与坐标原点,识别第一移动电子设备100的位置与轮廓,并绘制任务区的地图。定位模块1042还可通信地连接至第二移动电子设备120,进一步配置为向第二移动电子设备120发送所述地图,且第二移动电子设备120接收来自用户在地图上设定的任务位置。例如,定位模块1042对处理过后的平面图像建立平面直角坐标系,将充电桩所在处设为坐标原点,图像中的每一个点对应一个坐标值(X,Y),标记出第一移动电子设备100的位置与轮廓,实现对室内地图制定。用户的第二移动电子设备120,例如手机120与第一移 动电子设备100的无线信号收发器102通过WIFI连接。手机120的APP读取制定好的室内地图,用户利用手机APP在地图上设定任务位置,也即,单个或者多个需要清扫的位置(对应X,Y坐标信息),第一移动电子设备100的定位模块1042记录下此设定位置信息。同时,第一移动电子设备100利用天花板摄像头140和其内部图像处理模块1040搜寻第一移动电子设备100,例如扫地机器人100的图像,配合定位模块1042对机器人100实时所在地图中的坐标信息(X,Y)进行定位。The positioning module 1042 is communicably coupled to the image processor 1040 and configured to identify the location and contour of the first mobile electronic device 100 by establishing a coordinate system and comparing the identified feature information of the first mobile electronic device 100 with a coordinate origin. Draw a map of the mission area. The location module 1042 is also communicatively coupled to the second mobile electronic device 120, further configured to transmit the map to the second mobile electronic device 120, and the second mobile electronic device 120 receives the task location set from the user on the map. For example, the positioning module 1042 establishes a plane rectangular coordinate system for the processed planar image, sets the location of the charging pile as the coordinate origin, and each point in the image corresponds to a coordinate value (X, Y), marking the first mobile electronic device. The position and contour of the 100 is achieved for the development of indoor maps. The user's second mobile electronic device 120, such as the handset 120, is coupled to the wireless signal transceiver 102 of the first mobile electronic device 100 via WIFI. The APP of the mobile phone 120 reads the prepared indoor map, and the user sets the task position on the map by using the mobile APP, that is, a single or multiple positions to be cleaned (corresponding to X, Y coordinate information), and the first mobile electronic device 100 The positioning module 1042 records the set position information. At the same time, the first mobile electronic device 100 searches for the first mobile electronic device 100, such as the image of the cleaning robot 100, by using the ceiling camera 140 and its internal image processing module 1040, and cooperates with the positioning module 1042 to coordinate information in the real-time map of the robot 100. , Y) for positioning.
然后,运动模块106可通信地连接至定位模块1042和第二移动电子设备120,配置为接收来自第二移动电子设备120的任务位置,并根据任务位置和所识别的所述第一移动电子设备的位置与轮廓,规划路径,并根据所述路径进行运动。The motion module 106 is then communicably coupled to the positioning module 1042 and the second mobile electronic device 120, configured to receive a mission location from the second mobile electronic device 120, and based on the mission location and the identified first mobile electronic device The position and contour, the planning path, and the movement according to the path.
此外,第一移动电子设备100,例如,机器人100还包括机器人编码器和惯性测量模块IMU,以辅助摄像头140获取机器人的位置和姿态。例如当机器人被遮蔽,不在摄像头视线中时,编码器和IMU都还能提供机器人的位置和姿态。例如,编码器可以作为里程计,通过记录机器人轮子的转动信息,来计算机器人100走过的轨迹。In addition, the first mobile electronic device 100, for example, the robot 100 further includes a robot encoder and an inertial measurement module IMU to assist the camera 140 in acquiring the position and posture of the robot. For example, when the robot is shielded from the camera line of sight, both the encoder and the IMU can provide the position and attitude of the robot. For example, the encoder can be used as an odometer to calculate the trajectory of the robot 100 by recording the rotation information of the robot wheel.
可选地或者附加地,任务区的图像还包括至少一个第一障碍物。图像处理器1040还配置为识别任务区的图像中的该至少一个第一障碍物的特征信息。如上所讨论的,图像处理器1040可以使用SIFT或者SURF算法进行图像识别。定位模块1042还配置为比较至少一个第一障碍物的特征信息与坐标原点,识别至少一个障碍物的位置与轮廓。避障模块1048和运动模块106还配置为根据用户利用第二移动电子设备120,例如手机120所选定的任务位置、所定位的第一移动电子设备100的位置和所识别的至少一个第一障碍物的位置与轮廓,规划路径,并根据所述路径进行运动。例如,当清扫位置信息和机器人位置信息都确认之后,第一移动电子设备100中的路径规划模块1044将规划出能避开障碍物的最优路径,将 路径信息通过无线网络传至机器人100的运动模块106,则机器人100按照此路径出发至清扫地点开始任务。Alternatively or additionally, the image of the mission area further includes at least one first obstacle. The image processor 1040 is further configured to identify feature information of the at least one first obstacle in the image of the mission area. As discussed above, image processor 1040 can perform image recognition using SIFT or SURF algorithms. The positioning module 1042 is further configured to compare the feature information of the at least one first obstacle with a coordinate origin to identify the position and contour of the at least one obstacle. The obstacle avoidance module 1048 and the motion module 106 are further configured to determine a task location selected by the user using the second mobile electronic device 120, such as the handset 120, the location of the located first mobile electronic device 100, and the identified at least one first The position and contour of the obstacle, the path is planned, and motion is performed according to the path. For example, after both the cleaning position information and the robot position information are confirmed, the path planning module 1044 in the first mobile electronic device 100 plans an optimal path that can avoid the obstacle, and transmits the path information to the robot 100 through the wireless network. In the motion module 106, the robot 100 starts from this path to the cleaning location to start the task.
可选地或者附加地,运动模块106正在运动,且摄像头140实时地采集包括所述第一移动电子设备的任务区的图像。图像处理器1040和定位模块1042还用于动态地确定第一移动电子设备100的特征信息、位置和轮廓。Alternatively or additionally, the motion module 106 is moving and the camera 140 acquires an image of the mission area including the first mobile electronic device in real time. The image processor 1040 and the positioning module 1042 are also for dynamically determining feature information, location, and contours of the first mobile electronic device 100.
可选地或者附加地,第一移动电子设备100还可包含传感器108,传感器108将第一移动电子设备100周围的第二障碍物信息发送至运动模块106。运动模块106还配置为调整第一移动电子设备100的运动方位以避开所述第二障碍物。可以理解,因为安装的高度不同,安装在天花板上的摄像头140拍摄的俯视信息,而安装在第一移动电子设备100上的传感器获取的是平视信息,因此安装在天花板上的摄像头140所拍摄的障碍物信息与机器人中的传感器的障碍物可能不同,因为可能存在遮蔽。运动模块106综合安装在天花板上的摄像头140所拍摄的第一障碍物信息和传感器所感知的第二障碍物信息,进行避障规划路径。可选地或者可替代地,传感器包括超声波传感器和/或激光传感器。可以理解,第一障碍物和第二障碍物可以是相同的,也可以是不同的。因此,摄像头140和传感器108可以相互辅助。例如,如有遮蔽时,在被遮蔽的局部,机器人100需要靠自身的激光传感器、超声波传感器等来进行避障。Alternatively or additionally, the first mobile electronic device 100 may further include a sensor 108 that transmits second obstacle information around the first mobile electronic device 100 to the motion module 106. The motion module 106 is also configured to adjust the motion orientation of the first mobile electronic device 100 to avoid the second obstacle. It can be understood that because the height of the installation is different, the camera 140 mounted on the ceiling captures the top view information, and the sensor mounted on the first mobile electronic device 100 acquires the head-up information, so the camera 140 mounted on the ceiling captures the image. The obstacle information may be different from the obstacle of the sensor in the robot because there may be shadowing. The motion module 106 integrates the first obstacle information captured by the camera 140 mounted on the ceiling and the second obstacle information sensed by the sensor to perform an obstacle avoidance planning path. Alternatively or in the alternative, the sensor comprises an ultrasonic sensor and/or a laser sensor. It can be understood that the first obstacle and the second obstacle may be the same or different. Therefore, the camera 140 and the sensor 108 can assist each other. For example, if there is shielding, the robot 100 needs to perform obstacle avoidance by its own laser sensor, ultrasonic sensor, or the like in the shaded portion.
例如,在第一移动电子设备100,例如机器人100移动过程中,第一移动电子设备100中的图像处理器1040与定位模块1042利用天花板摄像头140,对机器人100进行动态跟踪,以确保机器人100位置信息准确,同时,机器人100搭载的激光传感器、超声波传感器也对机器人周围静态、动态环境进行检测,辅助避开静态、动态障碍物以及调整最优路径。For example, during the movement of the first mobile electronic device 100, for example, the robot 100, the image processor 1040 and the positioning module 1042 in the first mobile electronic device 100 dynamically track the robot 100 using the ceiling camera 140 to ensure the position of the robot 100. The information is accurate, and the laser sensor and ultrasonic sensor mounted on the robot 100 also detect the static and dynamic environment around the robot, and help avoid static and dynamic obstacles and adjust the optimal path.
可选地或者可替代地,摄像头140垂直安装在任务区的天花板上。 该摄像头140例如可以是广角摄像头,例如,鱼眼摄像头。因此,图像处理器1040使用由摄像头140拍摄的照片时,只需要考虑高度信息,从而对所拍摄的图像进行按比例缩放,就可以得到图像中的目标(例如,第一移动电子设备100,例如机器人与第一障碍物)之间的实际距离。该高度信息可以由高度仪或者测距装置,例如激光测距仪或者红外测距仪所获取,该高度信息可以由第一移动电子设备100所测量,也可以预先存储在第一移动电子设备100中。Alternatively or in the alternative, the camera 140 is mounted vertically on the ceiling of the mission area. The camera 140 can be, for example, a wide-angle camera, such as a fisheye camera. Therefore, when the image processor 1040 uses the photo taken by the camera 140, only the height information needs to be considered, so that the captured image is scaled, and the target in the image can be obtained (for example, the first mobile electronic device 100, for example, The actual distance between the robot and the first obstacle). The height information may be acquired by an altimeter or a distance measuring device, such as a laser range finder or an infrared range finder. The height information may be measured by the first mobile electronic device 100 or may be pre-stored in the first mobile electronic device 100. in.
可选地或者可替代地,摄像头140也可以重用监控摄像头。例如,该监控摄像头140可以安装在墙壁上,则第一移动电子设备100根据三角函数等,得到目标之间的距离。如要使用原有的监控摄像头,需要知道该监控摄像头的离地高度以及镜头角度,从而计算出图像的比例,以得到准确的尺寸数值。Alternatively or in the alternative, the camera 140 may also reuse the surveillance camera. For example, the surveillance camera 140 can be mounted on a wall, and the first mobile electronic device 100 obtains the distance between the targets according to a trigonometric function or the like. If you want to use the original surveillance camera, you need to know the height of the surveillance camera and the angle of the lens to calculate the proportion of the image to get the accurate size value.
可选地或者可替代地,如果单个摄像头的镜头视角有限,可同时使用多个安装在天花板上的摄像头,以覆盖更广的区域,甚至覆盖整个室内。Alternatively or in the alternative, if the lens angle of a single camera is limited, multiple cameras mounted on the ceiling can be used simultaneously to cover a wider area, even covering the entire room.
可选地或者可替代地,第一移动电子设备100还包括充电桩160,其中充电桩160包括处理器104。例如,充电桩包括图像处理器1040和定位模块1042。Alternatively or in the alternative, the first mobile electronic device 100 further includes a charging post 160, wherein the charging post 160 includes a processor 104. For example, the charging station includes an image processor 1040 and a positioning module 1042.
实施例二Embodiment 2
图3A-3D示出了根据本发明的一个实施例的第一移动电子设备100的示意图。以下以第一移动电子设备100为机器人300为例进行说明。3A-3D show schematic diagrams of a first mobile electronic device 100 in accordance with one embodiment of the present invention. Hereinafter, the first mobile electronic device 100 will be described as an example of the robot 300.
参照图3A-3D,机器人300主要由主机体310,无线收发器320,微处理器330,传感器340,和充电桩350五部分组成。用户可通过无线收发器320、微处理器330和充电桩350实现某区域内的定点功能,通过传感器340辅助主机体310完成指定的功能操作。Referring to Figures 3A-3D, the robot 300 is mainly composed of a main body 310, a wireless transceiver 320, a microprocessor 330, a sensor 340, and a charging post 350. The user can implement the fixed point function in a certain area through the wireless transceiver 320, the microprocessor 330 and the charging post 350, and assist the main body 310 through the sensor 340 to perform the specified function operation.
参照图3A和图3B,主机体310为微处理器330和传感器340的载体,微处理器330可以协助主机体310的控制单元执行移动等功能。Referring to FIGS. 3A and 3B, the main body 310 is a carrier of the microprocessor 330 and the sensor 340, and the microprocessor 330 can assist the control unit of the main body 310 to perform functions such as movement.
参照图3C和图3D,无线收发器320可通过手机、电脑等移动电子设备,发送蓝牙、WIFI、ZigBee、红外、超声波等通信信号,也可以与微处理器330和充电桩350进行位置信息交互功能,并将位置信息以指令的方式发送至微处理器330。Referring to FIG. 3C and FIG. 3D, the wireless transceiver 320 can transmit communication signals such as Bluetooth, WIFI, ZigBee, infrared, and ultrasonic waves through mobile electronic devices such as mobile phones and computers, and can also perform position information interaction with the microprocessor 330 and the charging pile 350. The function sends the location information to the microprocessor 330 in an instructional manner.
参照图3B,微处理器330设在主机体310上,可以接收无线收发器320发送的指令并对接收到的指令进行程序处理,将处理后的结果传输至主机体310控制单元,执行相应的任务。微处理器330也可以发射蓝牙、WIFI、ZigBee、红外、超声波等通信信号至充电桩350,实现自动回桩充电功能。Referring to FIG. 3B, the microprocessor 330 is disposed on the host body 310, can receive an instruction sent by the wireless transceiver 320, and performs program processing on the received instruction, and transmits the processed result to the control unit of the host body 310 to execute the corresponding task. The microprocessor 330 can also transmit communication signals such as Bluetooth, WIFI, ZigBee, infrared, ultrasonic, etc. to the charging pile 350 to realize an automatic back-stack charging function.
参照图3A,传感器340可以实时检测周围环境,实现智能避障、通信信号检测与发射的功能。Referring to FIG. 3A, the sensor 340 can detect the surrounding environment in real time, and realize the functions of intelligent obstacle avoidance, communication signal detection and transmission.
参照图3D,充电桩350可以接收无线收发器320发射的信号,同时也可以接收微处理器330发出的通信信号,实现位置信息交互功能,从而引导微处理器330控制主机体310实现回桩充电的功能。Referring to FIG. 3D, the charging post 350 can receive the signal transmitted by the wireless transceiver 320, and can also receive the communication signal sent by the microprocessor 330 to implement the location information interaction function, thereby guiding the microprocessor 330 to control the host body 310 to implement the back-charge charging. The function.
实施例三Embodiment 3
图4示出了根据本发明的一个实施例的在第一移动电子设备中的方法流程图。4 shows a flow chart of a method in a first mobile electronic device in accordance with one embodiment of the present invention.
图4示出了一种在第一移动电子设备中的方法400,其中,第一移动电子设备100包括无线信号收发器102、图像处理器1040、定位模块1042以及运动模块106。该方法400包括:在块410中通过可通信地连接到安装在室内的摄像头140的无线信号收发器102,获取由摄像头140所采集的包括第一移动电子设备100在内的任务区的图像;在块420中通过可通信地连接至无线信号收发器102的图像处理器1040,识别任务区的图像中的第一移动电子设备100的特征信息;在块430中,通过可通信地连 接至图像处理器1040的定位模块1042,通过建立坐标系并比较所识别的第一移动电子设备100的特征信息与坐标原点,识别第一移动电子设备100的位置与轮廓,并绘制所述任务区的地图;在块440中,通过可通信地连接至第二移动电子设备的定位模块1042,向第二移动电子设备发送所述地图,且第二移动电子设备接收来自用户在地图上设定的任务位置;以及在块450中,通过可通信地连接至定位模块1042和第二移动电子设备的运动模块106,接收来自第二移动电子设备的任务位置,并根据任务位置和所识别的第一移动电子设备100的位置与轮廓,规划路径,并根据该路径进行运动。4 illustrates a method 400 in a first mobile electronic device, wherein the first mobile electronic device 100 includes a wireless signal transceiver 102, an image processor 1040, a positioning module 1042, and a motion module 106. The method 400 includes acquiring, in block 410, an image of a mission area, including the first mobile electronic device 100, captured by the camera 140 by a wireless signal transceiver 102 communicably coupled to a camera 140 mounted indoors; The feature information of the first mobile electronic device 100 in the image of the mission area is identified in block 420 by the image processor 1040 communicatively coupled to the wireless signal transceiver 102; in block 430, the image is communicably coupled to the image The positioning module 1042 of the processor 1040 identifies the location and contour of the first mobile electronic device 100 by establishing a coordinate system and comparing the identified feature information of the first mobile electronic device 100 with the coordinate origin, and mapping the task area. In block 440, the map is transmitted to the second mobile electronic device via a positioning module 1042 communicably coupled to the second mobile electronic device, and the second mobile electronic device receives the task location set from the user on the map And in block 450, received by the motion module 106 communicatively coupled to the positioning module 1042 and the second mobile electronic device Task location of the mobile electronic device, and, based on the movement path and the position of the identified task location and a first mobile electronic device 100 and the outline of the planned path.
可选地或者可替代地,其中任务区的图像还包括至少一个第一障碍物,方法400还包括(图中未示出):通过图像处理器1040,识别任务区的图像中的至少一个第一障碍物的特征信息;通过定位模块1042,比较至少一个第一障碍物的特征信息与坐标原点,识别至少一个障碍物的位置与轮廓;通过运动模块106,根据任务位置、所定位的第一移动电子设备100的位置和所识别的至少一个第一障碍物的位置与轮廓,规划路径,并根据所述路径进行运动。Optionally or alternatively, wherein the image of the mission area further comprises at least one first obstacle, the method 400 further comprising (not shown): identifying, by the image processor 1040, at least one of the images of the mission area Feature information of an obstacle; comparing the feature information of the at least one first obstacle with the coordinate origin by the positioning module 1042, identifying the position and contour of the at least one obstacle; and the first position of the positioning according to the task position by the motion module 106 Moving the position of the electronic device 100 and the identified position and contour of the at least one first obstacle, planning a path, and moving according to the path.
可选地或者可替代地,方法400还包括(图中未示出):通过运动模块106进行运动;通过摄像头实时地采集包括第一移动电子设备106在内的任务区的图像;通过图像处理器1040和定位模块1042,动态地确定第一移动电子设备100的特征信息、位置和轮廓。Optionally or alternatively, the method 400 further includes (not shown): moving through the motion module 106; acquiring an image of the mission area including the first mobile electronic device 106 in real time by the camera; The device 1040 and the positioning module 1042 dynamically determine feature information, location, and contours of the first mobile electronic device 100.
可选地或者可替代地,方法400还包括(图中未示出):通过可通信地连接到所述处理器的编码器和惯性测量模块,辅助所述摄像头获取所述第一移动电子设备的位置和姿态。Alternatively or in the alternative, method 400 further includes (not shown) assisting the camera in acquiring the first mobile electronic device by an encoder and an inertial measurement module communicably coupled to the processor The position and posture.
可选地或者可替代地,摄像头垂直安装在所述任务区的天花板上。Alternatively or in the alternative, the camera is mounted vertically on the ceiling of the mission area.
可选地或者可替代地,其中第一移动电子设备100还包括充电桩160,其中充电桩160包括图像处理器1040和定位模块1042。Optionally or alternatively, wherein the first mobile electronic device 100 further includes a charging post 160, wherein the charging post 160 includes an image processor 1040 and a positioning module 1042.
可选地或者可替代地,其中第一移动电子设备100还可包含传感器108,方法400还包括(图中未示出)通过传感器108将第一移动电子设备100周围的第二障碍物信息发送至运动模块106;以及通过运动模块106调整第一移动电子设备100的运动方位以避开第二障碍物。Alternatively or in the alternative, wherein the first mobile electronic device 100 may further include a sensor 108, the method 400 further includes (not shown) sending the second obstacle information around the first mobile electronic device 100 through the sensor 108. To the motion module 106; and adjusting the motion orientation of the first mobile electronic device 100 by the motion module 106 to avoid the second obstacle.
可选地或者可替代地,其中所述传感器包括超声波传感器和/或激光传感器。Alternatively or additionally, wherein the sensor comprises an ultrasonic sensor and/or a laser sensor.
在前面的描述中,已经参考具体示例性实施例描述了本发明;然而,应当理解,在不脱离本文所阐述的本发明的范围的情况下,可以进行各种修改和变化。说明书和附图应以示例性的方式来看待,而不是限制性的,并且所有这些修改旨在被包括在本发明的范围内。因此,本发明的范围应由本文所述的一般实施例及其合法等效物、而不是仅由上述具体实施例来确定。例如,任何方法或过程实施例中所述的步骤可以任何顺序执行,并且不限于在具体实施例中呈现的明确顺序。另外,在任何装置实施例中所述的部件和/或元件可以各种排列组装或以其他方式操作地配置,以产生与本发明基本相同的结果,因此不限于具体实施例中所述的具体配置。In the foregoing description, the invention has been described with reference to the specific embodiments of the embodiments of the present invention, and the various modifications and changes can be made without departing from the scope of the invention as set forth herein. The specification and drawings are to be regarded as illustrative and not restrictive Therefore, the scope of the invention should be determined by the general embodiments described herein and their legal equivalents For example, the steps described in any method or process embodiment can be performed in any order and are not limited to the precise order presented in the particular embodiments. In addition, the components and/or components described in any device embodiment may be assembled in various arrangements or otherwise operatively configured to produce substantially the same results as the present invention, and thus are not limited to the specific embodiments described in the specific embodiments. Configuration.
以上已经关于具体实施例描述了益处、其他优点和问题的解决方案;然而,任何益处、优点或问题的解决方案,或可引起任何特定益处、优点或方案发生或变得更明显的任何元件不应被解释为关键的、必需的或基本的特征或部件。The benefits, other advantages, and solutions of the problems have been described above with regard to specific embodiments; however, any benefit, advantage, or solution of the problem, or any component that can cause any particular benefit, advantage, or solution to occur or become more apparent It should be interpreted as a critical, essential or essential feature or component.
如本文所使用的,术语“包括”、“包含”或其任何变型旨在引用非排他性的包含,使得包括元件列表的过程、方法、物品、组合物或装置不仅包括所述的那些元件,而且也可以包括未明确列出的或固有的主要的过程、方法、物品、组合物或装置。除了未具体叙述的那些之外,在本发明的实践中使用的上述结构、布局、应用、比例、元件、材料或部件的其它组合和/或修改可以被改变,或者以其他方式特别适用于特定的环境、制造规格、设计参数或其他操作要求,而不脱离其大体原则。The term "comprising," "comprising," or any variants thereof, as used herein, is intended to be inclusive of a non-exclusive inclusion, such that a process, method, article, composition, or device comprising a list of elements includes not only those elements described. It is also possible to include the main processes, methods, articles, compositions or devices that are not explicitly listed or inherent. Other combinations and/or modifications of the above-described structures, arrangements, applications, ratios, elements, materials or components used in the practice of the invention may be changed, or otherwise specifically adapted to the specifics. The environment, manufacturing specifications, design parameters or other operational requirements, without departing from the general principles.
虽然本文已经参考某些优选实施例描述了本发明,但是本领域技术人员将容易理解,在不脱离本发明的精神和范围的情况下,其他应用可以替代本文所阐述的那些。因此,本发明仅由下述权利要求书限定。Although the present invention has been described herein with reference to certain preferred embodiments thereof, those skilled in the art can readily understand that other applications may be substituted for those described herein without departing from the spirit and scope of the invention. Accordingly, the invention is limited only by the claims that follow.

Claims (16)

  1. 一种第一移动电子设备,包括无线信号收发器、图像处理器、定位模块以及运动模块,其中:A first mobile electronic device includes a wireless signal transceiver, an image processor, a positioning module, and a motion module, wherein:
    所述无线信号收发器可通信地连接到安装在室内的至少一个摄像头,配置为获取由所述至少一个摄像头所采集的包括所述第一移动电子设备在内的任务区的图像;The wireless signal transceiver is communicably coupled to at least one camera mounted indoors, configured to acquire an image of a mission area including the first mobile electronic device collected by the at least one camera;
    所述图像处理器可通信地连接至所述无线信号收发器,配置为识别所述任务区的图像中的所述第一移动电子设备的特征信息,以及为所述图像建立坐标系,并对所述任务区内的每一个点赋予相应的坐标值;The image processor communicatively coupled to the wireless signal transceiver, configured to identify feature information of the first mobile electronic device in an image of the mission area, and establish a coordinate system for the image, and Each point in the task area is assigned a corresponding coordinate value;
    所述定位模块可通信地连接至所述图像处理器,配置为在所建立的坐标系上比较所识别的第一移动电子设备的特征信息与坐标原点,识别所述第一移动电子设备的位置与轮廓,并绘制所述任务区的地图;The positioning module is communicably coupled to the image processor, configured to compare feature information of the identified first mobile electronic device with a coordinate origin on the established coordinate system, and identify a location of the first mobile electronic device With the outline, and draw a map of the mission area;
    所述定位模块还可通信地连接至第二移动电子设备,配置为向所述第二移动电子设备发送所述地图,且所述第二移动电子设备接收来自用户在所述地图上设定的任务位置;以及The positioning module is also communicably coupled to the second mobile electronic device, configured to transmit the map to the second mobile electronic device, and the second mobile electronic device receives a setting from the user on the map Task location;
    所述运动模块可通信地连接至所述定位模块和所述第二移动电子设备,配置为接收来自所述第二移动电子设备的所述任务位置,并根据所述任务位置和所识别的所述第一移动电子设备的位置与轮廓,规划路径,并根据所述路径进行运动。The motion module is communicably coupled to the positioning module and the second mobile electronic device, configured to receive the task location from the second mobile electronic device, and based on the task location and the identified location The position and contour of the first mobile electronic device are described, the path is planned, and motion is performed according to the path.
  2. 根据权利要求1所述的第一移动电子设备,其中所述任务区的图像还包括至少一个第一障碍物,The first mobile electronic device of claim 1 wherein the image of the mission area further comprises at least one first obstacle,
    所述图像处理器还配置为识别所述任务区的图像中的所述至少一个第一障碍物的特征信息;The image processor is further configured to identify feature information of the at least one first obstacle in an image of the task area;
    所述定位模块还配置为比较所述至少一个第一障碍物的特征信息与坐标原点,识别所述至少一个障碍物的位置与轮廓;The positioning module is further configured to compare feature information of the at least one first obstacle with a coordinate origin, and identify a position and an outline of the at least one obstacle;
    所述运动模块还配置为根据所述任务位置、所定位的所述第一移动 电子设备的位置和所识别的至少一个第一障碍物的位置与轮廓,规划路径,并根据所述路径进行运动。The motion module is further configured to plan a path according to the task location, the located position of the first mobile electronic device, and the identified position and contour of the at least one first obstacle, and perform motion according to the path .
  3. 根据权利要求1所述的第一移动电子设备,其中所述运动模块正在运动,且所述至少一个摄像头实时地采集包括所述第一移动电子设备的任务区的图像;The first mobile electronic device of claim 1, wherein the motion module is moving, and the at least one camera acquires an image including a mission area of the first mobile electronic device in real time;
    所述图像处理器和所述定位模块还用于动态地确定所述第一移动电子设备的所述特征信息、位置和轮廓。The image processor and the positioning module are further configured to dynamically determine the feature information, location, and contour of the first mobile electronic device.
  4. 根据权利要求1所述的第一移动电子设备,还包括The first mobile electronic device of claim 1 further comprising
    可通信地连接到所述处理器的编码器和惯性测量模块,配置为辅助所述至少一个摄像头获取所述第一移动电子设备的位置和姿态。An encoder and an inertial measurement module communicatively coupled to the processor are configured to assist the at least one camera in acquiring a position and attitude of the first mobile electronic device.
  5. 根据权利要求1所述的第一移动电子设备,其中,所述至少一个摄像头垂直安装在所述任务区的天花板上。The first mobile electronic device of claim 1, wherein the at least one camera is mounted vertically on a ceiling of the mission area.
  6. 根据权利要求1-5中任一项所述的第一移动电子设备,还包括充电桩,其中所述充电桩包括所述图像处理器和所述定位模块。A first mobile electronic device according to any one of claims 1 to 5, further comprising a charging post, wherein the charging post comprises the image processor and the positioning module.
  7. 根据权利要求1-5中任一项所述的第一移动电子设备,还可包含传感器,所述传感器将所述第一移动电子设备周围的第二障碍物信息发送至所述运动模块,所述运动模块还配置为调整所述第一移动电子设备的运动方位以避开所述第二障碍物。The first mobile electronic device according to any one of claims 1 to 5, further comprising a sensor that transmits second obstacle information around the first mobile electronic device to the motion module, The motion module is further configured to adjust a motion orientation of the first mobile electronic device to avoid the second obstacle.
  8. 根据权利要求7所述的第一移动电子设备,所述传感器包括超声波传感器和/或激光传感器。The first mobile electronic device of claim 7, the sensor comprising an ultrasonic sensor and/or a laser sensor.
  9. 一种在第一移动电子设备中的方法,所述第一移动电子设备包括无线信号收发器、图像处理器、定位模块以及运动模块,所述方法包括:A method in a first mobile electronic device, the first mobile electronic device comprising a wireless signal transceiver, an image processor, a positioning module, and a motion module, the method comprising:
    通过可通信地连接到安装在室内的至少一个摄像头的所述无线信号收发器,获取由所述至少一个摄像头所采集的包括所述第一移动电子设备在内的任务区的图像;Acquiring an image of a mission area including the first mobile electronic device collected by the at least one camera by the wireless signal transceiver communicably coupled to at least one camera mounted indoors;
    通过可通信地连接至所述无线信号收发器的所述图像处理器,识别所述任务区的图像中的所述第一移动电子设备的特征信息,以及为所述图 像建立坐标系,并对所述任务区内的每一个点赋予相应的坐标值;Identifying, by the image processor communicably coupled to the wireless signal transceiver, feature information of the first mobile electronic device in an image of the mission area, and establishing a coordinate system for the image, and Each point in the task area is assigned a corresponding coordinate value;
    通过可通信地连接至所述图像处理器的所述定位模块,通过在所建立的坐标系上比较所识别的第一移动电子设备的特征信息与坐标原点,识别所述第一移动电子设备的位置与轮廓,并绘制所述任务区的地图;Identifying the first mobile electronic device by comparing the feature information of the identified first mobile electronic device with the coordinate origin on the established coordinate system by the positioning module communicably connected to the image processor Position and outline, and draw a map of the mission area;
    通过可通信地连接至第二移动电子设备的所述定位模块,向所述第二移动电子设备发送所述地图,且所述第二移动电子设备接收来自用户在所述地图上设定的任务位置;以及Transmitting the map to the second mobile electronic device by the positioning module communicably coupled to the second mobile electronic device, and the second mobile electronic device receives a task from the user set on the map Location;
    通过可通信地连接至所述定位模块和所述第二移动电子设备的所述运动模块,接收来自所述第二移动电子设备的所述任务位置,并根据所述任务位置和所识别的所述第一移动电子设备的位置与轮廓,规划路径,并根据所述路径进行运动。Receiving, by the motion module communicatively coupled to the positioning module and the second mobile electronic device, the task location from the second mobile electronic device, and based on the task location and the identified location The position and contour of the first mobile electronic device are described, the path is planned, and motion is performed according to the path.
  10. 根据权利要求9所述的方法,其中所述任务区的图像还包括至少一个第一障碍物,所述方法还包括:The method of claim 9 wherein the image of the mission area further comprises at least one first obstacle, the method further comprising:
    通过所述图像处理器,识别所述任务区的图像中的所述至少一个第一障碍物的特征信息;Identifying, by the image processor, feature information of the at least one first obstacle in an image of the task area;
    通过所述定位模块,比较所述至少一个第一障碍物的特征信息与坐标原点,识别所述至少一个障碍物的位置与轮廓;And comparing, by the positioning module, feature information of the at least one first obstacle with a coordinate origin, and identifying a position and an outline of the at least one obstacle;
    通过所述运动模块,根据所述任务位置、所定位的所述第一移动电子设备的位置和所识别的至少一个第一障碍物的位置与轮廓,规划路径,并根据所述路径进行运动。And, by the motion module, planning a path according to the task position, the located position of the first mobile electronic device, and the identified position and contour of the at least one first obstacle, and performing motion according to the path.
  11. 根据权利要求9所述的方法,还包括:The method of claim 9 further comprising:
    通过所述运动模块进行运动;Performing motion through the motion module;
    通过所述至少一个摄像头实时地采集包括所述第一移动电子设备的任务区的图像;Acquiring, by the at least one camera, an image of a task area including the first mobile electronic device in real time;
    通过所述图像处理器和所述定位模块,动态地确定所述第一移动电子设备的所述特征信息、位置和轮廓。The feature information, location and contour of the first mobile electronic device are dynamically determined by the image processor and the positioning module.
  12. 根据权利要求11所述的方法,还包括:The method of claim 11 further comprising:
    通过可通信地连接到所述处理器的编码器和惯性测量模块,辅助所述至少一个摄像头获取所述第一移动电子设备的位置和姿态。The at least one camera is assisted in acquiring the position and attitude of the first mobile electronic device by an encoder and an inertial measurement module communicably coupled to the processor.
  13. 根据权利要求9-12中任一项所述的方法,其中,所述至少一个摄像头垂直安装在所述任务区的天花板上。A method according to any of claims 9-12, wherein the at least one camera is mounted vertically on the ceiling of the mission area.
  14. 根据权利要求9-12中任一项所述的方法,其中所述第一移动电子设备还包括充电桩,其中所述充电桩包括所述图像处理器和所述定位模块。The method of any of claims 9-12, wherein the first mobile electronic device further comprises a charging post, wherein the charging post comprises the image processor and the positioning module.
  15. 根据权利要求9-12中任一项所述的方法,其中所述第一移动电子设备还可包含传感器,所述方法还包括The method of any of claims 9-12, wherein the first mobile electronic device further comprises a sensor, the method further comprising
    通过所述传感器将所述第一移动电子设备周围的第二障碍物信息发送至所述运动模块;以及Transmitting, by the sensor, second obstacle information around the first mobile electronic device to the motion module;
    通过所述运动模块调整所述第一移动电子设备的运动方位以避开所述第二障碍物。Adjusting a motion orientation of the first mobile electronic device by the motion module to avoid the second obstacle.
  16. 根据权利要求15所述的方法,其中所述传感器包括超声波传感器和/或激光传感器。The method of claim 15 wherein the sensor comprises an ultrasonic sensor and/or a laser sensor.
PCT/CN2018/090176 2017-06-16 2018-06-07 Mobile electronic device and method therein WO2018228258A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710457159.X 2017-06-16
CN201710457159.XA CN108459595A (en) 2017-06-16 2017-06-16 A kind of method in mobile electronic device and the mobile electronic device

Publications (1)

Publication Number Publication Date
WO2018228258A1 true WO2018228258A1 (en) 2018-12-20

Family

ID=63220337

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/090176 WO2018228258A1 (en) 2017-06-16 2018-06-07 Mobile electronic device and method therein

Country Status (2)

Country Link
CN (1) CN108459595A (en)
WO (1) WO2018228258A1 (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110268225B (en) * 2019-05-09 2022-05-10 深圳阿科伯特机器人有限公司 Method for cooperative operation among multiple devices, server and electronic device
CN110162049B (en) * 2019-05-22 2020-06-12 重庆大学 Path finding method, device and storage medium
CN114727220B (en) * 2020-08-05 2022-12-27 华为技术有限公司 Equipment searching method and electronic equipment
CN112797980B (en) * 2020-12-02 2023-03-21 新石器慧义知行智驰(北京)科技有限公司 Indoor unmanned vehicle guiding method and device and electronic equipment

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20090025822A (en) * 2007-09-07 2009-03-11 (주)로봇에버 Method and position data generator for self-localization of robot by using landmark and wireless lan, and robot by using same
CN102866706A (en) * 2012-09-13 2013-01-09 深圳市银星智能科技股份有限公司 Cleaning robot adopting smart phone navigation and navigation cleaning method thereof
CN103926927A (en) * 2014-05-05 2014-07-16 重庆大学 Binocular vision positioning and three-dimensional mapping method for indoor mobile robot
CN104460663A (en) * 2013-09-23 2015-03-25 科沃斯机器人科技(苏州)有限公司 Method for controlling cleaning robot through smart phone
CN105204505A (en) * 2015-09-22 2015-12-30 深圳先进技术研究院 Positioning video acquiring and drawing system and method based on sweeping robot
CN205068153U (en) * 2015-08-07 2016-03-02 浙江海洋学院 Distributing type visual positioning system based on walking robot
WO2017066870A1 (en) * 2015-10-19 2017-04-27 Aseco Investment Corp. Vision-based system for navigating a robot through an indoor space
CN107490377A (en) * 2017-07-17 2017-12-19 五邑大学 Indoor map-free navigation system and navigation method
CN206833252U (en) * 2017-06-16 2018-01-02 炬大科技有限公司 A kind of mobile electronic device

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105467992B (en) * 2015-11-20 2019-11-19 纳恩博(北京)科技有限公司 The determination method and apparatus in mobile electronic device path
CN106595631B (en) * 2016-10-25 2019-08-23 纳恩博(北京)科技有限公司 A kind of method and electronic equipment of avoiding barrier

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20090025822A (en) * 2007-09-07 2009-03-11 (주)로봇에버 Method and position data generator for self-localization of robot by using landmark and wireless lan, and robot by using same
CN102866706A (en) * 2012-09-13 2013-01-09 深圳市银星智能科技股份有限公司 Cleaning robot adopting smart phone navigation and navigation cleaning method thereof
CN104460663A (en) * 2013-09-23 2015-03-25 科沃斯机器人科技(苏州)有限公司 Method for controlling cleaning robot through smart phone
CN103926927A (en) * 2014-05-05 2014-07-16 重庆大学 Binocular vision positioning and three-dimensional mapping method for indoor mobile robot
CN205068153U (en) * 2015-08-07 2016-03-02 浙江海洋学院 Distributing type visual positioning system based on walking robot
CN105204505A (en) * 2015-09-22 2015-12-30 深圳先进技术研究院 Positioning video acquiring and drawing system and method based on sweeping robot
WO2017066870A1 (en) * 2015-10-19 2017-04-27 Aseco Investment Corp. Vision-based system for navigating a robot through an indoor space
CN206833252U (en) * 2017-06-16 2018-01-02 炬大科技有限公司 A kind of mobile electronic device
CN107490377A (en) * 2017-07-17 2017-12-19 五邑大学 Indoor map-free navigation system and navigation method

Also Published As

Publication number Publication date
CN108459595A (en) 2018-08-28

Similar Documents

Publication Publication Date Title
WO2019001237A1 (en) Mobile electronic device, and method in mobile electronic device
KR102242713B1 (en) Moving robot and contorlling method and a terminal
US10809071B2 (en) Method for constructing a map while performing work
WO2018228258A1 (en) Mobile electronic device and method therein
TWI827649B (en) Apparatuses, systems and methods for vslam scale estimation
KR101538775B1 (en) Apparatus and method for localization using forward images
CN108459597B (en) Mobile electronic device and method for processing tasks in task area
US9020301B2 (en) Method and system for three dimensional mapping of an environment
US20160282107A1 (en) System and Method for Measuring by Laser Sweeps
US9183638B2 (en) Image based position determination
CN207488823U (en) A kind of mobile electronic device
CN207115193U (en) A kind of mobile electronic device for being used to handle the task of mission area
JP5775965B2 (en) Stereo camera system and moving body
KR20110122022A (en) Apparatus for building map and method thereof
WO2018228256A1 (en) System and method for determining indoor task target location by image recognition mode
WO2018228254A1 (en) Mobile electronic device and method for use in mobile electronic device
WO2019051832A1 (en) Movable object control method, device and system
KR20130099667A (en) Device and method for estimating location of mobile robot using raiser scanner and structure
CN206833252U (en) A kind of mobile electronic device
KR20200020295A (en) AUGMENTED REALITY SERVICE PROVIDING APPARATUS INTERACTING WITH ROBOT and METHOD OF THEREOF
JP2020149186A (en) Position attitude estimation device, learning device, mobile robot, position attitude estimation method, and learning method
CN113701750A (en) Fusion positioning system of underground multi-sensor
WO2020230410A1 (en) Mobile object
CN212044739U (en) Positioning device and robot based on inertial data and visual characteristics
WO2016048238A1 (en) Method and apparatus for navigation of a robotic device

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18818658

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18818658

Country of ref document: EP

Kind code of ref document: A1