JP2019159354A - Autonomous mobile device, memory defragmentation method and program - Google Patents

Abstract

To handle such an event that memory space is compressed in an improved way.SOLUTION: An autonomous mobile device 100, which creates a map and estimates a position using an image captured by an imaging unit 41, comprises a drive unit 42 for moving a self-machine, a storage unit 20 and a control unit 10. The storage unit 20 stores data on images captured by the imaging unit 41 and also stores data on positions of feature points in an image estimated by the control unit 10 using a plurality of pieces of data on the images stored in the storage unit 20. The control unit 10 controls the drive unit 42 to move the self-machine and deletes deletable data stored in the storage unit 20 at predetermined timing.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an autonomous mobile device, a memory organizing method, and a program.

  Autonomous mobile devices that move autonomously according to their use have become widespread. For example, an autonomous mobile device that moves autonomously for indoor cleaning is known. In general, such an autonomous mobile device needs to create a map of the real space and estimate its own position in the real space.

  As a method for creating a map in real space, for example, a SLAM (Simultaneous Localization And Mapping) method is known. In the SLAM method, the same feature point is tracked from a plurality of frames in a moving image captured by the camera, so that the self position (the three-dimensional position of the own device) and the three-dimensional position of the feature point are alternately estimated. Process. For example, Patent Document 1 discloses a mobile robot that can move while creating a surrounding map using the SLAM technique.

JP, 2006-52515, A

  In the SLAM method, a frame (key frame) used for estimating a three-dimensional position among images (frames) taken by a camera and a feature point (Map point) estimated from a three-dimensional position are stored in a memory. The self-position after movement and the three-dimensional position of a new feature point can be estimated. For this reason, when the SLAM method is used, the data of key frames and map points stored in the memory increases with time, and the memory space of the autonomous mobile device is compressed. Therefore, in the conventional autonomous mobile device, there is room for improvement in handling when the memory space is compressed. In addition to the SLAM method, a technique for self-position estimation using an image captured by an autonomous mobile device with a camera has a similar problem.

  The present invention has been made to solve the above-described problem, and an object of the present invention is to improve the response when the memory space is compressed.

In order to achieve the above object, the autonomous mobile device of the present invention provides:
An autonomous mobile device that creates a map and estimates a position using an image captured by an imaging unit,
A drive unit for moving the machine,
A storage unit;
A control unit;
With
The storage unit stores data related to the image captured by the imaging unit, and the feature points in the image estimated by the control unit using a plurality of data related to the image stored in the storage unit. Data about the location is saved,
The controller is
Move the aircraft by controlling the drive unit,
At a predetermined timing, the deleteable data stored in the storage unit is deleted.

  According to the present invention, it is possible to improve the response when the memory space is compressed.

It is a figure which shows the function structure of the autonomous mobile apparatus which concerns on Embodiment 1 of this invention. It is a figure which shows an example of the environmental map which concerns on Embodiment 1. FIG. 3 is a flowchart of SLAM processing according to the first embodiment. 3 is a flowchart of a self-device position estimation thread according to the first embodiment. 3 is a flowchart of a map creation thread according to the first embodiment. 3 is a flowchart of a loop closing thread according to the first embodiment. 3 is a flowchart of a memory organization process according to the first embodiment. 3 is a flowchart of a memory monitoring thread according to the first embodiment. 3 is a flowchart of an upper application program according to the first embodiment. It is a flowchart of the memory monitoring thread | sled which concerns on Embodiment 2 of this invention. 10 is a flowchart of a host application program according to the second embodiment. 10 is a flowchart of a charger feedback mode of the upper application program according to the second embodiment. It is a flowchart of the high-order application program which concerns on Embodiment 3 of this invention.

  Hereinafter, an autonomous mobile device according to an embodiment of the present invention will be described with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals.

(Embodiment 1)
The autonomous mobile device 100 according to the first embodiment of the present invention is a device that autonomously moves according to the application while creating a surrounding map. Examples of this use include security monitoring, indoor cleaning, pets, and toys.

  As shown in FIG. 1, the autonomous mobile device 100 according to the first embodiment of the present invention includes a control unit 10, a storage unit 20, a sensor unit 30, an imaging unit 41, a drive unit 42, and a communication unit 43.

  The control unit 10 is configured by a CPU (Central Processing Unit) or the like, and by executing a program stored in the storage unit 20, each unit (SLAM processing unit 11, environmental map creation unit 12, erasable data selection unit, which will be described later) 13, the functions of the memory organizing unit 14, the memory monitoring unit 15, and the movement control unit 16) are realized. Moreover, the control part 10 is provided with a timer (not shown), and can count elapsed time.

  The storage unit 20 includes a ROM (Read Only Memory), a RAM (Random Access Memory), and the like, and functionally includes a frame DB (DataBase) 21, a Map point DB 22, and an environment map storage unit 23. The ROM stores a program executed by the CPU of the control unit 10 and data necessary for executing the program in advance. The RAM stores data that is created or changed during program execution.

  In the frame DB 21, an image (frame) photographed by the imaging unit 41 is stored. However, in order to save the storage capacity, it is not necessary to store all captured images. The autonomous mobile device 100 generates data for SLAM processing (map point data described later) and estimates its own position by SLAM processing using a plurality of images stored in the frame DB 21. An image used for estimating the position of the own device is called a key frame, and information about the position of the own device (the position and orientation of the own device) when the key frame is photographed is stored in the frame DB 21 together with information on the image of the key frame. Is also remembered.

  The Map point DB 22 stores information on feature points (referred to as Map points) for which the three-dimensional position (X, Y, Z) is obtained among the feature points included in the key frame stored in the frame DB 21. Is done. A feature point is a point of a characteristic part in an image such as an edge part or a corner part in the image. The feature points can be obtained using an algorithm such as SIFT (Scale-Invariant Feature Transform) or SURF (Speeded Up Robust Features). In the Map point DB 22, the three-dimensional position and the feature amount of the feature point (for example, the feature amount obtained by SIFT or the like) are stored in association with each other as feature point information.

  In the SLAM processing, the correspondence between the same feature points is acquired between a plurality of key frames stored in the frame DB 21, and the three-dimensional position of the acquired corresponding feature points is acquired from the Map point DB 22. Estimate

  The environment map storage unit 23 stores an environment map created by the environment map creation unit 12 based on information from the sensor unit 30. As shown in FIG. 2, the environment map is obtained by dividing the floor surface into, for example, a 5 cm × 5 cm grid and recording the state of the environment (floor surface, obstacle, etc.) corresponding to the grid in units of grids. The environmental state includes, for example, a free space 303 that can pass freely without an obstacle, an obstacle 302 that cannot pass, an unknown space 304 whose state is unknown, and the like. In addition, the position of the charger 301 is also recorded in the environment map.

  The sensor unit 30 includes an obstacle sensor 31. The obstacle sensor 31 is a distance sensor that can detect an object (obstacle) present in the surroundings and measure the distance to the object (obstacle). For example, the obstacle sensor 31 is an infrared distance sensor or an ultrasonic sensor. is there. In addition, you may make it detect an obstruction using the imaging part 41, without mounting the independent obstruction sensor 31. FIG. In this case, the imaging unit 41 also serves as the obstacle sensor 31. Further, the obstacle sensor 31 may be provided with a bumper sensor that detects a collision with another object instead of the distance sensor. In this case, the autonomous mobile device 100 can detect that an obstacle exists at the position where the collision is detected by the bumper sensor.

  The imaging unit 41 includes a monocular imaging device (camera). The imaging unit 41 acquires an image (frame) at 30 fps (frames per second), for example. The autonomous mobile device 100 performs autonomous movement while recognizing its own position and surrounding environment in real time by SLAM processing based on images sequentially acquired by the imaging unit 41.

  The drive unit 42 includes independent two-wheel drive wheels and a motor, and moves the autonomous mobile device 100 according to an instruction (control) from the control unit 10. The autonomous mobile device 100 translates back and forth (translation) by driving the two wheels in the same direction, and rotates (changes direction) on the spot by driving the two wheels in the reverse direction. A swivel movement (translation + rotation (direction change) movement) can be performed by the changed drive. Each wheel is also equipped with a rotary encoder, which measures the rotational speed of the wheel with the rotary encoder and uses the geometric relationship such as the wheel diameter and the distance between the wheels to translate and move. The amount can be calculated.

For example, when the wheel diameter is D and the rotation speed is C, the translational movement amount at the ground contact portion of the wheel is π · D · C. Here, the rotation speed C can be measured by a rotary encoder provided in the drive unit. Further, if the wheel diameter is D, the distance between the wheels is I, the rotation speed of the right wheel is C _R , and the rotation speed of the left wheel is C _L , the rotation amount of the direction change is 360 (when the right rotation is positive). ° × D × (C _L -C _R ) / (2 × I). By sequentially adding the translational movement amount and the rotation amount to each other, the drive unit 42 functions as a mechaodometry, and measures its own position (position and orientation with reference to the position and orientation at the start of movement). be able to. The rotary encoder provided in the drive unit 42 functions as a distance measurement unit.

  In addition, you may make it provide a crawler instead of a wheel, and you may make it move by providing a plurality (for example, two) leg | foot and walking with a leg | foot. In these cases as well, the position and orientation of the aircraft can be measured based on the movements of the two crawlers, the movements of the feet, etc., as in the case of the wheels.

  The communication unit 43 is a module for communicating with an external device, and is a wireless module including an antenna when wirelessly communicating with the external device. For example, the communication unit 43 is a wireless module for performing short-range wireless communication based on Bluetooth (registered trademark). By using the communication unit 43, the autonomous mobile device 100 can exchange data with the outside. Further, the autonomous mobile device 100 may cause a part of the function of the control unit 10 to be executed by an external server by communicating with an external server (not shown) by the communication unit 43.

  Next, a functional configuration of the control unit 10 of the autonomous mobile device 100 will be described. The control unit 10 realizes the functions of the SLAM processing unit 11, the environment map creation unit 12, the erasable data selection unit 13, the memory organization unit 14, the memory monitoring unit 15, and the movement control unit 16, and the movement of the autonomous mobile device 100 Control and so on. Further, the control unit 10 corresponds to the multi-thread function, and can execute a plurality of threads (different processing flows) in parallel.

  The SLAM processing unit 11 uses a plurality of images photographed by the imaging unit 41 and stored in the frame DB 21, and based on feature point information obtained from those images, the SLAM processing unit 11 determines the attitude (position and orientation) of the own device. presume. When this SLAM processing is performed, feature points included in the image are extracted, and information on the Map points is stored in the Map point DB 22 for the feature points (Map points) for which the three-dimensional position can be calculated. Note that the information on the mechadomemetry that can be acquired from the drive unit 42 can also be used to estimate the posture (position and orientation) of the autonomous mobile device 100. Therefore, in the situation where SLAM processing can be performed, information on the position and orientation of the own device (this is also referred to as visual odometry) estimated by the SLAM processing is mainly used.

  The environment map creation unit 12 creates and creates an environment map in which the position of the obstacle 302 is recorded using information on the position and orientation of the own device estimated by the SLAM processing unit 11 and information from the sensor unit 30. The information on the environmental map is written in the environmental map storage unit 23.

  The erasable data selection unit 13 selects erasable data among the data recorded in the frame DB 21 and the Map point DB 22. For example, when there are a plurality of key frames recorded in the frame DB 21 that have similar attitudes (positions and orientations) when the subject is photographed, leave one key frame and leave the rest. Even if deleted, the SLAM processing is not hindered. Further, among the Map points recorded in the Map point DB 22, those having a large three-dimensional position error will adversely affect the SLAM processing unless they are deleted. The erasable data selection unit 13 selects data that does not interfere with SLAM processing even if it is deleted, and data that adversely affects SLAM processing if it is not deleted.

  The memory organizing unit 14 deletes the data selected by the erasable data selecting unit 13 from the frame DB 21 and the Map point DB 22. When deleting these data, the memory organizing unit 14 deletes the data after it is not used in the SLAM processing. By doing so, the memory organizing unit 14 prevents the SLAM processing from being hindered even if these data are deleted.

  When the memory usage of the frame DB 21 and the Map point DB 22 exceeds the reference capacity, or when the time exceeding the reference time has elapsed after deleting the data that can be deleted by the memory organizing unit 14, the memory monitoring unit 15 A memory organization request (deletion request for erasable data) is issued to the memory organization unit 14.

  The movement control unit 16 receives a destination instruction from a higher-level application program, which will be described later, sets a route and a moving speed, and controls the driving unit 42 so as to move the own device along the set route. When the movement control unit 16 sets a route, the route from the current position of the own device to the destination is set based on the environment map created by the environment map creation unit 12.

  The functional configuration of the autonomous mobile device 100 has been described above. Next, the SLAM process of the autonomous mobile device 100 will be described with reference to FIG. The autonomous mobile device 100 is connected to a charger 301 (charging station) when the power is turned off and is charged. When the power is turned on, the SLAM process is started at a position connected to the charger 301. When the autonomous mobile device 100 is turned on, in addition to the SLAM process, a higher-level application program corresponding to the application is started separately (in a separate thread). Receive. The upper application program will be described later.

  First, the control unit 10 of the autonomous mobile device 100 initializes various data (frame DB 21, Map point DB 22, environment map storage unit 23) stored in the storage unit 20 (step S101). Regarding the initialization of the environmental map, when the autonomous mobile device 100 is activated, the autonomous mobile device 100 starts moving from the position of the charger 301. At this time, the environmental map indicates that "the own apparatus exists at the position of the charger". Initialized with information.

  Next, the control unit 10 activates various threads for SLAM processing (step S102). Specifically, the own machine position estimation thread, the map creation thread, and the loop closing thread are activated. When these threads operate in parallel, the SLAM processing unit 11 extracts a feature point from the image captured by the imaging unit 41 and estimates its own position. Details of each thread for SLAM processing will be described later.

  Next, the control unit 10 activates a memory monitoring thread (step S103). Details of the memory monitoring thread will be described later.

  Next, the control unit 10 determines whether or not the operation has ended (an operation end instruction has been received from the upper application program) (step S104). If the operation has been completed (operation completion instruction has been received) (step S104; Yes), the SLAM process is terminated. If the operation has not ended (no operation end instruction has been received) (step S104; No), the environment map creation unit 12 creates and updates the environment map using the sensor unit 30 (step S105).

  Next, the movement control unit 16 receives the destination instruction from the higher-level application program and moves the own machine (step S106). Next, the memory organizing unit 14 performs memory organizing (step S107). Details of the memory organization will be described later. Then, the SLAM process returns to step S104.

  The above is the flow of SLAM processing. Next, various threads for SLAM processing (own position estimation thread, map creation thread, loop closing thread) activated in step S102 of the SLAM processing will be described in order.

  First, the own position estimation thread will be described with reference to FIG. In this thread, the SLAM processing unit 11 first performs initialization processing of its own position estimation, and then estimates its position (position and orientation by visual odometry using images acquired by the imaging unit 41). ) Continue to do.

  First, the SLAM processing unit 11 determines whether or not the operation is finished (step S201). If the operation ends (step S201; Yes), the operation ends. If the operation does not end (step S201; No), it is determined whether an instruction to stop the SLAM processing is given (step S202). If the stop of the SLAM process is instructed (step S202; Yes), the process returns to step S201. If stop of the SLAM process is not instructed (step S202; No), it is determined whether or not the own position estimation process has been initialized (step S203).

  If it has not been initialized (step S203; No), the following initialization process is performed. In the initialization process, the SLAM processing unit 11 first acquires an image by the imaging unit 41, and acquires a feature point from the acquired image (step S204). Next, the SLAM processing unit 11 acquires an image again by the imaging unit 41, and acquires a feature point from the acquired image (step S205). Then, the SLAM processing unit 11 has a feature point correspondence between the two images (the image acquired in step S204 and the image acquired in step S205) (the same point in the real environment exists in each image, What can be dealt with is acquired (step S206). Then, it is determined whether the corresponding number of feature points is less than 5 (step S207).

  If the number of feature point correspondences is less than 5 (step S207; Yes), since the relative posture described later cannot be estimated, the process returns to step S204 to reacquire the initial image. If the number of corresponding feature points is not less than 5 (step S207; No), the relative posture is estimated by using the Two-view Structure from Motion method (step S208). What is estimated here is actually the posture between two images (difference in position (translation vector t) and difference in orientation (rotation matrix R) at which each image was acquired), This is the position (translation vector t) and direction (rotation matrix R) of the own device when the second image is acquired, based on the position and orientation of the own device when the first image is acquired. It can be considered as a relative posture based on the posture of the own device when the first image is acquired. However, since the value of each element of the translation vector t obtained here is different from the value in the real environment, these values are regarded as values in the SLAM space. In the following, the coordinates in the SLAM space (SLAM coordinates) are It explains using.

Next, the SLAM processing unit 11 obtains a three-dimensional position of the feature points (corresponding feature points) corresponding to the two images in the SLAM coordinates (step S209). The method for obtaining the three-dimensional position is as follows. First, attention is paid to one corresponding feature point. Assuming that coordinates (frame coordinates: known) in an image of a certain feature point are (u, v) and a three-dimensional position (unknown) in SLAM coordinates is (X, Y, Z), the following relational expression (1) Can be expressed as
(U v 1) ′ to P (XY Y 1) ′ (1)
Here, the symbol “˜” represents “equal except for a non-zero constant multiple” (that is, equal or is a constant (non-zero) multiple), and the “′” symbol represents “transpose”. .

P is a 3 × 4 perspective projection matrix, a 3 × 3 matrix A (constant determined by the focal length and the image sensor size) indicating internal parameters of the camera of the imaging unit 41, and a rotation matrix indicating the orientation of the image. From R and the translation vector t, it is expressed by the following equation (2).
P = A (R | t) (2)
Here, (R | t) represents a matrix in which the translation column vector t is arranged to the right of the rotation matrix R.

Then, a relational expression between the coordinates (u ₁ , v ₁ ) in the _first image of the corresponding feature point of interest and the three-dimensional position (X, Y, Z) in the SLAM coordinates is based on Expression (1). When it stands, it becomes Formula (3).
(U ₁ v ₁ 1) ′ to A (I | 0) (XY Z 1) ′ (3)
Here, I is a unit matrix, 0 is a zero vector, and (I | 0) is a matrix in which zero vectors represented by column vectors are arranged to the right of the unit matrix I.

Further, a relational expression between the coordinates (u ₂ , v ₂ ) in the _second image of the corresponding feature point and the three-dimensional position (X, Y, Z) in the SLAM coordinates is established based on the formula (1). And Equation (4).
(U ₂ v ₂ 1) ′ to A (R | t) (XY Z 1) ′ (4)

In the above formulas (3) and (4), the three-dimensional positions (X, Y, Z) of the corresponding feature points in the SLAM coordinates are the same, and therefore each of u ₁ , v ₁ , u ₂ , v ₂ Since four equations can be formed and there are three unknowns, X, Y, and Z, these become simultaneous linear equations with excess conditions. In general, there is no solution for the simultaneous linear equations in excess conditions, but the SLAM processing unit 11 uses the least squares method to obtain a three-dimensional position (X, Y, Z) can be determined.

  When the three-dimensional position (X, Y, Z) of the corresponding feature point in the SLAM coordinate is obtained, the SLAM processing unit 11 registers it as the Map point in the Map point DB 22 (Step S210). The elements to be registered in the Map point DB 22 include at least “X, Y, Z which is a three-dimensional position of a feature point in SLAM coordinates” and “a feature amount of the feature point” (for example, a feature amount obtained by SIFT or the like). ) And a “marking flag” (used when marking a map point that can be deleted in the processing of a map creation thread to be described later. For example, the initial value is 0, and 1 when marked). Further, a “time stamp” (a value of a key frame number NKF, which will be described later, at the time of registration in the Map point DB 22) may be included in the registration element in the Map point DB 22.

  Since there are five or more corresponding feature points, the SLAM processing unit 11 repeats step S209 and step S210 for each corresponding feature point.

  Next, the SLAM processing unit 11 initializes a variable NKF indicating the number of a key frame (a frame to be processed in various threads for SLAM processing) to 0 (step S211), and sets an initialized flag (step S211). S212). Then, in order to obtain the scale correspondence between the SLAM coordinates and the real environment coordinates, the SLAM processing unit 11 calculates the translation distance (determined by the coordinates in the real environment) by the mechaodometry in the SLAM coordinates estimated by the above processing. A scale S is obtained by dividing by the translation distance d (step S213).

  Next, the SLAM processing unit 11 registers the second image as a key frame in the frame DB 21 (step S214). The elements to be registered in the frame DB 21 are “key frame number” (NKF value at the time of registration), “posture” (position (translation vector t) in the SLAM coordinates of the device at the time of image capture, and orientation (rotation matrix). R)), “posture in the real environment measured by the mechaodometry” (the position and orientation of the aircraft determined based on the movement distance by the drive unit 42 in the real environment), “all extracted feature points”, These are “the three-dimensional position of the Map point where the three-dimensional position is known among the feature points”, “the characteristic of the key frame itself”, and “the marking flag”.

  Among the above, the “posture in the real environment measured by the mechaodometry” can also be expressed by the translation vector t and the rotation matrix R. However, since the autonomous mobile device 100 normally moves on a two-dimensional plane, It may be simplified to dimension data and expressed as two-dimensional coordinates (X, Y) and direction φ based on the position (origin) and direction at the start of movement. The “feature of the key frame itself” is data for improving the efficiency of obtaining the image similarity between the key frames. For example, a histogram of feature points in the image can be used. May be used as a “characteristic of the key frame itself”. The “marking flag” is used when marking a key frame that can be deleted in the processing of a map creation thread described later. For example, the initial value is 0, and becomes 1 when marked.

  Next, the SLAM processing unit 11 sets the key frame counter NKF in the key frame queue of the map creation thread to notify the map creation thread that the key frame has been generated (step S215). Note that the queue has a first-in first-out data structure.

  Then, the process of the own position estimation thread returns to step S201, determines that it has been initialized in step S203 via step S202 (step S203; Yes), and the SLAM processing unit 11 performs initialization. Proceed to processing in the case of completed.

  Next, processing in the case of initialization is described. This process is a normal process of the own machine position estimation thread, and is a process for sequentially estimating the current position and orientation of the own machine (translation vector t and rotation matrix R in SLAM coordinates).

  The SLAM processing unit 11 captures an image with the image capturing unit 41 and acquires feature points included in the captured image (step S221). Next, from the information of the previous key frame (for example, the image whose key frame number is NKF) registered in the frame DB 21, the three-dimensional position is known among the feature points included in the information of the image. A feature point (which is a Map point registered in the Map point DB 22) is acquired, and a feature point (corresponding feature point) that can be correlated with the image that has just been captured is extracted (step S222).

  Then, the SLAM processing unit 11 determines whether or not the number of corresponding feature points (feature point correspondence number) is less than a reference correspondence feature point number (for example, 10) (step S223), and if it is less than the reference correspondence feature point number (step S223; Yes), the position is not estimated and the process returns to step S221. Here, instead of immediately returning to step S221, the process returns to step S222 to search for key frames registered in the frame DB 21 that have the number of corresponding feature points equal to or greater than the reference corresponding feature number. May be. In this case, if no key feature registered in the frame DB 21 has been found whose number of corresponding feature points is equal to or greater than the reference corresponding feature number, the process returns to step S221.

If the number of corresponding feature points is equal to or greater than the reference corresponding feature point number (step S223; No), the SLAM processing unit 11 acquires the three-dimensional position (X, Y, Z) of each corresponding feature point from the Map point DB 22 (step S224). Then, the coordinates (u _i , v _i ) in the image of the i-th corresponding feature point (i takes a value from 1 to the number of corresponding feature points) included in the image acquired in step S221 are as follows: It should ideally match (ux _i , vx _i ) expressed by equation (5).
(Ux _i , vx _i , 1) ′ to A (R | t) (X _i Y _i Z _i 1) ′ (5)
Here, (X _i , Y _i , Z _i ) is the three-dimensional position of the corresponding feature point, and R and t are the postures (rotation matrix R and R) of the autonomous mobile device 100 when the image is acquired in step S221. Translation vector t).

Actually, since (X _i , Y _i , Z _i ) and (u _i , v _i ) also contain errors, (ux _i , vx _i ) and (u _i , v _i ) match. There is little to do. The unknown number is a rotation matrix R representing a posture and a translation vector t (R and t are three-dimensional in a three-dimensional space, and the number of unknowns is 3 + 3 = 6). Since there are multiple times (because there are equations for u and v of the frame coordinates for one corresponding feature point), it becomes a simultaneous linear equation with excess conditions. Therefore, the SLAM processing unit 11 can obtain the most likely attitude of the autonomous mobile device 100 (translation vector t and rotation matrix R in SLAM coordinates) by using the least square method (step S225).

  Since the current attitude (translation vector t and rotation matrix R) in the SLAM coordinates is obtained, the SLAM processing unit 11 multiplies this by the scale S to obtain VO (visual odometry) (step) S226). The VO can be used as the position and orientation of the aircraft in the real environment.

  Next, the SLAM processing unit 11 captures the reference distance (for example, the current position of the autonomous mobile device 100 from the position when the previous key frame registered in the frame DB 21 (an image with a key frame number of NKF) is captured. 1m) It is determined whether or not it has moved (step S227). If it has moved beyond the reference distance (step S227; Yes), 1 is added to NKF (step S228). Then, the SLAM processing unit 11 registers the current frame as a key frame in the frame DB 21 (step S229), and proceeds to step S215.

  If the autonomous mobile device 100 has moved less than the reference distance (step S227; No), the process returns to step S201. In addition, the movement distance of the autonomous mobile device 100 compared with the reference distance may be obtained from the measurement distance by calculating the translation distance from the previous key frame to the current frame (the absolute value of the difference between the translation vectors of both frames), You may obtain | require from VO (visual odometry) mentioned above. The variable NKF indicating the key frame number, the variable S indicating the scale, the frame DB 21 and the Map point DB 22 are stored in the storage unit 20 so that the values can be referred to across the threads.

  Next, the map creation thread will be described with reference to FIG. In this thread, the SLAM processing unit 11 continues to perform the process of calculating the three-dimensional position of the corresponding feature point in the key frame registered in the frame DB 21 by the own device position estimation thread and registering it in the Map point DB 22.

  First, the SLAM processing unit 11 determines whether or not the operation is finished (step S301). If the operation is finished (step S301; Yes), the process is finished. If the operation is not finished (step S301; No), it is determined whether or not the key frame queue is empty (step S302). If the key frame queue is empty (step S302; Yes), the process returns to step S301. If the key frame queue is not empty (step S302; No), data is extracted from the key frame queue and set to MKF (a variable indicating the key frame number of the key frame to be processed by the map creation thread) (step S303). Next, the SLAM processing unit 11 determines whether MKF is greater than 0 (step S304). If MKF is 0 (step S304; No), the process returns to step S301 to wait for data to enter the key frame queue. If MKF is greater than 0 (step S304; Yes), the following processing is performed.

  The SLAM processing unit 11 refers to the frame DB 21 and corresponds with the feature point of the previous key frame (key frame number is MKF-1) and the feature point of the current key frame (key frame number is MKF). Feature points (corresponding feature points) that can be taken are extracted (step S305). Since the posture (translation vector t and rotation matrix R) of each key frame is also registered in the frame DB 21, the corresponding feature points are three-dimensionally processed in the same manner as in the process of initializing the own position estimation thread. The position can be calculated. The SLAM processing unit 11 registers the corresponding feature point for which the three-dimensional position can be calculated as the Map point in the Map point DB 22 (Step S306). Then, the SLAM processing unit 11 registers the three-dimensional position for the feature point for which the current three-dimensional position can be calculated for the key frame data registered in the frame DB 21 (step S307).

  If the corresponding feature point extracted by the SLAM processing unit 11 has already been registered in the Map point DB 22, the calculation of the three-dimensional position is skipped and the next corresponding feature point (one not registered in the Map point DB) is skipped. The process may proceed, or the 3D position may be calculated again to update the 3D position registered in the Map point DB 22 or the corresponding feature point in the frame DB 21.

  Next, the SLAM processing unit 11 determines whether or not the key frame queue is empty (step S308). If the key frame queue is not empty (step S308; No), the SLAM processing unit 11 sets the MKF in the key frame queue of the loop closing thread (step S312), and returns to step S301.

  If the key frame queue is empty (step S308; Yes), the SLAM processing unit 11 performs bundle adjustment processing on the postures of all key frames and the three-dimensional positions of all map points to improve accuracy (step). S309).

  The bundle adjustment process is a non-linear optimization method that simultaneously estimates the camera posture (key frame posture) and the three-dimensional position of the Map point, and the error that occurs when the Map point is projected onto the key frame is minimized. Optimization is performed so that By performing the bundle adjustment process, it is possible to improve the accuracy of the key frame posture and the three-dimensional position of the Map point.

  Next, the erasable data selection unit 13 obtains an error of the 3D position of the Map point based on the postures of the plurality of key frames, the Map point on the key frame, and the 3D position of the Map point. The Map point where the error is larger than the reference error is marked (step S310). If the posture and the feature point are fixed so as not to be estimated, only the three-dimensional position of the Map point can be estimated by the above-described bundle adjustment processing or the like, and therefore the estimated three-dimensional position and the Map point DB 22 are registered. An error can be calculated based on the three-dimensional position. Note that step S310 is not necessarily performed separately from step S309, and marking may be performed for a Map point having a large error in the bundle adjustment process performed in step S309.

  Next, when there are a plurality of key frames with similar postures in the frame DB 21, the erasable data selection unit 13 marks one of the key frames other than the key frame (S311). The process proceeds to step S312.

  Here, determination of a key frame having a close posture will be described. For example, for two key frames to be determined, the position difference (distance between translation vectors t) of the autonomous mobile device 100 when the key frames are photographed is less than a reference similarity determination distance (for example, 1 m), And when the difference in direction at that time (the difference in angle obtained from the rotation matrix R) is less than the reference similarity determination angle (for example, 5 degrees), it is determined that these key frames are close in posture.

  In addition, when the postures of two key frames are close, a large number of the same Map points exist in those key frames. Therefore, the determination of key frames that are close in posture is performed instead of the difference in position and orientation of the autonomous mobile device 100 when the key frames are captured (or in addition to the difference in position or orientation). It may be determined using the number of common Map points existing in. For example, if “the same Map point has a reference similarity determination score (for example, 100) and the position difference (translation vector difference) is less than the reference similarity determination distance (for example, 1 m)”, it is determined that “the posture is close”. Good.

  Next, the loop closing thread will be described with reference to FIG. In this thread, the SLAM processing unit 11 continues to check whether the loop closing process is possible, and performs the loop closing process if possible. Note that the loop closing process uses the difference between the posture value and the current posture value when you were in the same location before when you recognized that you have returned to the same location that you have been before. This refers to correcting the three-dimensional position of the key frame and the related Map point in the trajectory from the previous time to the present.

  First, the SLAM processing unit 11 determines whether or not the operation is finished (step S401). If the operation is finished (step S401; Yes), the process is finished. If the operation is not finished (step S401; No), it is determined whether or not the key frame queue is empty (step S402). If the key frame queue is empty (step S402; Yes), the process returns to step S401. If the key frame queue is not empty (step S402; No), data is extracted from the key frame queue and set in LKF (a variable indicating the key frame number of the key frame processed by the loop closing thread) (step S403). Next, the SLAM processing unit 11 determines whether LKF is greater than 1 (step S404). If LKF is 0 or 1 (step S404; No), the process returns to step S401 and waits for data to enter the key frame queue. If LKF is larger than 1 (step S404; Yes), the following processing is performed.

  The SLAM processing unit 11 refers to the frame DB 21 and searches the frame DB 21 for a key frame whose similarity between the current key frame (key frame with the key frame number LKF) and the “characteristic of the key frame itself” is equal to or higher than the reference image similarity. Search is performed (step S405). Here, when the feature of an image (key frame) is represented by a feature vector, this similarity is obtained by calculating the inner product of two images obtained by normalizing the norms of the feature vectors of two images to be compared with each other. It can be the similarity of two images. The reciprocal of the distance between the feature vectors of two images (the square root of the sum of squares of the differences between the elements) may be used as the similarity.

  Then, the SLAM processing unit 11 determines whether or not a key frame having a similarity of “characteristic of the key frame itself” equal to or higher than the reference image similarity is found (step S406). If not found (step S406; No), the process returns to step S401, and if found (step S406; Yes), the posture of the key frame in the locus from the found key frame to the current key frame and the key in the locus The three-dimensional position of the Map point included in the frame is corrected (step S407), and the process returns to step S401. In step S407, for example, the SLAM processing unit 11 corrects the posture of the current key frame to the same posture as the found key frame. Then, using the difference between the attitude of the found key frame and the attitude of the current key frame, linear correction is applied to the attitude of each key frame in the trajectory from the found key frame to the current key frame. Further, the three-dimensional position of the Map point included in each key frame is also corrected according to the correction amount of the posture of each key frame.

  Next, the memory organizing process executed in step S107 of the SLAM process (FIG. 3) will be described with reference to FIG. This process is a process of deleting the key frame and the Map point marked by the above-described map creation thread without causing any trouble in the SLAM process.

  First, the memory organizing unit 14 determines whether or not there is a memory organizing instruction from the higher-level application program (step S501). If there is no instruction for organizing the memory (step S501; No), the process is terminated. If there is a memory organization instruction (step S501; Yes), the SLAM processing unit 11 is instructed to stop SLAM processing (step S502).

  Next, the memory organizing unit 14 determines whether or not the SLAM process is stopped (step S503). As described above, the various SLAM processing threads (own position estimation thread, mapping thread, loop closing thread) operate in parallel by the own position estimation thread setting the key frame number in the key frame queue. . When the own position estimation thread receives an instruction to stop the SLAM process and returns from step S202 to step S201, a new key frame is not registered. Then, the mapping thread returns from step S302 to step S301 when the key frame queue is empty, and the loop closing thread returns from step S402 to step S401 when the key frame queue is empty. This state is a state where the SLAM process is stopped. The memory organizing unit 14 determines whether or not the SLAM processing by the SLAM processing unit 11 is in such a stopped state.

  If the SLAM process is not stopped (step S503; No), the process returns to step S503 and waits until it is stopped. If the SLAM process is stopped (step S503; Yes), the memory organizing unit 14 deletes the marked key frame from the frame DB 21 (step S504). Then, the memory organizing unit 14 deletes the marked Map point from the Map point DB 22 (Step S505). Since deletion of these data may cause memory fragmentation, the memory organizing unit 14 performs memory garbage collection as a countermeasure against the fragmentation (step S506). Then, the memory organizing unit 14 notifies the upper application program that the memory organizing is completed (step S507), and ends the process.

  For example, memory garbage collection is performed as follows. First, after all key frames are saved from the frame DB 21 to an empty area in the storage unit 20, the frame DB 21 is cleared. Thereafter, the frame DB 21 is constructed again based on the saved key frame data. Similarly, for the Map point, first, all the Map points are saved from the Map point DB 22 to an empty area in the storage unit 20, and then the Map point DB 22 is cleared. Thereafter, the Map point DB 22 is constructed again based on the saved Map point data. In this way, memory fragmentation can be eliminated and the memory can be used effectively.

  Next, the memory monitoring thread activated in step S103 of the SLAM process (FIG. 3) will be described with reference to FIG. In this thread, the memory usage of the SLAM process is monitored, and memory organization is instructed as necessary. Note that the memory usage of the SLAM process is the amount of data (including those marked) registered in the frame DB 21 and the Map point DB 22.

  First, the memory monitoring unit 15 determines whether or not the operation is finished (step S601). If the operation is finished (step S601; Yes), the process is finished. If the operation is not finished (step S601; No), it is determined whether the memory usage of the SLAM process is larger than the reference capacity (step S602). If the memory usage of the SLAM process is larger than the reference capacity (step S602; Yes), the upper application program is requested to organize the memory (step S603).

  If the memory usage of the SLAM process is not larger than the reference capacity (step S602; No), the memory monitoring unit 15 determines whether or not a time exceeding the reference time has elapsed since the previous memory organization (step S604). ). If the time exceeding the reference time has elapsed since the previous memory organization (step S604; Yes), the process proceeds to step S603. If the time exceeding the reference time has not elapsed since the previous memory organization (step S604; No), the process returns to step S601. In order to allow the memory monitoring unit 15 to make this determination, the control unit 10 records the time at that time when notifying completion of memory organization (step S507) in the memory organization process (FIG. 7). May be. Then, the memory monitoring unit 15 can determine whether or not the time exceeding the reference time has elapsed since the previous memory organization by obtaining the difference between the recorded time and the current time.

  Next, the upper application program will be described. The host application program is software determined according to the use of the autonomous mobile device 100, for example, an indoor cleaning application program. Various host application programs are conceivable depending on applications, but generally, the operation mode is changed according to the situation, and the autonomous mobile device 100 is controlled based on the operation mode. The operation mode defines an action mode of the autonomous mobile device 100. The autonomous mobile device 100 is connected to a charger in a “walk mode” that moves randomly, a “charger return mode” that returns to the charger, and a charger. Then, there are operation modes such as “charging mode” in which charging is performed, “map creation mode” in which the range of creation of environmental maps is expanded, and “conversation mode” in which movement is stopped and conversation with the user is performed.

  And the autonomous mobile apparatus 100 changes an operation mode according to a condition. For example, the initial value is the map creation mode, when the map is created to some extent (for example, 10 minutes in the map creation mode), the walk mode is entered, and when the remaining battery level is low, the charger return mode is entered. Conditions for changing the operation mode may be set in advance, or may be set by an instruction from the outside (user, external server, etc.).

  The upper application program is a program that controls the main operations of the autonomous mobile device 100 and includes various processes. Here, the processing of the upper application program is limited to the part related to memory organization, as shown in FIG. Will be described with reference to FIG. The host application program is started when the autonomous mobile device 100 is turned on, as in the SLAM process (FIG. 3).

  First, the control unit 10 determines whether or not to end the operation of the autonomous mobile device 100 (step S701). For example, the autonomous mobile device 100 ends the operation when receiving an operation end instruction from the user or an operation end instruction from an external server or the like. If the operation is to be terminated (step S701; Yes), the termination of the other thread processing such as SLAM processing is instructed (step S702), and the processing is terminated.

  If the operation is not terminated (step S701; No), the control unit 10 determines whether or not there is a memory organization request from the memory monitoring thread (step S703). If there is a request for memory organization (step S703; Yes), the operation mode is set to “charger feedback mode” (step S704), and the process proceeds to step S705. If there is no request for memory organization (step S703; No), the process proceeds to step S705 as it is.

  In step S <b> 705, the control unit 10 determines whether or not the operation mode is “walking mode”. If it is “walking mode” (step S705; Yes), a destination is set at random (step S706), and the process returns to step S701. If it is not “walk mode” (step S705; No), it is determined whether or not the operation mode is “charger feedback mode” (step S707).

  If the operation mode is not “charger feedback mode” (step S707; No), the control unit 10 sets a destination according to the operation mode (step S708), and returns to step S701. Since there are various destination settings according to the operation mode, some specific examples will be described. For example, if the operation mode is “map creation mode”, a place where the environment map is gradually expanded (for example, the boundary between where the environment map can be created and where it cannot be created) is set as the destination. If the operation mode is “conversation mode”, the movement is stopped and the user talks with the user, so the current position of the autonomous mobile device 100 is set as the destination.

  If the operation mode is “charger feedback mode” (step S707; Yes), the control unit 10 sets the position of the charger as the destination (step S709). And the control part 10 determines whether the autonomous mobile apparatus 100 arrived at the charger (step S710). If it has not arrived at the charger (step S710; No), the process returns to step S701, and the determination in step S710 is repeated until it arrives at the charger. When the battery arrives at the charger (step S710; Yes), the control unit 10 sets the operation mode to “charging mode” (step S711), instructs the SLAM processing unit 11 to organize the memory (step S712), and the SLAM processing unit. 11 until a notification of completion of memory organization is received. Thereafter, the control unit 10 deletes the memory organization request received from the memory monitoring thread (step S713) and returns to step S701.

  Through the various processes described above, the autonomous mobile device 100 can delete data that can be deleted when the memory usage of the SLAM process exceeds the reference capacity. Therefore, even when the autonomous mobile device 100 has few memory resources, it can be used for a long time or in a wide area.

(Modification of Embodiment 1)
In the first embodiment, the memory organizing process (FIG. 7) is performed only in the charging mode. However, if the autonomous mobile device 100 is in the operation mode in which it does not move, the memory organizing process is not performed in the charging mode. May be performed. In such a modified example of the first embodiment, the host application program does not unconditionally set the operation mode to the “charger feedback mode” when a memory organization request is received from the memory monitoring thread. For example, if the operation mode is set to “conversation mode” at that time, the memory organizing process is performed while maintaining the conversation mode. This is because the autonomous mobile device 100 has a conversation with the user without moving in the conversation mode, so that the memory can be organized on the spot without moving to the position of the charger.

  If the battery is already fully charged and it is unnatural to return to the charger, the operation mode is set to a mode for approaching the user for conversation (“user access mode”). When the user arrives near the user, the “conversation mode” may be set to organize the memory. In addition, as an operation mode that can organize the memory immediately on the spot, “stop dance mode” that dances on the spot without moving (for example, on the upper body) and “memory organize mode” that stops to organize the memory are prepared. It is also possible to do.

  Further, the host application program (FIG. 9) displays the charger return cost for moving to the charger position (expected time and expected memory usage required for returning) and the user approach cost for approaching the user for conversation ( After calculating the expected time and the estimated memory usage required to approach, the memory may be instructed after shifting to an operation mode that makes the user feel as uncomfortable as possible.

  In this case, in step S704, the control unit 10 sets the operation mode to the charger feedback mode if the charger feedback cost is smaller than the user approach cost by a predetermined reference cost difference or more, and the user approach cost is less than the charger return cost. If the difference is smaller than a predetermined reference cost difference, the user approach mode is set. If both costs are larger than the predetermined reference cost threshold, the stop dance mode or the memory organization mode is set. When the operation mode is set to the user approach mode, when the user approaches the user, the operation mode is set to the conversation mode, and then memory organization is instructed. When the operation mode is set to the stop dance mode or the memory organizing mode, the memory organizing is instructed immediately on the spot.

  In addition, for example, when the sensor unit 30 includes a cliff sensor that detects a hole or a cliff on the road surface, the autonomous mobile device 100 can determine whether or not it is a dangerous situation to move. In this case, the host application program determines whether or not the autonomous mobile device 100 is currently in a movable state. If the autonomous mobile device 100 is not in a movable state, the operation mode is set to the memory organizing mode and immediately on the spot. Memory organization may be instructed.

(Embodiment 2)
In the first embodiment, the memory organizing process (FIG. 7) is performed only in the charging mode. In this case, the memory monitoring thread requests the host application program to organize the memory, and then returns to the charger. Until this is done, the memory is not organized. Therefore, in the memory monitoring thread, it is necessary to request the upper application program to organize the memory with sufficient memory usage so that the memory usage does not exceed the limit value until it returns to the charger. is there. Therefore, a second embodiment will be described in which memory organization can be performed as soon as memory organization (emergency memory organization) is requested.

  Although the configuration of the autonomous mobile device 101 according to the second embodiment is the same as that of the autonomous mobile device 100, the processing contents of the memory monitoring thread and the higher-level application program are partially different, so these processes will be described.

  As shown in FIG. 10, the processing contents of the memory monitoring thread of the autonomous mobile device 101 are obtained by adding the processing of step S605 and step S606 to the processing content of the memory monitoring thread (FIG. 8) of the autonomous mobile device 100. Yes.

  In step S605, the memory monitoring unit 15 determines whether the memory usage of the SLAM process is larger than the second reference capacity. Here, the second reference capacity is a value that is larger than the reference capacity in step S602 and closer to the limit value of the memory usage. If the memory usage of the SLAM processing is larger than the second reference capacity (step S605; Yes), the memory monitoring unit 15 requests emergency memory organization from the upper application program (step S606), and returns to step S601. If the memory usage of the SLAM process is equal to or smaller than the second reference capacity (step S605; No), the process proceeds to step S602.

  As shown in FIG. 11 and FIG. 12, the processing contents of the upper application program of the autonomous mobile device 101 are the same as the processing contents of the upper application program (FIG. 9) of the autonomous mobile device 100, step S720, step S721, step S722, and step S723. It is what added the processing of.

  In step S720, the control unit 10 determines whether or not there is an emergency memory rearrangement request from the memory monitoring thread. If there is no request for emergency memory rearrangement (step S720; No), the process proceeds to step S703. If there is an emergency memory organization request (step S720; Yes), the control unit 10 stops the movement by stopping the drive unit 42, and sets the operation mode to the memory organization mode (step S721). Then, the control unit 10 instructs the SLAM processing unit 11 to organize the memory (step S722), and waits until receiving a notification of the completion of memory organization from the SLAM processing unit 11. Thereafter, the control unit 10 deletes the emergency memory organization request and the memory organization request received from the memory monitoring thread (step S723), and the process returns to step S701.

  With the above processing, the autonomous mobile device 101 can request “emergency memory cleanup” from the upper application program when the memory usage amount approaches the limit value. Then, the host application program that has received the request for emergency memory organization stops the autonomous mobile device 101 on the spot and shifts to the memory organization mode. Therefore, the autonomous mobile device 101 can execute memory organization immediately, and can be used for a long time or in a wide area even when there are few memory resources.

(Embodiment 3)
In the first embodiment and the second embodiment, the autonomous mobile devices 100 and 101 stop moving when performing the memory organizing process. Therefore, a third embodiment in which the memory organization process can be performed without stopping the movement will be described.

  The autonomous mobile apparatus 102 according to the third embodiment has two own position estimation modes, namely, SLAM mode and mechadomemetry mode, as the own apparatus position estimation modes, in addition to the operation modes described above. The SLAM mode is a mode for estimating its own position by SLAM processing in the same manner as the autonomous mobile device 100, and the mechadomemetry mode estimates its own position based on the mechadomemetry obtained from the drive unit 42 (such as a rotary encoder). Mode). In the mechadomemetry mode, the position of the own aircraft is estimated based on the mekaodometry, so that the process of estimating the own vehicle position by the SLAM process can be stopped. The initial value of the own position estimation mode is the SLAM mode. The configuration of the autonomous mobile device 102 is the same as that of the autonomous mobile device 100, but the processing contents of the higher-level application program are partially different, so this processing will be described.

  As shown in FIG. 13, the processing contents of the upper application program of the autonomous mobile device 102 are the same as the processing contents of the upper application program (FIG. 9) of the autonomous mobile device 100, step S730, step S731, step S732, step S733 and step S734. This process is added, and the processes in steps S704, S712, and S713 are deleted.

  In the autonomous mobile device 102, the control unit 10 determines whether or not there is a memory organization request in step S703. If there is no memory organization request (step S703; No), the control unit 10 proceeds to step S733. If there is a request for memory rearrangement (step S703; Yes), the control unit 10 changes its own position estimation mode to the mechadomemetry mode (step S730), and instructs the SLAM processing unit 11 to perform memory rearrangement (step S731). . Then, the control unit 10 deletes the memory organization request received from the memory monitoring thread (step S732), and proceeds to step S733.

  In step S733, the control unit 10 determines whether or not the memory organization completion notification is received from the SLAM processing unit 11. When the notification of the completion of memory organization is received (step S733; Yes), the control unit 10 changes the own position estimation mode to the SLAM mode (step S734), and proceeds to step S705. And the control part 10 returns to step S701 after step S711.

  In the mecha domemetry mode, the position of the own device can be estimated (measured) without performing the position estimation of the own device by the SLAM process. Therefore, the autonomous mobile device 102 according to the third embodiment can stop the movement without stopping by the above processing. , Memory can be organized.

  The functions of the autonomous mobile devices 100, 101, and 102 can also be implemented by a computer such as a normal PC (Personal Computer). Specifically, in the embodiment described above, the autonomous movement control processing program performed by the autonomous mobile devices 100, 101, and 102 has been described as being stored in advance in the ROM of the storage unit 20. However, the program is stored and distributed on a computer-readable recording medium such as a flexible disk, CD-ROM (Compact Disc Read Only Memory), DVD (Digital Versatile Disc), and MO (Magneto-Optical Disc). A computer capable of realizing each of the functions described above may be configured by reading and installing the program on a computer.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to the specific embodiment which concerns, This invention includes the invention described in the claim, and its equivalent range It is. Hereinafter, the invention described in the scope of claims of the present application will be appended.

(Appendix 1)
An autonomous mobile device that creates a map and estimates a position using an image captured by an imaging unit,
A drive unit for moving the machine,
A storage unit;
A control unit;
With
The storage unit stores data related to the image captured by the imaging unit, and the feature points in the image estimated by the control unit using a plurality of data related to the image stored in the storage unit. Data about the location is saved,
The controller is
Move the aircraft by controlling the drive unit,
Deleting the erasable data stored in the storage unit at a predetermined timing;
Autonomous mobile device.

(Appendix 2)
The controller is
Data related to the image captured by the imaging unit is stored in the storage unit,
As data relating to the position of the feature point, data relating to the three-dimensional position of the feature point is stored in the storage unit,
Estimate the position of the machine using the data related to the image stored in the storage unit and the data related to the three-dimensional position of the feature point,
Data that can be deleted is selected from the data related to the image stored in the storage unit or the data related to the three-dimensional position of the feature point,
Deleting the selected data from the storage unit;
The autonomous mobile device according to attachment 1.

(Appendix 3)
The process of deleting the selected data is performed when the autonomous mobile device is not moving,
The autonomous mobile device according to attachment 2.

(Appendix 4)
The control unit performs processing of a plurality of threads in parallel, and a thread that executes processing for deleting the selected data executes the deletion processing when the storage unit is not used by another thread. To
The autonomous mobile device according to appendix 2 or 3.

(Appendix 5)
The controller is
From among the data related to the three-dimensional position of the feature point stored in the storage unit, the data related to the three-dimensional position of the feature point whose error in the three-dimensional position is larger than a reference error is selected as data that can be deleted.
The autonomous mobile device according to any one of appendices 2 to 4.

(Appendix 6)
The controller is
From the data related to the image stored in the storage unit, extract data related to an image in which the posture of the own device is close when the image is captured,
Selecting data other than one of the extracted image data as deleteable data;
The autonomous mobile device according to any one of appendices 2 to 5.

(Appendix 7)
The controller is
Among the data related to the image stored in the storage unit, the data related to the image having the position difference less than the reference similarity determination distance and the direction difference less than the reference similarity determination angle when the image is captured Extract as data related to the image of the aircraft
The autonomous mobile device according to attachment 6.

(Appendix 8)
The controller is
Among the data related to the image stored in the storage unit, the difference in the position of the own device when the image is taken is less than the reference similarity determination distance, and the feature points having the same three-dimensional position are the reference similarity determination points. Extract data related to existing images as data related to images with close proximity
The autonomous mobile device according to attachment 6.

(Appendix 6)
The drive unit includes a distance measurement unit that measures the distance moved,
The controller is
While deleting the selected data from the storage unit, the distance measurement is not performed using the data related to the image stored in the storage unit and the data related to the three-dimensional position of the feature point. Measure the position of your machine using the
The autonomous mobile device according to any one of appendices 2 to 8.

(Appendix 10)
The autonomous mobile device has a plurality of operation modes including an operation mode without movement,
The controller is
Before the selected data is deleted from the storage unit, the operation mode is changed to an operation mode without the movement;
The autonomous mobile device according to any one of appendices 2 to 9.

(Appendix 11)
The operation mode not involving the movement is a charging mode in which charging is performed by connecting to a charger.
The autonomous mobile device according to attachment 10.

(Appendix 12)
The autonomous mobile device has a plurality of operation modes including an operation mode that stops after a certain time,
The controller is
Before the selected data is deleted from the storage unit, the operation mode is changed to an operation mode that stops after the predetermined time;
The autonomous mobile device according to any one of appendices 2 to 11.

(Appendix 13)
The operation mode that stops after the predetermined time is a charger feedback mode that returns to the charger.
The autonomous mobile device according to attachment 12.

(Appendix 14)
The controller is
While deleting the selected data from the storage unit, the drive unit is stopped so as not to move.
The autonomous mobile device according to any one of appendices 2 to 13.

(Appendix 15)
The controller is
Removing the selected data from the storage unit, and then defragmenting the storage unit;
The autonomous mobile device according to any one of appendices 2 to 14.

(Appendix 16)
A memory organizing method for organizing data stored in a storage unit of an autonomous mobile device that creates a map and estimates a position using an image captured by an imaging unit,
Data related to the image captured by the imaging unit is stored in the storage unit,
Estimating a three-dimensional position of a feature point in the image using a plurality of data relating to the stored image;
Storing data on the estimated three-dimensional position of the feature points in the storage unit;
Data that can be deleted is selected from the data relating to the stored image or the data relating to the three-dimensional position of the feature point,
Deleting the selected data from the storage unit;
Memory organization method.

(Appendix 17)
To the computer of the autonomous mobile device that creates a map and estimates the position using the image captured by the imaging unit,
Save data related to the image captured by the imaging unit in the storage unit,
Estimating a three-dimensional position of a feature point in the image using a plurality of data relating to the stored image;
Storing data on the estimated three-dimensional position of the feature points in the storage unit;
Data that can be deleted is selected from the data relating to the stored image or the data relating to the three-dimensional position of the feature point,
Deleting the selected data from the storage unit;
Program for executing processing.

DESCRIPTION OF SYMBOLS 10 ... Control part, 11 ... SLAM processing part, 12 ... Environmental map creation part, 13 ... Deletable data selection part, 14 ... Memory arrangement part, 15 ... Memory monitoring part, 16 ... Movement control part, 20 ... Memory | storage part, 21 ... frame DB, 22 ... Map point DB, 23 ... environmental map storage unit, 30 ... sensor unit, 31 ... obstacle sensor, 41 ... imaging unit, 42 ... drive unit, 43 ... communication unit, 100, 101, 102 ... autonomous Mobile device 301 ... Charger 302 ... Obstacle 303 ... Free space 304 ... Unknown space

Claims (17)

An autonomous mobile device that creates a map and estimates a position using an image captured by an imaging unit,
A drive unit for moving the machine,
A storage unit;
A control unit;
With
The storage unit stores data related to the image captured by the imaging unit, and the feature points in the image estimated by the control unit using a plurality of data related to the image stored in the storage unit. Data about the location is saved,
The controller is
Move the aircraft by controlling the drive unit,
Deleting the erasable data stored in the storage unit at a predetermined timing;
Autonomous mobile device. The controller is
Data related to the image captured by the imaging unit is stored in the storage unit,
As data relating to the position of the feature point, data relating to the three-dimensional position of the feature point is stored in the storage unit,
Estimate the position of the machine using the data related to the image stored in the storage unit and the data related to the three-dimensional position of the feature point,
Data that can be deleted is selected from the data related to the image stored in the storage unit or the data related to the three-dimensional position of the feature point,
Deleting the selected data from the storage unit;
The autonomous mobile device according to claim 1. The process of deleting the selected data is performed when the autonomous mobile device is not moving,
The autonomous mobile device according to claim 2. The control unit performs processing of a plurality of threads in parallel, and a thread that executes processing for deleting the selected data executes the deletion processing when the storage unit is not used by another thread. To
The autonomous mobile device according to claim 2 or 3. The controller is
From among the data related to the three-dimensional position of the feature point stored in the storage unit, the data related to the three-dimensional position of the feature point whose error in the three-dimensional position is larger than a reference error is selected as data that can be deleted.
The autonomous mobile device according to any one of claims 2 to 4. The controller is
From the data related to the image stored in the storage unit, extract data related to an image in which the posture of the own device is close when the image is captured,
Selecting data other than one of the extracted image data as deleteable data;
The autonomous mobile device according to any one of claims 2 to 5. The controller is
Among the data related to the image stored in the storage unit, the data related to the image having the position difference less than the reference similarity determination distance and the direction difference less than the reference similarity determination angle when the image is captured Extract as data related to the image of the aircraft
The autonomous mobile device according to claim 6. The controller is
Among the data related to the image stored in the storage unit, the difference in the position of the own device when the image is taken is less than the reference similarity determination distance, and the feature points having the same three-dimensional position are the reference similarity determination points. Extract data related to existing images as data related to images with close proximity
The autonomous mobile device according to claim 6. The drive unit includes a distance measurement unit that measures the distance moved,
The controller is
While deleting the selected data from the storage unit, the distance measurement is not performed using the data related to the image stored in the storage unit and the data related to the three-dimensional position of the feature point. Measure the position of your machine using the
The autonomous mobile device according to any one of claims 2 to 8. The autonomous mobile device has a plurality of operation modes including an operation mode without movement,
The controller is
Before the selected data is deleted from the storage unit, the operation mode is changed to an operation mode without the movement;
The autonomous mobile device according to any one of claims 2 to 9. The operation mode not involving the movement is a charging mode in which charging is performed by connecting to a charger.
The autonomous mobile device according to claim 10. The autonomous mobile device has a plurality of operation modes including an operation mode that stops after a certain time,
The controller is
Before the selected data is deleted from the storage unit, the operation mode is changed to an operation mode that stops after the predetermined time;
The autonomous mobile device according to any one of claims 2 to 11. The operation mode that stops after the predetermined time is a charger feedback mode that returns to the charger.
The autonomous mobile device according to claim 12. The controller is
While deleting the selected data from the storage unit, the drive unit is stopped so as not to move.
The autonomous mobile device according to any one of claims 2 to 13. The controller is
Removing the selected data from the storage unit, and then defragmenting the storage unit;
The autonomous mobile device according to any one of claims 2 to 14. A memory organizing method for organizing data stored in a storage unit of an autonomous mobile device that creates a map and estimates a position using an image captured by an imaging unit,
Data related to the image captured by the imaging unit is stored in the storage unit,
Estimating a three-dimensional position of a feature point in the image using a plurality of data relating to the stored image;
Storing data on the estimated three-dimensional position of the feature points in the storage unit;
Data that can be deleted is selected from the data relating to the stored image or the data relating to the three-dimensional position of the feature point,
Deleting the selected data from the storage unit;
Memory organization method. To the computer of the autonomous mobile device that creates a map and estimates the position using the image captured by the imaging unit,
Save data related to the image captured by the imaging unit in the storage unit,
Estimating a three-dimensional position of a feature point in the image using a plurality of data relating to the stored image;
Storing data on the estimated three-dimensional position of the feature points in the storage unit;
Data that can be deleted is selected from the data relating to the stored image or the data relating to the three-dimensional position of the feature point,
Deleting the selected data from the storage unit;
Program for executing processing.

Images