JP6259233B2 - Mobile robot, mobile robot control system, and program - Google Patents

Description

  The present invention relates to a mobile robot that moves autonomously.

  Conventionally, in a mobile robot, a movement range or the like has been limited. For example, Patent Document 1 describes that a virtual wall is defined by preparing a plurality of objects on which a barcode pattern is displayed, thereby restricting the movement of the cleaning robot. Further, for example, Patent Document 2 describes that a paper on which a barcode is printed is pasted on a floor surface at a predetermined interval to guide a cleaning robot or the like. Further, Patent Document 3 describes that a bounded area is defined by landmarks defined by a barcode or the like so that the robot does not move to an adjacent bounded area. Further, Patent Document 4 describes that a cleaning robot detects and records the position of a sign body, and performs cleaning as if the sign body exists even after the sign body is removed. Yes. In addition, by placing infrared emitting parts in the moving space of the cleaning robot, a virtual wall can be provided to limit the entrance of the cleaning robot, or the cleaning robot can automatically return to the home base for charging. To be done.

JP 2012-212490 A International Publication No. 2005/092632 JP 2013-144112 A JP 2012-239897 A

  However, in a conventional mobile robot, it is necessary to place an object or attach a barcode to an area where movement is desired to be restricted or an area where movement is desired. For example, in Patent Document 1, in order to define a virtual line used to limit the movement of the cleaning robot using two or more specific patterns, two or more objects each having two or more specific patterns are virtually displayed. It was necessary to arrange at the position of the line. Also, for example, in Patent Document 4, until the cleaning robot detects and records the position of the sign object, the sign object is arranged in an area that the cleaning robot does not want to pass or the cleaning robot wants to clean. It was necessary to place a sign on the surface. In addition, the virtual wall, the moving route, and the like are determined by a bar code or the like arranged in the real space, and the range thereof cannot be arbitrarily set. In addition, when a component that emits infrared rays is used, the component itself is expensive, and a power source such as a rechargeable battery is necessary for the operation of the component. Further, in the case of control by infrared rays, there is a problem that malfunction may occur due to infrared rays contained in natural light where natural light is incident such as near a window.

  The present invention has been made to solve the above-described problem, and the one used for setting a virtual wall or the like for controlling the movement of the mobile robot may not necessarily be arranged near the virtual wall or the like. The purpose is to do so.

In order to achieve the above object, a mobile robot according to the present invention is a mobile robot that moves autonomously, and is a two-dimensional figure displayed on a sheet, which is a virtual figure that is a virtual figure. Virtual figure information is acquired from a photographing means for photographing a control figure, which is a two-dimensional figure for displaying a code including virtual figure information indicating a relative positional relationship, and a control figure included in a photographed image photographed by the photographing means. The positional relationship for acquiring the positional relationship between the mobile robot and the virtual figure using the acquisition means, the control graphic included in the captured image, the size information indicating the size of the control graphic, and the virtual graphic information acquired by the acquisition means An acquisition means, a movement means for moving the mobile robot, and a movement control means for controlling movement by the movement means in accordance with the positional relationship acquired by the positional relationship acquisition means. It is.
With this configuration, the movement of the mobile robot can be controlled by the virtual graphic information included in the control graphic code. Therefore, since an arbitrary virtual figure can be set based on the virtual figure information, the control figure does not necessarily have to be arranged near the virtual figure, and the degree of freedom in setting the movement control of the mobile robot is increased. For example, the control figure does not necessarily have to be arranged at the position of the virtual wall. For example, a virtual wall having an arbitrary shape or range can be set. Further, since it is not necessary to use a component that emits infrared rays for setting the virtual figure, the virtual figure can be set easily, and it does not malfunction due to natural light.

  Further, in the mobile robot according to the present invention, the positional relationship acquisition means uses coordinate control information indicating coordinate conversion between the coordinate system of the mobile robot and the coordinate system of the control graphic using the control graphic and the size information included in the captured image. And the positional relationship between the mobile robot and the virtual figure may be acquired using the coordinate conversion information and the virtual figure information.

In the mobile robot according to the present invention, the code may also include position / posture information indicating the position and posture of the control graphic, and the acquisition unit may also acquire the position / posture information. In this case, the positional relationship acquisition unit calculates coordinate conversion information indicating coordinate conversion between the coordinate system of the mobile robot and the coordinate system of the control figure using the position and orientation information acquired by the acquisition unit. .
With this configuration, it is possible to specify the position of the mobile robot in an arbitrary coordinate system corresponding to the control figure. Therefore, for example, when the coordinate system corresponding to the control figure corresponds to the coordinate system of the area where the mobile robot exists, the mobile robot uses the control figure to obtain the absolute value of the mobile robot in that area. It is also possible to acquire a correct position.

In the mobile robot according to the present invention, the code also includes size information, the acquisition unit acquires the size information, and the size information used by the positional relationship acquisition unit may be the size information acquired by the acquisition unit. Good.
With such a configuration, even if the size information is different for each control information, the positional information between the mobile robot and the virtual figure information can be appropriately acquired by including the size information in the code. Become.

In the mobile robot according to the present invention, the virtual figure indicates a boundary relating to the movement, and the movement control means may control the movement by the moving means so as to move without exceeding the boundary indicated by the virtual figure. Good.
With such a configuration, a virtual boundary can be set for the mobile robot by a virtual figure.

In the mobile robot according to the present invention, the control figure may be a two-dimensional code.
With such a configuration, it is possible to read out the virtual graphic information and other information included in the virtual graphic code with high accuracy.

A mobile robot control system according to the present invention includes the mobile robot and one or more sheets on which the control graphic used for controlling the movement of the mobile robot is displayed. The movement of the mobile robot is controlled by arranging the displayed sheet in the movement space of the mobile robot.
With such a configuration, the movement of the mobile robot can be controlled by using the sheet on which the control graphic is displayed.

The sheet displaying the control graphic according to the present invention is a sheet displaying the control graphic photographed by the mobile robot, and is a sheet displaying the control graphic displaying a code including virtual graphic information.
With such a configuration, the movement of the mobile robot can be controlled using the virtual graphic information included in the control graphic code displayed on the sheet by arranging the sheet in the moving range of the mobile robot. it can.

  According to the mobile robot and the like according to the present invention, the movement of the mobile robot can be controlled by the virtual graphic information included in the code of the control graphic.

Schematic diagram showing the relationship between the mobile robot and control figures according to Embodiment 1 of the present invention. The block diagram which shows the structure of the mobile robot by the embodiment The figure which shows the relationship between the control figure and the picked-up image in the embodiment A flowchart showing the operation of the mobile robot according to the embodiment The figure which shows an example of a structure of the computer system in the embodiment

  Hereinafter, a mobile robot according to the present invention and a sheet on which a control graphic is displayed will be described using embodiments. In the following embodiments, components and steps denoted by the same reference numerals are the same or equivalent, and repetitive description may be omitted.

(Embodiment 1)
A mobile robot according to Embodiment 1 of the present invention will be described with reference to the drawings. The mobile robot according to the present embodiment is a robot in which control relating to movement is performed using a control graphic which is a two-dimensional graphic.

  FIG. 1 is a diagram showing the relationship between a mobile robot 1 according to the present embodiment, a mobile robot control system including a sheet 3 on which a control graphic 2 is displayed, and a virtual graphic 4. The control graphic 2 is a two-dimensional graphic that displays a code including virtual graphic information. The virtual graphic information is information indicating the relative positional relationship of the virtual graphic 4 that is a virtual graphic with respect to the control graphic 2. The mobile robot 1 captures the control graphic 2 by the image capturing means 11 and uses the control graphic 2 included in the captured image and the size of the control graphic 2 in the real space, so that the control graphic 2 for the mobile robot 1 is captured. Position (vector C) and orientation (for example, Euler angles, rotation matrix corresponding to Euler angles, normal vector of control figure 2, etc. The same applies to other orientations below) in the UVW coordinate system (moving It can be known as information on the local coordinate system of the robot 1. Further, the mobile robot 1 knows the positional relationship of the virtual graphic 4 in the UVW coordinate system, that is, the positional relationship of the virtual graphic 4 with respect to the control graphic 2 by reading the virtual graphic information from the control graphic 2 included in the captured image. it can. As a result, the mobile robot 1 can know the positional relationship between the mobile robot 1 and the virtual graphic 4 on the UVW coordinate system, and can perform movement control according to the positional relationship with the virtual graphic 4. As will be described later, the virtual figure may be, for example, a figure indicating a virtual wall or the like. Further, the mobile robot 1 includes the position (vector B) and orientation of the control figure 2 in the XYZ coordinate system, which is the world coordinate system (an absolute coordinate system fixed in the movement space), in the code of the control figure 2, for example. You may know it. The mobile robot 1 can obtain the transformation between the XYZ coordinate system and the UVW coordinate system from the attitude of the control figure 2 in the XYZ coordinate system and the attitude of the control figure 2 in the UVW coordinate system obtained earlier. Thereby, the vector C in the UVW coordinate system obtained previously can be obtained as a vector in the XYZ coordinate system. As a result, the mobile robot 1 can also know the position (vector A = vector B−vector C) and posture of the mobile robot 1 in the XYZ coordinate system. Hereinafter, details of such a mobile robot 1 will be described.

  FIG. 2 is a block diagram showing a configuration of the mobile robot 1 according to the present embodiment. The mobile robot 1 according to the present embodiment is a robot that moves autonomously, and includes a photographing unit 11, a storage unit 12, an acquisition unit 13, a positional relationship acquisition unit 14, a movement unit 15, and a movement control unit. 16. In addition, autonomously moving means not moving according to an operation instruction received from a user or the like. The mobile robot 1 may be, for example, a cleaning robot, an entertainment robot, a surveillance robot, or a robot for other purposes that moves autonomously.

  The photographing unit 11 photographs the control figure 2 displayed on the sheet 3. The photographing unit 11 can be realized by an image sensor such as a CCD or a CMOS, for example. Further, the photographing means 11 may have an optical system for forming an image of light from the photographing target on the light receiving plane of the image sensor. The data format of the captured image is not limited. Usually, since the photographing unit 11 is fixed to the mobile robot 1, the photographing target of the photographing unit 11 varies depending on the movement of the mobile robot 1. Therefore, the imaging unit 11 may continuously perform imaging, and when the control figure 2 is included in the imaging range, the captured image including the control figure 2 may be accumulated in the storage unit 12. Whether or not the control figure 2 is included in the shooting range may be determined by, for example, pattern matching of the control figure 2 or by other methods.

  Here, the control figure 2 will be described. As described above, the control graphic 2 is a two-dimensional graphic that displays a code. The shape of the control figure 2 may be, for example, a square, a rectangle, other quadrangular shapes, other polygonal shapes, a circular shape, an elliptical shape, a combination thereof, or the like. However, it is preferable that the control figure 2 has three or more points that can be specified. The identifiable point may be, for example, a polygonal vertex, a mark such as a cutout in a circular shape, or an intersection with a long axis or a single axis in an ellipse, Other points that can be specified may be used. For example, the distance between the identifiable points may be indicated by size information described later. In the captured image captured by the imaging unit 11, the control graphic 2 may be uneven or curved within a range where the at least three points and the code including the virtual graphic information can be recognized. Also good. However, the control figure 2 is preferably a plane figure from the viewpoint of ease of recognition of each point or code, ease of creation, ease of sticking, etc., such as paper, film, foil, fabric, etc. A printed material on the sheet 3 is preferable. In addition, the color of the sheet, the background pattern, and the like can be appropriately selected as long as the control figure can be easily recognized from the aesthetics of the usage space. The code displayed on the control graphic 2 includes virtual graphic information. The code may include size information or may include position and orientation information. The code of the control graphic 2 may be, for example, a two-dimensional code or another code. In the former case, the control figure 2 may be a two-dimensional code itself. The two-dimensional code is a code having information in two dimensional directions (for example, a horizontal direction and a vertical direction). For example, the two-dimensional code may be a QR code (registered trademark) which is a matrix type two-dimensional code and is a stack. It may be a two-dimensional code of an expression, or a two-dimensional code of another form. In the present embodiment, a case where the control graphic 2 is a QR code (registered trademark) will be mainly described. When the control figure 2 is not a two-dimensional code, the code including size information or the like may be a character string indicating size information or the like, for example. In that case, the control figure 2 may display a frame such as a rectangle or a polygon and a character string displayed in the frame, for example. The character string may also include numbers and symbols.

The virtual graphic information is information indicating the relative positional relationship of the virtual graphic with respect to the control graphic 2. The virtual figure may be, for example, a solid, a plane or a curved surface, a straight line or a curved line, a point, or a combination thereof. When the virtual figure information indicates a virtual figure that is a plane, the plane of the virtual figure may be a rectangle that exists in a plane that is an extension surface of the control figure 2. When the upper and lower sides of the rectangle are horizontal, and the right and left sides are vertical, the virtual graphic information indicates the distance to each of the four sides of the virtual graphic with reference to the center position of the control graphic 2. It may be the information shown. If the plane of the virtual figure does not exist within the extended plane of the control figure 2, the virtual figure information includes information indicating the position of the virtual figure with respect to the control figure 2 and the attitude of the virtual figure with respect to the control figure 2. Information may be further included. The virtual graphic information may be information indicating the positional relationship of the virtual graphic in the IJK coordinate system or the XYZ coordinate system with the control graphic 2 as a reference, which is in a predetermined positional relationship with the control graphic 2. For example, the virtual figure may be a coordinate value in the XYZ coordinate system, may be a plane represented by a ₁ X + a ₂ Y + a ₃ Z + a ₄ = 0, or the like, and (X−b ₁ ) / c ₁ = A straight line represented by (Y−b ₂ ) / c ₂ = (Z−b ₃ ) / c ₃ or the like may be used, or another figure may be used. Note that a _{1 to} a ₄ , b _{1 to} b ₃ , and c _{1 to} c ₃ are coefficients. The use of virtual graphic information will be described later.

  The size information is information indicating the size of the control figure 2. The size is usually the size in real space. As will be described later, if the position and orientation of the control figure 2 in the coordinate system of the mobile robot 1 can be acquired using the control figure 2 included in the captured image and its size information, Such information may be used. The size information may indicate, for example, the size (length) between a plurality of identifiable points in the control graphic 2. The plurality of identifiable points may exist on the contour of the control graphic 2 or may exist within the contour. That is, the size information may be information indicating a size related to the outline of the control graphic 2 or may be information indicating a size related to an area other than the outline of the control graphic 2. The plurality of points may be three or more points. Further, the size information may not be the size itself as long as the size between a plurality of identifiable points can be known as a result. For example, when the control figure 2 is a square, the size information may be the length of one side. In this case, the length between the diagonal vertices can be calculated using the length of one side. In addition, it is preferable that the correspondence between the distance between identifiable points in the control graphic 2 included in the photographed image and the length in the real space indicated by the size information can be understood. For example, when the control figure 2 is a rectangle and the size information is the length of the long side and the length of the short side, the long side and the short side of the rectangle may not be distinguished in the captured image. Therefore, in such a case, for example, the long side may be determined to be in the horizontal direction, and at least one of the long side and the short side may be distinguished from the other. In the latter case, for example, a mark such as a notch or a protrusion may be provided on one side, and the color of the long side and the short side may be changed. In the present embodiment, a case will be mainly described in which the size information indicates the length of one side of a square QR code (registered trademark). Note that the size information may be included in the code of the control graphic 2 or may be stored in a recording medium (not shown) of the mobile robot 1. In the former case, the size information may be different for each control figure 2. On the other hand, in the latter case, it is necessary to use a control figure 2 having predetermined size information.

  The position / orientation information is information indicating the position and orientation of the control graphic 2. The position and orientation information may be information indicating the position and orientation of the control figure 2 in the XYZ coordinate system of FIG. The position may be, for example, the position of the representative point of the control figure 2 in the XYZ coordinate system. The representative point may be, for example, a center point, a specific vertex, or another point in the control graphic 2. The representative point is preferably a point that can be specified also in the control graphic 2 included in the captured image. Further, as described above, the posture may be an Euler angle of the control graphic 2 with respect to the XYZ coordinate system, a rotation matrix corresponding to the Euler angle, a normal vector of the control graphic 2 in the XYZ coordinate system, or the like.

  Note that the size information and the position / orientation information may not be information indicating the size itself or information indicating the position and orientation itself as long as the information can be obtained as a result of the size, position, and orientation. Specifically, when the control figure 2 is a square or a rectangle, the size, position, and orientation can be acquired if the coordinate values of the four vertices in the three-dimensional coordinate system are known. Therefore, in such a case, the coordinate values of the four vertices of the control figure 2 can be considered as size information and position / orientation information.

  Here, a method for generating the sheet 3 on which the control graphic 2 displaying the code including at least the virtual graphic information is displayed will be briefly described. The user determines the positional relationship between the control graphic 2 and the virtual graphic, and acquires virtual graphic information corresponding to the positional relationship. The virtual graphic information may be a numeric string indicating the position of the virtual graphic in accordance with a predetermined format (for example, a coordinate value, a coefficient of a plane expression, a coefficient of a straight line expression, etc.). And a user produces | generates the control figure 2 which displays the code | symbol containing the virtual figure information by using a computer. For example, when the control graphic 2 is a two-dimensional code, the control graphic 2 that is a two-dimensional code can be generated by encoding the virtual graphic information with an encoder of the two-dimensional code. Then, by printing out the control graphic 2, a sheet 3 on which the control graphic 2 is displayed can be generated. Note that the code of the control graphic 2 may include size information and position / orientation information as described above. For example, since the size of the control graphic 2 may differ depending on the printer or the like, the user specifies size information by measuring the size of the printed control graphic 2 with a ruler or the like, and the size information is You may make it produce | generate the code containing. For example, when the code of the control graphic 2 is a character string, the user prints out the control graphic 2 including the character string indicating the virtual graphic information or the like, or writes the control graphic 2 by handwriting. May be generated.

  The storage unit 12 stores a photographed image photographed by the photographing unit 11. The captured image may have the control graphic 2 as described above. The photographed image may be accumulated by the photographing unit 11, for example. The storage in the storage unit 12 may be temporary storage in a RAM or the like, or may be long-term storage. The storage unit 12 can be realized by a predetermined recording medium (for example, a semiconductor memory, a magnetic disk, an optical disk, etc.).

  The acquisition unit 13 acquires virtual graphic information from the control graphic 2 included in the captured image captured by the imaging unit 11. Specifically, virtual graphic information is acquired from the code of the control graphic 2. When the size information and the position / orientation information are included in the code of the control graphic 2, the acquisition unit 13 may also acquire the size information and the position / orientation information from the control graphic 2 included in the captured image. When the control graphic 2 is a two-dimensional code, the acquisition unit 13 can acquire virtual graphic information and the like by decoding the two-dimensional code. When the control graphic 2 is a code including a character string, the acquisition unit 13 can acquire virtual graphic information or the like by recognizing the character string. Note that the acquisition unit 13 may perform character recognition after removing the distortion by trapezoid correction or the like. The virtual graphic information acquired by the acquisition unit 13 may be stored in a recording medium (not shown).

  The positional relationship acquisition unit 14 uses the control figure 2 included in the photographed image, the size information indicating the size of the control figure 2, and the virtual figure information acquired by the acquisition unit 13. Get the positional relationship. The size information may be acquired by the acquiring unit 13 or may be stored on a recording medium (not shown). The positional relationship between the mobile robot 1 and the virtual graphic may be information indicating the positional relationship of the virtual graphic in the coordinate system of the mobile robot 1. Specifically, the positional relationship acquisition unit 14 uses coordinate control information indicating coordinate conversion between the coordinate system of the mobile robot 1 and the coordinate system of the control graphic 2 using the control graphic 2 and the size information included in the captured image. May be calculated. When the unit of length in the coordinate system of the mobile robot 1 is different from the unit of length of the size information, the positional relationship acquisition unit 14 converts the size information into the unit of length of the coordinate system of the mobile robot 1. It may be used after conversion. The coordinate system of the mobile robot 1 may be, for example, a coordinate system used by the mobile robot 1 for movement control. The coordinate system of the control graphic 2 may be a coordinate system in a predetermined positional relationship with the control graphic 2. The coordinate conversion information may be any information as long as it can perform coordinate conversion of both coordinate systems, for example, information indicating the position of the origin of the other coordinate system in one coordinate system; , Information indicating the orientation of the other coordinate system in one coordinate system (for example, Euler angles, rotation matrices, etc.) may be included. The positional relationship acquisition means 14 may acquire the positional relationship between the mobile robot 1 and the virtual graphic using the coordinate conversion information and the virtual graphic information. When the acquisition unit 13 acquires the position and orientation information, the positional relationship acquisition unit 14 also uses the position and orientation information acquired by the acquisition unit 13 to use the coordinate system of the mobile robot 1 and the coordinate system of the control figure 2. The coordinate conversion information indicating the coordinate conversion between and may be calculated. Information indicating the positional relationship acquired by the positional relationship acquisition unit 14 may be stored in a recording medium (not shown). The details of the positional relationship acquisition process will be described later.

  The moving means 15 moves the mobile robot 1. The method of the movement is not ask | required. The moving means 15 may be, for example, a traveling means that travels on a plane using tires, wheels, an endless track, or the like, may be a legged moving means that the walking robot has, and uses a propeller or the like. It may be a flying means for flying in the air, or may be other means for movement.

  The movement control unit 16 controls the movement by the movement unit 15 according to the positional relationship acquired by the positional relationship acquisition unit 14. That is, the movement control means 16 controls the position of the mobile robot 1 according to the positional relationship of the virtual figure with respect to the mobile robot 1. For example, when the virtual figure indicates a boundary relating to movement, the movement control means 16 may control the movement by the movement means 15 so that the mobile robot 1 moves without exceeding the boundary indicated by the virtual figure. Specifically, the movement control means 16 may acquire the current position of the mobile robot 1 and perform control so that the locus of the current position does not intersect with the boundary indicated by the virtual figure. For example, when the mobile robot 1 is located in front of the boundary, the movement control unit 16 may control the movement unit 15 to change the direction, or may control the movement unit 15 to stop. . Further, for example, when the virtual figure indicates the position to which the mobile robot 1 should pass, the movement control means 16 causes the mobile robot 1 to reach the position indicated by the virtual figure or the route indicated by the virtual figure. The moving means 15 may be controlled so that the mobile robot 1 moves. Specifically, the movement control means 16 may set the target position of the mobile robot 1 to the position indicated by the virtual figure, and may control the movement means 15 so that the mobile robot 1 reaches the position of the virtual figure. Also, for example, when the virtual figure indicates an area where the mobile robot 1 should stay for a longer time, the movement control means 16 determines that the stay time of the mobile robot 1 in the area indicated by the virtual figure is relative to other areas. You may control the moving means 15 so that it may become long. Such control is useful, for example, when the mobile robot 1 that is a cleaning robot wants to carefully clean a specific area, or when the mobile robot 1 that is a monitoring robot wants to carefully monitor a specific area. is there. Moreover, the information which shows the use of a virtual figure is contained in the code | cord | chord which the control figure 2 displays, and the movement control means 16 may perform control according to the use. The use may be, for example, a boundary, a destination, a route, a stay, or the like. It should be noted that the movement control that does not cross the boundary, the movement control that reaches the destination, the movement control that passes the designated route, and the like are already known, and the description thereof is omitted. In addition, the method for acquiring the current position may be, for example, a method using odometry, a method using GPS or the like, or a method using an acceleration sensor or a gyro, Other methods may be used.

Here, a method for acquiring the positional relationship between the mobile robot 1 and the virtual figure by the positional relationship acquisition unit 14 will be described. Here, it is assumed that the control figure 2 is a square two-dimensional code including virtual figure information, size information, and position and orientation information. The size information is assumed to be the length “L” of one side in the real space of the two-dimensional code. The position and orientation information, the control figure 2 and vector ^W p _c indicating the center position of the control figure 2 in the XYZ coordinate system is a predetermined position three-dimensional orthogonal coordinate system in a relationship of the control graphic 2 with respect to the world coordinate system It is assumed that the Euler angles (θ, φ, ψ) of zxz. When the upper left subscript of p is W, it indicates a vector in the world coordinate system, and when it is L, it indicates a vector in the local coordinate system of the mobile robot 1 described later. Shall.

As shown in FIG. 3, the local coordinate system xyz has the viewpoint of the photographing means 11 as the origin, takes the xy coordinates in the direction parallel to the light receiving plane, and takes the z axis in the viewing direction perpendicular to the xy orthogonal coordinate system. 3D orthogonal coordinate system. The four vertices of the control graphic 2 in the photographed image are vertices V1 to V4, and the position vector of the vertex Vi in the local coordinate system is ^L p _si = (p _six , p _siy , f) ^T. However, i = 1, 2, 3, and 4, and T shows transposition. _Further, p _{six, p SiY} is the coordinates on the xy plane of the vertex Vi, obtained by the captured image, to measure the distance in the x-axis and y-axis directions to each vertex V1~V4 from the reference point it can. The reference point is a point on the captured image where x = y = 0 in the local coordinate system. Further, f is a focal length, which is a value determined by the optical system of the photographing unit 11. Note that the length unit is the same in each of the local coordinate system, the world coordinate system, and the real space.

The positions of the four vertices of the control figure 2 in the real space can be expressed as follows using the scalar k _i in the local coordinate system.
^L p _i = k _i ^L p _si (1)
When the size information “L” acquired from the two-dimensional code of the control graphic 2 is used, the following three expressions are obtained.
^{_{| L p 1 - L p 2}} | = L (2)
^{_{| L p 3 - L p 1}} | = L (3)
^{_{| L p 2 - L p 3}} | = 2 1/2 L (4)
Therefore, k ₁ , k ₂ , and k ₃ can be obtained by using the equations (1) to (4). As a result, the position of each vertex of the control figure 2 in the local coordinate system can be obtained from the equation (1). it can be calculated vector ^L p _i.

Also, R _QR (θ, φ, ψ) and R _cam (α, β, γ) are respectively converted into rotation matrices of zxz Euler angles (θ, φ, ψ), (α, β, γ). Suppose that As described above, (θ, φ, ψ) is the Euler angle of zxz indicating the attitude of the control figure 2 that is a QR code (registered trademark) in the world coordinate system, and (α, β, [gamma]) is a Euler angle of zxz indicating the attitude of the light receiving plane of the photographing means 11 in the world coordinate system.

The vectors of the three vertices of the control graphic 2 when the center position of the control graphic 2 is arranged at the origin of the world coordinate system so that the Euler angles (θ, φ, ψ) = (0, 0, 0) are ⁰ respectively. Assuming p ₁ , ⁰ p ₂ , and ⁰ p ₃ , the following equation is obtained.
⁰ p ₁ = (− L / 2, + L / 2, 0) ^T (5)
⁰ p ₂ = (+ L / 2, + L / 2, 0) ^T (6)
⁰ p ₃ = (− L / 2, −L / 2, 0) ^T (7)

Furthermore, the vector ^W p _i of each vertex of the control figure 2 in the world coordinate system can be as follows.
_{^{W p i = R QR 0 p}} i + W p c (8)
Here, ^W _pc is a position vector indicating the center position of the control graphic 2 in the world coordinate system, as described above. ^W _pc and R _QR are information obtained from the position and orientation information included in the code of the control graphic 2, and ⁰ p _i is the size information included in the code of the control graphic 2 and the above (5) to (7 ) Information that can be easily obtained using the above formula.

Further, when both sides of the equation (8) are multiplied by R _cam and arranged, the following equation is obtained for i = 1, 2, 3.
^{_{R cam R QR 0 p i =}} R cam (W p i - W p c)
^{_{= L p i - L p c}} (9)
Here, since ^L _pc = ( ^L p ₂ + ^L p ₃ ) / 2, the Euler angles (α, β, γ) can be obtained by solving the equation (9) using this. . Further, if a vector from the origin of the local coordinate system to the center position of the control figure 2 in the world coordinate system is ^W p _d ,
^{_{W p d = R cam -1 ·}} L p c (10)
It becomes. Therefore, the vector ^W p _cam from the origin of the world coordinate system to the origin of the local coordinate system can be calculated as follows:
^{_{W p cam = W p c -}} W p d
^{_{= W p c -R cam -1 ·}} L p c (11)
The above description is an example of calculating ^W p _cam and R _cam , and they can be calculated by other methods. For example, using the position vector ^L p _i of each vertex of the control figure 2 in the local coordinate system, it is also possible to calculate the Euler angles indicating the posture of a control graphic 2 in the local coordinate system, by using it, It is also possible to calculate R _{cam and the} like.

Thus, since the vector ^W p _cam and the rotation matrix R _cam which are coordinate conversion information indicating the coordinate conversion between the world coordinate system and the local coordinate system can be obtained, the virtual figure of the world coordinate system can be obtained using them. Can be converted into virtual figure information in the local coordinate system. Therefore, the positional relationship acquisition means 14 can acquire the positional relationship of the virtual graphic in the local coordinate system, that is, the positional relationship between the mobile robot 1 and the virtual graphic.

  Control of the mobile robot 1 using two or more control figures 2 is also possible. In that case, a virtual figure can be set for each control figure 2. For example, for the virtual graphic indicated by the virtual graphic information included in the code of the first control graphic 2, the positional relationship in the local coordinate system of the mobile robot 1 is acquired, and the virtual graphic included in the code of the second control graphic 2 As for the virtual figure indicated by the information, the mobile robot 1 is controlled in accordance with the two virtual figures by acquiring the positional relationship of the mobile robot 1 in the local coordinate system. Further, the code of one control figure 2 may include virtual figure information indicating two or more virtual figures. The same virtual figure may be set by two or more control figures 2. If the control graphic 2 is not photographed, control according to the virtual graphic information included in the code of the control graphic 2 cannot be performed. Therefore, it is preferable to set the same virtual figure by two or more control figures 2 in order to increase the possibility that the control figure 2 is photographed. In that case, it is preferable that those control figures 2 are arranged at different positions.

  In addition, although the case where the coordinate system of the mobile robot 1 is a coordinate system in which the viewpoint is the origin and the light receiving plane is parallel to the xy plane has been described, this need not be the case. Separately, a coordinate system of the mobile robot 1 may exist. In that case, by performing coordinate conversion between the above-described xyz coordinate system and the coordinate system of the mobile robot 1, the positional relationship of the virtual figure in the above-described xyz coordinate system can be changed to the It can be converted into a positional relationship. Further, when the shooting direction and the shooting position can be changed in the mobile robot 1 (for example, when the shooting means 11 is panned or tilted in the mobile robot 1 in order to search for the control figure 2), the mobile robot 1 moves. The xyz coordinate system and the coordinate system of the mobile robot 1 may be converted in consideration of the reference position of the robot 1, the viewpoint position with respect to the reference plane, and the posture of the light receiving plane.

In the above description, the case where the position and orientation of the control figure 2 are indicated by the position and orientation information included in the two-dimensional code has been described. The position and orientation information may be determined in advance. The advance position and orientation information determined, for ^example, W _p c = ^(0,0,0) is a _^T, when _{R QR} is three-dimensional matrix, i.e., the center position of the control figure 2 is the world coordinate It may be the origin of the system and the control figure 2 is on the XY plane. When _RQR is a three-dimensional unit matrix, the Euler angles are usually (θ, φ, ψ) = (0, 0, 0). Further, other predetermined position / orientation information may be used. In this case, for example, the predetermined position / orientation information may be included in the code of the control graphic 2, or the predetermined position / orientation information is stored in a recording medium (not shown) of the mobile robot 1. The positional relationship acquisition unit 14 may store the position / orientation information by reading it out. Here, as described above, the advantages when the position and orientation of the control figure 2 are indicated by the position and orientation information included in the two-dimensional code will be briefly described. In that case, for example, the origin of the world coordinate system can be used as a reference point (for example, a reference point of a room) of the moving space of the mobile robot 1. When the mobile robot 1 holds map information of the moving space in the world coordinate system in advance, the mobile robot 1 calculates ^W p _cam and R _cam using the captured control figure 2. , You can know the position on the map. Therefore, in that case, the mobile robot 1 can know not only the position of the virtual figure but also its own position by using the control figure 2.

Next, the operation of the mobile robot 1 will be described using the flowchart of FIG. In this flowchart, a case where virtual graphic information, size information, and position and orientation information are included in the code of the control graphic 2 will be described.
(Step S101) The photographing means 11 determines whether or not the control graphic 2 is included in the photographed image. If the control graphic 2 is included, the process proceeds to step S102; otherwise, the process of step S101 is repeated until the control graphic 2 is included in the captured image. Note that the photographing unit 11 may determine whether or not the control figure 2 has been photographed by pattern matching or the like as described above.

  (Step S102) The photographing means 11 determines whether or not the control graphic 2 included in the photographed image has already acquired virtual graphic information and the like. If the control graphic 2 has already acquired virtual graphic information or the like, the process returns to step S101 without accumulating the captured image. If not, the captured image is stored in the storage unit 12. Accumulate and proceed to step S103. The photographing means 11 stores an image of the control graphic 2 photographed in the past on a recording medium (not shown), and the control graphic 2 included in the photographed image is one of the control graphics 2 stored on the recording medium. If they match, it may be determined that the control graphic 2 has already acquired virtual graphic information or the like. In this case, the photographing unit 11 accumulates a newly photographed image of the control figure 2 in a recording medium (not shown) when proceeding to step S103.

  (Step S103) The acquisition unit 13 acquires virtual graphic information and the like from the code of the control graphic 2 included in the captured image stored in the storage unit 12. At the time of acquisition, the acquisition unit 13 may perform processing such as decoding and character recognition. Further, the acquired virtual graphic information and the like may be stored in a recording medium (not shown).

(Step S104) The positional relationship acquisition means 14 acquires coordinate conversion information related to coordinate conversion between the coordinate system of the mobile robot 1 and the coordinate system corresponding to the control figure 2. The acquired coordinate conversion information may be stored in a recording medium (not shown). Specifically, the positional relationship acquisition unit 14 uses the size information, the position of the vertex of the control figure 2 in the captured image, and the expressions (1) to (4) to calculate ^L p ₁ , ^L p ₂ , ^L p ₃ is calculated, and ^L _pc is calculated using the result. Further, the positional relationship acquisition unit 14 calculates R _QR from information indicating the posture of the control figure 2 included in the position and posture information, and the R _QR and the equations (5) to (7) and (9), The rotation matrix R _cam is obtained by calculating the Euler angles (α, β, γ) of the light receiving plane of the imaging unit 11 using the size information. The positional relationship acquisition means 14, (11) and, using the center position ^W p _c of the control graphic 2 included in the position and orientation information, the calculated rotation matrix _{R ^cam,} calculates the W _{p cam.} In this manner, the positional relationship acquisition unit 14 can acquire the vector ^W p _cam and the rotation matrix R _cam that are coordinate conversion information.

(Step S <b> 105) The positional relationship acquisition unit 14 performs a coordinate conversion according to the coordinate conversion information on the virtual figure indicated by the virtual figure information acquired by the acquisition unit 13, thereby generating a virtual figure in the coordinate system of the mobile robot 1. Information indicating the positional relationship is acquired. Specifically, the positional relationship acquisition means 14 changes the positional relationship of the virtual graphic in the world coordinate system indicated by the virtual graphic information to the positional relationship of the virtual graphic in the local coordinate system using the vector ^W p _cam and the rotation matrix R _cam. Convert.

(Step S <b> 106) The positional relationship acquisition unit 14 sets control information, which is information indicating the positional relationship of virtual figures in the coordinate system of the mobile robot 1, to the movement control unit 16. Then, the process returns to step S101.
Although not included in the flowchart of FIG. 4, the movement control unit 16 controls movement by the movement unit 15 in accordance with the set control information. Further, the moving unit 15 moves the mobile robot 1 in accordance with the control of the movement control unit 16. As a result, for example, the mobile robot 1 moves so as not to exceed the virtual graphic indicating the boundary, or moves so as to follow the virtual graphic indicating the route. In the flowchart of FIG. 4, the process is terminated by power-off or a process termination interrupt.

Next, experimental examples will be described. In this experiment, QR code (registered trademark) was used as a two-dimensional code on the top of the tripod, and a paper sheet 3 on which a control figure 2 was printed by QR code (registered trademark) was fixed. It is assumed that the following information is included in the two-dimensional code. In this specific example, a virtual figure exists in the extended surface of the control figure 2, and the next t, b, l, and r are the upper side, the lower side, the left side, and the right side of the virtual figure from the center position of the control figure 2, respectively. It is said that the length is up to. The unit of length is mm.
Virtual figure information: (t, b, l, r) = (500, 1000, 2000, 800)
Position and orientation information:
Center position (X, Y, Z) of control figure 2 = (12000, 2500, 1200)
Attitude of control figure 2 (θ, φ, ψ) = (90 °, 0, 0)
Size information: L = 180

  As the photographing mechanism of the photographing means 11, a digital camera (SONY DSC-T9) having a 6M pixel, 1 / 2.5-type CCD was used. The focal length of the digital camera was set to 6.33 mm. Further, the photographing means 11 is arranged at a position of about 3 m from the two-dimensional code. Then, using the captured image captured by the digital camera, virtual figure information, position and orientation information, and size information are acquired (steps S101 to S103), and the coordinate conversion information is acquired as described above, whereby the mobile robot 1 The position of the virtual figure in the local coordinate system was calculated (steps S104 and S105). As a result, the virtual figure was a virtual surface having four sides at a distance corresponding to the above-described t, b, l, and r from the center position of the two-dimensional code. Therefore, by controlling the mobile robot 1 using the virtual plane, for example, the movement range of the mobile robot 1 can be limited.

  In this specific example, the case where the sheet 3 is paper has been described. However, the sheet 3 may be anything as long as it can display the control figure 2 which is a two-dimensional figure. Good. The sheet 3 may be, for example, a film, a foil, a film, a woven fabric, a plate, a plate, or the like, or may be another capable of displaying the control graphic 2.

Moreover, although the case where a user produces the sheet | seat 3 on which the control figure 2 was displayed was demonstrated, it may not be so. For example, a plurality of sheets 3 may be prepared in advance, and the user may select a sheet 3 that meets the purpose from among them and place it in the movement space of the mobile robot 1. Specifically, the following set of sheets 3 may be prepared.
Information about virtual figures (virtual walls) Number of sheets No entry to the rear 5 No entry after 1m forward 2 No entry within 2m radius 2 No entry 30cm behind, 1m each in the left and right range 2 pieces 30cm behind, 30m left 1m range entry prohibition 2 sheets 30cm behind, 2m area entry prohibition left 2 sheets 30cm behind, right entry 1m area entry prohibition 2 sheets 30cm behind, 2m area entry prohibition 2m right entry prohibition 2 sheets No entry into the range of 1m to the right direction 2 sheets No entry into the range of 1m to the left direction toward the sheet 2 sheets::
::
Those sheets 3 are in a predetermined range (for example, A1 meter to A2 meter) such as front, rear, top, bottom, right, left, diagonal upper right, diagonal lower right, diagonal upper left, diagonal lower left, etc. Etc.) is used to perform movement control such as entry prohibition and proximity promotion of the mobile robot 1.

It should be noted that information such as a character string or a figure indicating the positional relationship of the virtual graphic such as “prohibit backward entry” may be displayed in association with the sheet 3 on which the control graphic 2 is displayed. For example, information indicating the positional relationship of the virtual figure may be described on the back surface of the sheet 3 or may be described outside the frame of the seal-like sheet 3 or the like. Further, for example, the control graphic 2 displayed on the sheet 3 of “prohibition of backward entry” may display a code including virtual graphic information indicating a virtual wall that is an extension surface of the control graphic 2. . Further, for example, the control graphic 2 displayed on the sheet 3 of “prohibition of entry after 1 m ahead” is a virtual graphic indicating a virtual wall extending in a direction parallel to the control graphic 2 at a position 1 m ahead of the control graphic 2. A code including information may be displayed. Further, for example, the control graphic 2 displayed on the sheet 3 “prohibit intrusion in the range of 0.2 to 1 m in the right direction after 30 cm behind” is a range from 20 cm to 1 m on the right side of the control graphic 2, You may display the code | symbol containing the virtual figure information which shows the virtual wall of the position 30 cm behind (the back side of the control figure 2) from the surface of the figure 2. FIG. By sticking the sheet 3 on the wall, for example, when there is a door behind the virtual wall, the mobile robot 1 can be controlled not to move to the door. In addition, the user controls the mobile robot 1 so as not to approach within 2 meters from the center position of the control figure 2 by sticking a sheet 3 corresponding to “prohibition of entry within a radius of 2 m” to a wall or the like, for example. can do.
Further, the sheet has an adhesive layer formed on the back side, a large number of the control figures are stored side by side, and the control figure parts are individually cut out, so that the user can peel off the necessary control figures as appropriate. It may be. In addition, if a specific control graphic is lost, a control graphic print original can be added as a base paper so that the user can print out the control graphic that is insufficient.

  As described above, according to the mobile robot 1 according to the present embodiment, the code of the control graphic 2 for controlling the movement of the mobile robot 1 includes the virtual graphic information, and the mobile robot 1 uses the virtual graphic information. By being controlled, the degree of freedom of control of the mobile robot 1 is increased. For example, in the conventional example, the position where the barcode or the like is arranged is to determine the virtual wall or the like. However, in the case of the present embodiment, the virtual wall or the like is arbitrarily positioned with respect to the control figure 2. It can be set to. Further, by using the virtual graphic information such as t, b, l, and r described above, the range of the virtual wall and the like can be determined, and finer movement control can be realized. Further, when the size information is included in the code, the mobile robot 1 can be appropriately controlled even if the control figure 2 having a different size is used. In addition, when the size information and the position / orientation information are also included in the code of the control figure 2, the information is contained in only the information contained in the code of the control figure 2, and the size information is transmitted to the mobile robot 1. It is not necessary to give etc.

  In the above description, the coordinate system corresponding to the control figure 2 is the world coordinate system, and the coordinate system corresponding to the mobile robot 1 is the local coordinate system. However, their names are simply used to distinguish both coordinate systems. The world coordinate system and the local coordinate system may be reversed.

  In the above embodiment, each process or each function may be realized by centralized processing by a single device or a single system, or may be distributedly processed by a plurality of devices or a plurality of systems. It may be realized by doing.

  In the above embodiment, the information exchange between the components is performed by one component when, for example, the two components that exchange the information are physically different from each other. It may be performed by outputting information and receiving information by the other component, or when two components that exchange information are physically the same, one component May be performed by moving from the phase of the process corresponding to to the phase of the process corresponding to the other component.

  In the above embodiment, information related to processing executed by each component, for example, information received, acquired, and generated by each component, and threshold values used by each component for processing Information such as numerical formulas, setting values, etc. may not be specified in the above description, but may be held temporarily or for a long time in a recording medium (not shown). Further, the storage of information in the recording medium (not shown) may be performed by each component or a storage unit (not shown). Further, reading of information from the recording medium (not shown) may be performed by each component or a reading unit (not shown).

  Further, in the above embodiment, when information used by each component or the like, for example, information such as a threshold value or various setting values used by each component may be changed by the user, Even if it is not specified in the description, the user may be able to change the information as appropriate, or may not be so. If the information can be changed by the user, the change is realized by, for example, a not-shown receiving unit that receives a change instruction from the user and a changing unit (not shown) that changes the information in accordance with the change instruction. May be. The change instruction received by the receiving unit (not shown) may be received from an input device, information received via a communication line, or information read from a predetermined recording medium, for example. .

  In the above embodiment, each component may be configured by dedicated hardware, or a component that can be realized by software may be realized by executing a program. For example, each component can be realized by a program execution unit such as a CPU reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory. At the time of execution, the program execution unit may execute the program while accessing the storage unit or the recording medium. In addition, the software which implement | achieves at least one part function of the mobile robot 1 in the said embodiment is the following programs. In other words, this program is a program for causing a computer to execute processing in a mobile robot that moves autonomously, and is a two-dimensional figure displayed on a sheet, which is a virtual figure that is a virtual figure. An acquisition step of acquiring virtual graphic information from a control graphic included in a photographed image taken by a photographing means for photographing a control graphic that is a two-dimensional graphic that displays a code including virtual graphic information indicating a relative positional relationship with respect to the graphic. And a positional relationship acquisition step of acquiring a positional relationship between the mobile robot and the virtual graphic using the control graphic included in the captured image, the size information indicating the size of the control graphic, and the virtual graphic information acquired in the acquisition step. And a movement control that controls movement by the moving means for moving the mobile robot according to the positional relationship acquired in the positional relationship acquisition step. Is a program for executing the steps, to the computer.

  In the above program, the acquisition step for acquiring information does not include at least processing performed only by hardware, for example, processing performed by a modem or an interface card in the transmission step.

  Further, this program may be executed by being downloaded from a server or the like, and a program recorded on a predetermined recording medium (for example, an optical disk such as a CD-ROM, a magnetic disk, a semiconductor memory, or the like) is read out. May be executed by Further, this program may be used as a program constituting a program product. Further, the computer that executes this program may be singular or plural. That is, centralized processing may be performed, or distributed processing may be performed.

  FIG. 5 is a diagram showing an internal configuration of a computer system 900 that executes the program and realizes at least a part of the functions of the mobile robot 1 according to the embodiment. In FIG. 5, the computer system 900 includes a computer 901 including a CD-ROM drive 905, a keyboard 902, a mouse 903, and a monitor 904. In addition to the CD-ROM drive 905, the computer 901 is connected to the MPU 911, a ROM 912 for storing a program such as a bootup program, and the MPU 911, and temporarily stores an instruction of an application program. It includes a RAM 913 that provides a storage space, a hard disk 914 that stores application programs, system programs, and data, and a bus 915 that interconnects the MPU 911, the ROM 912, and the like. The computer 901 may include a network card (not shown) that provides connection to a LAN, WAN, or the like.

  A program that causes the computer system 900 to execute at least a part of the functions of the mobile robot 1 according to the above embodiment may be stored in the CD-ROM 921, inserted into the CD-ROM drive 905, and transferred to the hard disk 914. . Instead, the program may be transmitted to the computer 901 via a network (not shown) and stored in the hard disk 914. The program is loaded into the RAM 913 when executed. The program may be loaded directly from the CD-ROM 921 or the network. Further, the program may be read into the computer system 900 via another recording medium (for example, a DVD) instead of the CD-ROM 921.

  The program does not necessarily include an operating system (OS) or a third-party program that causes the computer 901 to execute at least a part of the functions of the mobile robot 1 according to the above-described embodiment. The program may include only a part of an instruction that calls an appropriate function or module in a controlled manner and obtains a desired result. How the computer system 900 operates is well known and will not be described in detail.

  Further, the present invention is not limited to the above-described embodiment, and various modifications are possible, and it goes without saying that these are also included in the scope of the present invention.

  As described above, according to the mobile robot and the like according to the present invention, the effect that the movement of the mobile robot can be controlled by the virtual graphic information included in the code of the control graphic is obtained, which is useful as a mobile robot that moves autonomously. .

DESCRIPTION OF SYMBOLS 1 Mobile robot 2 Control figure 3 Sheet | seat 11 Image | photography means 12 Memory | storage means 13 Acquisition means 14 Positional relationship acquisition means 15 Movement means 16 Movement control means

Claims (6)

A mobile robot that moves autonomously,
A virtual figure that is a two-dimensional figure displayed on a sheet, is a virtual figure, and indicates a relative positional relationship of the virtual figure, which is a figure that affects the movement control of the mobile robot, with respect to the two-dimensional figure Photographing means for photographing a control figure which is a two-dimensional figure for displaying a code including information;
An acquisition means for acquiring the virtual graphic information from a control graphic included in a photographed image photographed by the photographing means;
The position for acquiring the positional relationship between the mobile robot and the virtual figure using the control figure included in the captured image, the size information indicating the size of the control figure, and the virtual figure information acquired by the acquisition unit Relationship acquisition means;
Moving means for moving the mobile robot;
A movement control means for controlling movement by the moving means according to the positional relation acquired by the positional relation acquisition means,
The code also includes the size information,
The acquisition means also acquires the size information,
The mobile robot, wherein the size information used by the positional relationship acquisition unit is the size information acquired by the acquisition unit. The positional relationship acquisition means calculates coordinate conversion information indicating coordinate conversion between the coordinate system of the mobile robot and the coordinate system of the control graphic, using the control graphic and size information included in the captured image, The mobile robot according to claim 1, wherein a positional relationship between the mobile robot and the virtual graphic is acquired using coordinate conversion information and the virtual graphic information. The code also includes position and orientation information indicating the position and orientation of the control figure,
The acquisition means also acquires the position and orientation information,
The positional relationship acquisition unit calculates coordinate conversion information indicating coordinate conversion between the coordinate system of the mobile robot and the coordinate system of the control figure using the position and orientation information acquired by the acquisition unit. The described mobile robot. The virtual figure indicates a boundary relating to movement,
The mobile robot according to any one of claims 1 to 3, wherein the movement control means controls movement by the movement means so as to move without exceeding a boundary indicated by the virtual figure. The mobile robot according to any one of claims 1 to 4,
One or more sheets on which the control figures used for controlling the movement of the mobile robot are displayed,
A mobile robot control system in which a user controls movement of the mobile robot by placing a sheet on which the control graphic is displayed in a mobile space of the mobile robot. A program for causing a computer to execute processing in a mobile robot that moves autonomously,
A virtual figure that is a two-dimensional figure displayed on a sheet, is a virtual figure, and indicates a relative positional relationship of the virtual figure, which is a figure that affects the movement control of the mobile robot, with respect to the two-dimensional figure An acquisition step of acquiring the virtual graphic information from a control graphic included in a captured image captured by a photographing unit that captures a control graphic that is a two-dimensional graphic displaying a code including information;
The position for acquiring the positional relationship between the mobile robot and the virtual figure using the control figure included in the captured image, the size information indicating the size of the control figure, and the virtual figure information acquired in the acquisition step A relationship acquisition step;
According to the positional relationship acquired in the positional relationship acquisition step, the computer executes a movement control step for controlling movement by a moving unit that moves the mobile robot,
The code also includes the size information,
In the obtaining step, the size information is also obtained,
The size information used in the positional relationship acquisition step is a program that is the size information acquired in the acquisition step.

Images