JP2012198898A - Position specification device, operation instruction device, and self-propelled robot - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a position specification device capable of improving an accuracy when specifying an instruction object such as an object to be conveyed or an instruction object position such as a conveying destination, and to provide an operation instruction device, and a self-propelled robot.SOLUTION: A position specification device 101 includes: an image acquiring section 51 for acquiring a pattern projected to an instruction object in a working space by an operation of a user; and a position information extracting section 124 for extracting coordinates that are position information of the instruction object from the pattern acquired by the image acquiring section 51. The position specification device 101 is used for a self-propelled robot that executes an operation to the instruction object and autonomously moves in the working space based on the operation instruction to the instruction object based on the position information of the specified instruction object.

Description

  The present invention relates to a position specifying device, an operation instruction device, and a self-propelled robot for an instruction object or an instruction object position.

  When a self-propelled robot used in daily life is to carry an object by an instruction from the user, it is necessary for the person to instruct the object to be transported, the position / posture of the object, and the transport destination. There is.

  A technique for performing these instructions using a laser pointer is disclosed in Patent Document 1. In this technique, a user irradiates a laser pointer to an object to be transported or a transport position, and a self-propelled robot detects the position to execute an instructed operation.

JP 2006-231447 A

  However, in the above technique, it may be difficult to recognize the bright spot of the laser pointer, and in particular, in the environment of direct sunlight, the bright spot recognition accuracy is remarkably lowered, so the indicated position cannot be specified. There is a problem that it is not possible to accurately instruct the operation to the self-propelled robot.

  The present invention has been made to solve the above problem, and a position specifying device capable of improving the accuracy in specifying the position of an instruction target object such as a transported object or an instruction target position such as a transport destination. An object is to provide an operation instruction device and a self-propelled robot.

  In order to solve the above problems, the present invention employs the following means.

  The present invention is a position specifying device that projects a video pattern having a different pattern according to a position onto a predetermined area including a pointing object in a work space, and specifies the position of the pointing object based on the video pattern. An image acquisition unit that acquires the pattern projected onto the pointing object in the work space by a user operation, and positional information of the pointing object from the pattern acquired by the image acquisition unit. A position information extraction unit that extracts coordinates, and performs an operation on the pointing object based on an operation instruction on the pointing object based on the specified position information of the pointing object. A position specifying device used for a self-propelled robot that moves autonomously is adopted.

  According to the present invention, the position information extraction unit can extract the position information of the pointing object based on the pattern projected onto the pointing object. As a result, it is not necessary to register the position information of the pointing object in advance, and the position information of the pointing object placed at an arbitrary position can be extracted, so that the position of the pointing object can be easily specified. It becomes.

In the above invention, the video pattern may have a plurality of dots, and the arrangement of the dots may differ depending on the position.
By using, for example, an Anoto pattern (registered trademark) as the video pattern, it is possible to extract the position information of the pointing object.

In the above invention, the video pattern may have a plurality of color-coded regions, and the color may be different depending on the position.
By using, for example, a pattern color-coded according to the position as the video pattern, it is possible to extract the position information of the pointing object.

  The present invention is a position specifying device that specifies the position of an object to be pointed in a work space based on a plurality of images of the work space photographed by a plurality of image pickup devices arranged at predetermined positions. An image acquisition unit that acquires an image of the pointing object in the work space by an operation, a feature amount calculated from the image of the pointing object acquired by the image acquisition unit, and the work space imaged by the imaging device A comparison unit that compares the feature amounts calculated from the plurality of images, an indication target specifying unit that specifies the indication target based on a comparison result by the comparison unit, and a positional relationship between the plurality of imaging devices A position information extraction unit that extracts coordinates as position information of the pointing object based on the position of the pointing object in the image of the work space imaged by each imaging device; A self-propelled robot that autonomously moves in the work space that performs an operation on the pointing object based on an operation instruction on the pointing object based on the positional information of the specified pointing object Adopt the location device used.

  According to the present invention, based on the positional relationship of the imaging device and the plurality of images of the workspace taken by the imaging device, the location information extraction unit obtains the location information of the pointing object in the workspace acquired by the image acquisition unit. Extracted by Thereby, even when the name of the pointing object is not clear or when it is difficult to express the pointing object in a language, the image of the pointing object in the work space acquired by the image acquisition unit by the user's operation Since the position information can be obtained using, the position of the pointing object can be easily specified. Further, by extracting the position information of the pointing object by using a plurality of imaging devices, it is not necessary to register the position information of the pointing object in advance, and the position of the pointing object placed at an arbitrary position is specified. It becomes possible. The positional relationship when viewing the same point from two viewpoints in a three-dimensional space can be estimated by a known calculation method using epipolar geometry.

In the above invention, the comparison unit directly calculates a feature amount calculated from the image of the pointing object acquired by the image acquisition unit and a feature amount calculated from the image of the space photographed by the imaging device. Comparison may be performed, and the pointing object specifying unit may specify the pointing object based on a direct comparison result by the comparing unit.
By doing in this way, control at the time of specifying a pointed object can be performed easily.

In the above invention, the comparison unit calculates a feature amount calculated from the image of the pointing object acquired by the image acquisition unit and a feature amount calculated from the image of the space photographed by the imaging device. Indirect comparison is performed via a feature amount calculated from a registered image of the pointing object registered in advance, and the pointing object specifying unit specifies the pointing object based on an indirect comparison result by the comparing unit. It is good.
Since each imaging device and the image acquisition unit differ in the angle and position at which the pointing object is photographed, the distortion of the pointing object in each image differs, and it may be difficult to identify the pointing object from each image. . On the other hand, by performing an indirect comparison via a registered image of the pointing object registered in advance, it is possible to improve accuracy when specifying the pointing object.

In the above invention, the registered image of the pointing object may be an image of the pointing object taken from a plurality of angles.
When the pointing object has a three-dimensional shape, how the pointing object looks is different depending on the shooting angle and position. On the other hand, by registering in advance the images of the pointing object taken from a plurality of angles as the registered images of the pointing object, the accuracy when specifying the pointing object can be improved.

Moreover, in the said invention, it is good also as providing the laser irradiation part which irradiates a laser in the imaging | photography direction of the said image acquisition part.
By doing in this way, the user can visually observe the bright spot of the laser irradiated in the shooting direction of the image acquisition unit, and the shooting target position or the pointing target position intended by the user can be reliably shot.

Moreover, in the said invention, it is good also as providing the display part which displays the image which the said image acquisition part acquired.
By doing in this way, the image which the image acquisition part acquired can be visually recognized by the user in a display part, and the indication target object or indication target position which a user intends can be image | photographed reliably.

Further, in the above invention, when there are a plurality of candidates for the pointing object, the display unit displays each of the candidates and prompts the user to select the pointing object from among the displayed candidates. It is good.
By doing in this way, it becomes possible to select the instruction | indication target object which a user intends reliably.

In the above invention, a distance measuring unit that measures a distance to the pointing object is provided, and the pointing object specifying unit specifies the pointing object using the distance measured by the distance measuring unit. It is good as well.
By specifying the pointing object using not only the image acquired by the image acquiring unit but also the distance measured by the distance measuring unit, it is possible to identify the pointing object or the pointing target position having different sizes. Become.

  The present invention employs an operation instruction device including an operation instruction unit that issues an operation instruction to the pointing object based on the position information of the pointing object specified by the position specifying device.

  According to the present invention, even when the name of the pointing object is not clear or when it is difficult to express the pointing object in a language, it is possible to easily give an operation instruction to the pointing object.

  The present invention employs a self-propelled robot that performs an operation on the specified object based on an instruction from the operation instruction device.

  According to the present invention, even when the name of the pointing object is not clear or when it is difficult to express the pointing object in a language, it is possible to execute an operation on the pointing object.

  According to the present invention, there is an effect that it is possible to improve the accuracy in specifying the position of an instruction target object such as an object to be conveyed or an instruction target position such as a conveyance destination.

It is a hardware block diagram of the robot system which concerns on the reference example 1 of this invention. It is a front view of the robot main body shown in FIG. It is a left view of the robot main body shown in FIG. It is a functional block diagram of the robot system shown in FIG. It is a figure explaining the effect | action of the robot system shown in FIG. It is a functional block diagram of the 1st modification of the robot system shown in FIG. It is a figure explaining the effect | action of the 2nd modification of the robot system shown in FIG. It is a figure explaining the effect | action of the 3rd modification of the robot system shown in FIG. It is a figure explaining the effect | action of the 4th modification of the robot system shown in FIG. It is a figure explaining the effect | action of the 5th modification of the robot system shown in FIG. It is a hardware block diagram of the robot system which concerns on the 1st Embodiment of this invention. It is a figure explaining Anoto pattern (registered trademark). It is a functional block diagram of the robot system shown in FIG. It is a figure explaining the effect | action of the robot system shown in FIG. It is a figure explaining the effect | action of the modification of the robot system shown in FIG. It is a functional block diagram of the robot system which concerns on the reference example 2 of this invention. It is a figure explaining the effect | action of the robot system shown in FIG. It is a hardware block diagram of the robot system which concerns on the 2nd Embodiment of this invention. It is a functional block diagram of the robot system shown in FIG. It is a figure explaining the effect | action of the robot system shown in FIG. It is a figure explaining the effect | action of the 1st modification of the robot system shown in FIG. It is a figure explaining the effect | action of the 2nd modification of the robot system shown in FIG.

[Reference Example 1]
A reference example 1 of a robot system including a position specifying device, an operation instruction device, and a self-propelled robot (hereinafter referred to as “robot”) according to the present invention will be described below with reference to the drawings.
As shown in FIG. 1, the robot system 100 according to the first reference example includes a robot 1 and an operation instruction device 50 that instructs the robot 1 to perform an operation.

The operation instruction device 50 includes an image acquisition unit 51 that acquires an image of an instruction target and an operation instruction unit 52 that instructs the robot 1 to perform an operation. The image acquisition unit 51 is, for example, a camera that includes a lens and a CCD.
The robot 1 includes a control device 40 that controls the robot 1 based on an instruction from the operation instruction device 50. The control device 40 is configured to specify the position of the pointing object based on the image of the pointing object acquired by the image acquisition unit 51 provided in the operation instruction device 50.

Below, the structure of the robot 1 is demonstrated using FIG. 2 and FIG.
2 is a front view of the robot according to the first reference example, and FIG. 3 is a left side view of the robot shown in FIG.
As shown in FIGS. 2 and 3, the robot body 1 includes a head 2, a chest 3 that supports the head 2 from below, a right arm 4 a provided on the right side of the chest 3, and a chest 3. A left arm portion 4b provided on the left side, a waist portion 5 connected below the chest portion 3, a skirt portion 6 connected below the waist portion 5, and a leg portion 7 connected below the skirt portion 6; Is provided.

One omnidirectional camera 11 is provided near the top of the head 2. A plurality of infrared LEDs 12 are arranged on the ring at predetermined intervals along the outer periphery of the omnidirectional camera 11.
In the vicinity of the center of the front surface of the head 2, as shown in FIG. 1, the front camera 13 for imaging the front is one on the right side when viewed from the front, and the microphone 14 is one on the left side when viewed from the front. Is provided.

  One monitor 15 is provided near the center of the front surface of the chest 3. One ultrasonic distance sensor 16 for detecting a person is provided above the monitor 15. One power switch 17 is provided below the monitor 15. Above the ultrasonic distance sensor 16, two speakers 18 are provided one on each side. Further, as shown in FIG. 3, the back of the chest 3 is provided with a school bag portion 33 that can store luggage. The school bag 33 is provided with an opening / closing door 33a that can be rotated around a hinge provided at the top. As shown in FIG. 1, one shoulder switch 19 that functions as a man-machine interface is provided on each of the left and right shoulders of the chest 3. For the shoulder switch 19, for example, a touch sensor is employed.

  A multi-joint structure is adopted for the right arm portion 4a and the left arm portion 4b. In the right arm portion 4a and the left arm portion 4b, side switches 20 are provided in the vicinity of the connection portion with the chest portion 3 to detect the pinching of a body or an object and stop the movement of the arm. As shown in FIG. 2, a handshake switch 21 that functions as a man-machine interface is built in the palm of the right arm 4a. For the side switch 20 and the handshake switch 21, for example, a pressure sensor is employed.

  In the vicinity of the center of the front surface of the waist 5, one ultrasonic distance sensor 22 for detecting a person is provided on each side. Below these ultrasonic distance sensors 22, a sensor region 24 in which a plurality of infrared sensors 23 are arranged is provided. These infrared sensors 22 are for detecting an obstacle or the like in the lower front of the robot body 1. As shown in FIGS. 2 and 3, a total of four microphones 25 for detecting the direction of the sound source are provided below the waist 5 on the front and back sides, one on each side. As shown in FIG. 3, one handle portion 26 used for lifting the main body is provided on each of the left and right sides of the waist portion 5. The handle 26 is a recess so that the operator's hand can be inserted.

  Below the front surface of the skirt portion 6, a total of three infrared sensors 27 for detecting a step are provided in the center and on the left and right. As shown in FIG. 3, a charging connector 28 is provided on the back surface of the skirt portion 6.

As shown in FIG. 2, one infrared sensor 29 for detecting a lateral distance is provided on the front surface of the leg portion 7 on the left and right sides. These infrared sensors 29 are mainly used for level difference detection.
As shown in FIG. 3, a hook 30 for fixing the position of the robot body 1 to the charging station is provided on the back surface of the leg portion 7. The leg portion 7 is a carriage provided with traveling wheels 31 and four ball casters 32.
In the robot described above, the ultrasonic distance sensor 16 in the chest 3, the ultrasonic distance sensor 22 in the waist 5, and the microphone 25 function as a human detection sensor 34 that detects a person around the robot.

  Such a robot has a configuration capable of moving independently in a work space by supplying power from a battery built in the robot body 1, and coexists with a human being as a work space indoors. For example, it is used to provide various services for assisting, supporting, and caring for the lives of users such as robot owners and operators in general households. Therefore, in addition to a conversation function that realizes a conversation with the user, the robot has a function of watching the user's action, assisting the user's action, and acting with the user.

  Such a function is realized by, for example, the control device 40 including a microcomputer or the like built in the robot body 1. 2 and 3 are connected to the control device 40, and image information from the camera and sensor detection information from the sensor are acquired, and various programs are based on these information. By executing the above, various functions described above are realized. In addition, the control device 40 acquires a signal from the operation instruction device 50 and specifies the position of the instruction target based on the signal. In the following description, it is assumed that the position of the pointing object is specified by the position specifying device 101.

FIG. 4 shows a functional block diagram in which the functions of the position specifying device 101 are developed and expressed.
As illustrated in FIG. 4, the position specifying device 101 includes a storage unit 121 that stores an image and position information, an image acquisition unit 51 that acquires an image of an instruction target, and a comparison unit 122 that compares a plurality of images. The pointing object specifying unit 123 that specifies the pointing object and the position information extracting unit 124 that extracts the position information of the pointing object are provided.

The storage unit 121 stores a feature amount calculated from images of a plurality of objects captured in advance and position information of the objects in association with each other. Here, the feature amount calculated from the image is information indicating the feature of the image extracted from the image in order to perform comparison processing with more accuracy, robustness, high speed, and small memory. Etc.
The image acquisition unit 51 acquires an image of the pointing object by a user operation, and outputs a feature amount calculated from the acquired image of the pointing object to the comparison unit 122.

  The comparison unit 122 reads the feature amount calculated from the images of the plurality of objects stored in the storage unit 121, the feature amount calculated from the read images of the plurality of objects, and the instruction target acquired by the image acquisition unit 51 The comparison is made with the feature amount calculated from the image of the object, and the comparison result is output to the instruction target specifying unit 123. In comparison, for example, US Patent 6,711,293, David G. Lowe, “Object recognition from local scale-invariant features,” International Conference on Computer Vision, The similarity between images can be calculated by a known method shown in Corfu, Greece (September 1999), pp. 1150-1157.

  The indication target specifying unit 123 specifies the target that most closely matches the indication target based on the comparison result by the comparison unit 122, and outputs the specified result to the position information extraction unit. .

The position information extraction unit 124 reads the position information of the object corresponding to the specified object specified by the specified object specifying unit 123 from the storage unit 121, and uses the read position information of the object as the position information of the specified object. It is output to the operation instruction unit 52.
The operation instruction unit 52 is configured to output the position information of the pointing object extracted by the position information extraction unit 124 and the operation with respect to the position information to the robot body 1.

The operation of the position specifying device 101 configured as described above will be described below mainly using FIG.
In the storage unit 121, the feature amount calculated from the image P _{A of the} object A and the feature amount calculated from the image P _{B of the} object B are stored in association with each position information.

Here, if the user selects an object B as an instruction target, the image Q _B indicated object by the image acquisition unit 51 by the operation of the user is acquired.
Next, the comparison unit 122, the feature amount and the object calculated from the image P _A of the object A that feature amount calculated from image Q _B of the obtained instruction object and read out from the storage unit 121 by the image acquisition unit 51 Comparison with the feature amount calculated from the image P _{B of B} is performed.

Then, based on the comparison result by the comparison unit 122, the image P _B is selected as the best match image to the image Q _B by the instruction object identification unit 123, as an object B indicated object corresponding to the image P _B Identified.

Next, the position information associated with the feature amount calculated from the image P _{B of the} object B identified as the pointing object by the position information extraction unit, that is, the coordinates of the center of the table in FIG. 121 is read out.

  Thus, the position information of the object B read from the storage unit 121 is extracted as the position information of the instruction object, and the operation instruction unit 52 issues an operation instruction such as a conveyance instruction to the robot body 1.

  As described above, according to the position specifying device 101 according to the first reference example, the image acquisition unit 51 uses the feature amounts and the position information calculated from the images of the plurality of objects stored in the storage unit 121. The acquired pointing object is specified by the pointing object specifying unit 123, and the position information of the specified pointing object is extracted by the position information extracting unit 124. Accordingly, even when the name of the pointing object is not clear or when it is difficult to express the pointing object in a language, the position information is obtained using the image of the pointing object acquired by the image acquisition unit 51. Thus, the position of the pointing object can be easily specified.

  As a first modification of the first reference example, as shown in FIG. 6, an image code acquisition unit 54 that acquires image code information such as a barcode and a QR code is provided instead of the image acquisition unit 51. The storage unit 121 may store information of the image code instead of the image of the object.

By doing so, for example, it is possible to identify a plurality of objects having similar appearances, and it is possible to improve the accuracy when specifying the pointing object. In addition, for example, barcodes are already attached to many products, and it is possible to acquire information such as names of designated objects by using a database that associates barcodes and designated objects.
Further, by using the image acquisition unit 51 and the image code acquisition unit 54 together, it is possible to easily specify the pointing object with high accuracy.

As a second modification of the first reference example, as shown in FIG. 7, a laser irradiation unit 55 that irradiates a laser in the photographing direction of the image acquisition unit 51 may be provided. Here, the laser irradiation unit 55 is, for example, a laser pointer.
By doing in this way, the user can visually observe the bright spot of the laser irradiated in the shooting direction of the image acquisition unit 51, and the pointing object intended by the user can be reliably shot.

Further, as a third modification of the present reference example 1, as shown in FIG. 8, the distance measuring unit 56 that measures the distance to the pointing target is provided, and the pointing target position specifying unit 123 is operated by the distance measuring unit 56. The pointing object may be specified using the measured distance. Here, the distance measuring unit 56 is, for example, a laser range finder.
By specifying the target object using not only the image acquired by the image acquisition unit 51 but also the distance measured by the distance measuring unit 56, it is possible to identify the target object having a different size.

In addition, as a fourth modification of the first reference example, as illustrated in FIG. 9, a display unit 57 that displays an image acquired by the image acquisition unit 51 may be provided. Here, the display unit 57 is a monitor such as a CRT (cathode-ray tube) or an LCD (liquid Crystal Display).
By doing in this way, the image which the image acquisition part 51 acquired can be visually recognized by the user on the display part 57, and the instruction | indication target object which a user intends can be image | photographed reliably.

In addition, as a fifth modification of the first reference example, as illustrated in FIG. 10, when there are a plurality of candidates for an instruction target, each candidate is displayed on the display unit 57, and from among the displayed candidates The user may be prompted to select an instruction target.
By doing in this way, it becomes possible to select the instruction | indication target object which a user intends reliably. Each candidate may be displayed not on the display unit 57 but on the touch panel 58 provided separately from the operation instruction device 60.

[First Embodiment]
Next, a first embodiment of the present invention will be described with reference to FIGS.
The robot system of the present embodiment is different from the reference example 1 in that a projection device projects a video pattern having a different pattern depending on the position, and extracts the position information of the pointing object based on the video pattern. . Hereinafter, with respect to the robot system of the present embodiment, description of points that are common to Reference Example 1 is omitted, and different points are mainly described.

As shown in FIG. 11, the robot system 200 according to the present embodiment projects a robot 1, an operation instruction device 60 that instructs the robot 1 to operate, and a projection that projects a video pattern having a different pattern depending on the position. Device 63.
Here, for example, an Anoto pattern (registered trademark) shown in FIG. 12 is used as a video pattern having a different pattern depending on the position. Anoto pattern (registered trademark) is a pattern in which a plurality of dots are arranged on a grid-like line segment and the arrangement of the dots differs depending on the position.

  The projection device 63 is, for example, a projection device that projects near-infrared rays that are invisible to humans, and the image acquisition unit 61 is, for example, a near-infrared camera that can acquire near-infrared light. As a result, the image acquisition unit 61 can acquire the image pattern projected by the projection device 63 without causing the person to recognize the image pattern.

  In the robot system 200 having the above-described configuration, the image acquisition unit 61 acquires a video pattern projected onto the pointing object by the projection device 63, and the control device 40 specifies the position of the pointing object based on the acquired video pattern. Is called. In the following description, it is assumed that the position of the pointing object is specified by the position specifying device 103.

FIG. 13 shows a functional block diagram in which the functions of the position specifying device 103 are developed and expressed.
As illustrated in FIG. 13, the position specifying device 103 includes an image acquisition unit 61 that acquires a pattern projected on the pointing object, and a position information extraction unit 134 that extracts position information of the pointing object.

  The image acquisition unit 61 acquires a video pattern projected onto the pointing object by the projection unit 63 by a user operation, and outputs the acquired video pattern to the position information extraction unit 134.

The position information extraction unit 134 extracts the position information of the pointing object from the video pattern acquired by the image acquisition unit 61, and outputs the extracted position information of the pointing object to the operation instruction unit 62.
The operation instruction unit 62 is configured to output to the robot body 1 the position information of the pointing object extracted by the position information extraction unit 134 and the operation with respect to the position information.

The operation of the position specifying device 102 configured as described above will be described below mainly using FIG.
From the projection device 63, for example, an Anoto pattern (registered trademark) video pattern is projected onto a predetermined area including the pointing object.
Here, if the user selects an object B as an instruction target, the image R _B of image pattern projected onto the object B by the image acquisition unit 61 by the operation of the user is acquired.

Then, the position information extraction unit 134, the position information of the image R _B of image pattern projected onto the object B, ie, the coordinates of the center portion of the table 14 are extracted.

In this way, the position information of the target object is extracted, and an operation instruction such as a conveyance instruction is given to the robot body 1 by the operation instruction unit 62.
As in Reference Example 1, the feature quantity calculated from the images of the plurality of objects is stored in the image DB, and the image acquisition unit 61 acquires not only the video pattern but also the image of the object, thereby It is also possible to specify the object itself.

  As described above, according to the position specifying device 103 according to the present embodiment, a pattern projected on the pointing object is acquired by the image acquisition unit 61, and the position information of the pointing object is obtained based on the pattern as position information. It can be extracted by the extraction unit 134. Thereby, it is not necessary to register the position information of the pointing object in advance, and it is possible to extract the position information of the pointing object placed at an arbitrary position.

As a modification of the present embodiment, as shown in FIG. 15, the projection device 63 may project a video pattern having a different color depending on the position, and the image acquisition unit 61 may acquire the pattern.
By doing so, it is possible to extract the position information of the pointing object based on the color pattern projected onto the pointing object.

[Reference Example 2]
Next, Reference Example 2 of the present invention will be described with reference to FIGS. 16 and 17.
The robot system of the present reference example 2 is different from the above-described embodiment or reference example in that position information of the instruction target position is extracted based on an image of a space in which work is performed. Hereinafter, with respect to the robot system according to the second reference example, the description of the points common to the above-described embodiment or the reference example will be omitted, and different points will be mainly described.

FIG. 16 shows a functional block diagram in which the functions of the position specifying device 104 for specifying the target position are developed.
As illustrated in FIG. 16, the position specifying device 104 calculates a feature amount and position information calculated from an image, an image acquisition unit 51 that acquires an image of an instruction target position, and a plurality of images. A comparison unit 122 that compares feature amounts, an instruction target position specifying unit 133 that specifies an instruction target position, and a position information extraction unit 124 that extracts position information of the instruction target position are provided.

The storage unit 121 stores a feature amount calculated from an image of a space in which work such as a wall, a ceiling, or a floor is performed, and positional information of the space in association with each other.
The image acquisition unit 51 acquires an image of the instruction target position by a user operation, and outputs a feature amount calculated from the acquired image of the instruction target position to the comparison unit 122.

The comparison unit 122 reads the feature amount calculated from the image of the space in which the work stored in the storage unit 121 is performed, and calculates the feature amount calculated from the image of the read space and the instruction target position acquired by the image acquisition unit 51. Comparison is made with the feature amount calculated from the image, and the comparison result is output to the instruction target position specifying unit 133.
The instruction target position specifying unit 133 specifies a space that most closely matches the instruction target position based on the comparison result by the comparison unit 122 as an instruction target position, and outputs the specified result to the position information extraction unit.

The position information extraction unit 124 reads the position information of the space corresponding to the instruction target position specified by the instruction target position specifying unit 133 from the storage unit 121, and instructs the operation using the read position information of the space as the position information of the instruction target position. The data is output to the unit 52.
The operation instruction unit 52 outputs the position information of the instruction target position extracted by the position information extraction unit 124 and the operation with respect to the position information to the robot body 1.

The operation of the position specifying device 104 configured as described above will be described below mainly using FIG.
As shown in FIG. 17A, the storage unit 121 stores the feature amount calculated from the image S _A of the wall surface A of the space in which the work is performed, the feature amount calculated from the image S _{B of the} wall surface B, and the image of the wall surface C. feature amount calculated from S _C have been stored in association with the respective position information.

Here, as illustrated in FIG. 17B, when the user selects the region X of the wall surface C as the instruction target position, the image acquisition unit 51 acquires the image T _X of the instruction target position by the user's operation. .
Next, the comparison unit 122, feature amount calculated from image S _A wall A read from the feature storage unit 121 calculated from the image T _X of the obtained instruction target position by the image acquiring unit 51, the wall B feature amount calculated from image S _B, and comparison between the feature amount calculated from image S _C of the wall C is performed.

Then, based on the comparison result by the comparison unit 122, the area X in the image S _C as an image that best matches the image T _X is selected by the instruction target position specifying unit 133, the position of the region X is a referent position Identified.
Next, the position information associated with the region X specified as the instruction target position by the position information extraction unit 124, that is, the coordinates of the upper left portion of the shelf arranged on the wall surface C in FIG. Read from the storage unit 121.

  Thus, the position information of the region X read from the storage unit 121 is extracted as the position information of the instruction target position, and the operation instruction unit 52 issues an operation instruction such as a conveyance instruction to the robot body 1.

As described above, according to the position specifying device 104 according to the second reference example, based on the feature amount and the position information calculated from the image of the space where the work such as the wall surface, the floor surface, or the ceiling surface is performed, The position information of the instruction target position acquired by the image acquisition unit 51 can be extracted.
Further, as shown in FIG. 17C, it is possible to estimate the position and direction of the operation instruction device 70 that is a device from the size and distortion of a window or the like in the image of the space where the work is performed. Become.

[Second Embodiment]
Next, a second embodiment of the present invention will be described with reference to FIGS.
The robot system of this embodiment is different from the above-described embodiment or reference example in that the position of the pointing object is based on an image of a space in which work is performed by a plurality of imaging devices arranged at predetermined positions. It is a point to identify. Hereinafter, with respect to the robot system of the present embodiment, description of points common to the above-described embodiment or reference example will be omitted, and different points will be mainly described.

As illustrated in FIG. 18, the robot system 300 according to the present embodiment includes a robot 1, an operation instruction device 70 that instructs the robot 1 to perform an operation, and a plurality of imaging devices 73 that capture a space in which work is performed. It has.
The imaging device 73 is, for example, a camera that is disposed on a ceiling surface at a corner of a space where work is performed and includes a lens and a CCD.

  In the robot system 300 having the above-described configuration, the image of the pointing object is acquired by the image acquisition unit 71, and the image of the pointing object acquired by the image acquisition unit 71 and the space where the work photographed by the imaging device 73 is performed are performed. Based on the image, the position of the pointing object is specified by the control device 40. In the following description, it is assumed that the position of the pointing object is specified by the position specifying device 105.

FIG. 19 is a functional block diagram in which the functions of the pointing object position specifying device 105 are expanded and represented.
As illustrated in FIG. 19, the position specifying device 105 specifies an instruction target object, an image acquisition unit 71 that acquires an image of the target object, a comparison unit 122 that compares feature amounts calculated from a plurality of images, and the target object. The pointing object specifying unit 123 and a position information extracting unit 144 that extracts position information of the pointing object are provided.

The image acquisition unit 71 acquires an image of an instruction target object by a user operation, and outputs a feature amount calculated from the acquired image of the instruction target object to the comparison unit 122.
The comparison unit 122 compares the feature amount calculated from the image of the space where the work photographed by the imaging device 73 is performed with the feature amount calculated from the image of the pointing object acquired by the image acquisition unit 71, and compares The result is output to the instruction target specifying unit 123.

  The indication target specifying unit 123 specifies the target that most closely matches the indication target based on the comparison result by the comparison unit 122, and outputs the specified result to the position information extraction unit 144. Yes.

The position information extraction unit 144 extracts the position information of the pointing object based on the positional relationship of the imaging device 73 and the position of the pointing object in the image of the space where the work photographed by the imaging device 73 is performed, The extracted position information of the instruction target is output to the operation instruction unit 72.
The operation instruction unit 72 is configured to output to the robot body 1 the position information of the pointing object extracted by the position information extraction unit 144 and the operation with respect to the position information.

The operation of the position specifying device 105 configured as described above will be described below mainly using FIG.
A space in which work is performed is photographed by the imaging device 73 (a) and the imaging device 73 (b). An image taken by the imaging device 73 (a) is an image U _A , and an image taken by the imaging device 73 (b) is an image U _B.

Here, when the user selects the object D as the instruction object, the image acquisition unit 71 acquires the image V of the instruction object by the user's operation.
Next, the feature amount calculated from the image V of the pointing object acquired by the image acquisition unit 71 by the comparison unit 122, the feature amount calculated from the image U _A taken by the imaging device 73 (a), and the imaging device comparison of the feature amount calculated from the photographed image U _B is performed by 73 (b).

Then, based on the comparison result by the comparison unit 122, as an image that matches the image V by an instruction object identification unit 123, the region Y and the image U region Z in the _B in the image U _A is selected, the area The objects corresponding to Y and the area Z are specified as the pointing objects.

Next, based on the positional relationship between the imaging device 73 (a) and the imaging device 73 (b) and the position of the pointing object in the images U _A and U _B , the position information extraction unit 144 performs the pointing object. Is extracted. At this time, the position information of the pointing object is, for example, Richard Hartley and Andrew Zisserman (2003). Multiple View Geometry in computer vision. Cambridge University Press. ISBN 0-521-54051-8. It can be estimated by a calculation method using a known epipolar geometry shown in FIG.

  As described above, according to the position specifying device 105 according to the present embodiment, the positional relationship between the imaging devices 73 (a) and 73 (b) and the imaging device 73 (a) and the imaging device 73 (b). The position information extraction unit 144 extracts the position information of the pointing object acquired by the image acquisition unit 71 based on the image of the space in which the photographed work is performed. In this way, by extracting the position information of the pointing object by using a plurality of imaging devices, it is not necessary to register the position information of the pointing object in advance, and the position of the pointing object placed at an arbitrary position can be determined. It becomes possible to specify.

  As a first modification of the present embodiment, as shown in FIG. 21, the feature amount calculated from the image of the pointing object is registered in advance in the image DB, and the comparison unit 122 is used by the image acquisition unit 71. The feature amount calculated from the acquired image of the pointing object and the feature amount calculated from the image of the space where the work photographed by the imaging device 73 (a) and the imaging device 73 (b) is performed are registered in the image DB. The indirect comparison may be performed via the feature amount calculated from the image, and the indication target specifying unit 123 may specify the indication target based on the indirect comparison result by the comparison unit 122.

  Since the imaging device 73 (a), the imaging device 73 (b), and the image acquisition unit 71 have different angles and positions for shooting the pointing object, the distortion and the like of the pointing object in each image are different, and the pointing object is different from each image. It may be difficult to identify things. On the other hand, by performing an indirect comparison via the image of the pointing object registered in advance in the image DB, it is possible to improve the accuracy when specifying the pointing object.

Further, as a second modification of the present embodiment, as shown in FIG. 22, the feature amount calculated from the image of the pointing object registered in advance in the image DB is obtained from the image of the pointing object captured from a plurality of angles. It may be a calculated feature amount.
When the pointing object has a three-dimensional shape, how the pointing object looks is different depending on the shooting angle and position. On the other hand, the accuracy of specifying the pointing object can be improved by using the pointing object image taken from a plurality of angles as the registered image of the pointing object.

  As mentioned above, although embodiment of this invention was explained in full detail with reference to drawings, the specific structure is not restricted to this embodiment, The design change etc. of the range which does not deviate from the summary of this invention are included.

DESCRIPTION OF SYMBOLS 1 Robot main body 40 Control apparatus 50, 60, 70 Operation instruction apparatus 51, 61, 71 Image acquisition part 52, 62, 72 Operation instruction part 54 Image code acquisition part 55 Laser irradiation part 56 Distance measurement part 57 Display part 58 Touch panel 63 Projection Device 73, 73 (a), 72 (b) Imaging device 100, 200, 300 Robot system 101, 103, 104, 105 Position specifying device 121 Storage unit 122 Comparison unit 123 Pointed object specifying unit 124, 134, 144 Position information Extraction unit 133 Instruction target position specifying unit

Claims (13)

A position specifying device that projects a video pattern having a different pattern according to a position onto a predetermined area including a pointing object in a work space, and specifies the position of the pointing object based on the video pattern,
An image acquisition unit that acquires the pattern projected on the pointing object in the work space by a user operation;
A position information extraction unit that extracts coordinates that are position information of the pointing object from the pattern acquired by the image acquisition unit;
Used for a self-propelled robot that autonomously moves in the work space that performs an operation on the specified object based on an operation instruction for the specified object based on the positional information of the specified specified object Positioning device.   The position specifying device according to claim 1, wherein the video pattern has a plurality of dots, and the arrangement of the dots varies depending on the position.   The position specifying device according to claim 1, wherein the video pattern has a plurality of color-coded regions, and the colors differ depending on the position. A position specifying device for specifying the position of an instruction target in the work space based on a plurality of images of the work space taken by a plurality of image pickup devices arranged at a predetermined position,
An image acquisition unit that acquires an image of the pointing object in the work space by a user operation;
A comparison unit that compares the feature amount calculated from the image of the pointing object acquired by the image acquisition unit with the feature amount calculated from a plurality of images of the work space photographed by the imaging device;
An instruction object specifying unit for specifying the instruction object based on a comparison result by the comparison unit;
Position information for extracting coordinates, which are position information of the pointing object, based on the positional relationship of the plurality of imaging devices and the position of the pointing object in the image of the work space photographed by each of the imaging devices An extraction unit,
Used for a self-propelled robot that autonomously moves in the work space that performs an operation on the specified object based on an operation instruction for the specified object based on the positional information of the specified specified object Positioning device. The comparison unit performs a direct comparison between the feature amount calculated from the image of the pointing object acquired by the image acquisition unit and the feature amount calculated from the image of the space photographed by the imaging device,
The position specifying device according to claim 4, wherein the pointing object specifying unit specifies the pointing object based on a direct comparison result by the comparing unit. The instruction that is registered in advance with the feature amount calculated from the image of the pointing object acquired by the image acquisition unit and the feature amount calculated from the image of the space photographed by the imaging device by the comparison unit. Perform an indirect comparison via the feature value calculated from the registered image of the object,
The position specifying device according to claim 5, wherein the specified object specifying unit specifies the specified object based on an indirect comparison result by the comparing unit.   The position specifying device according to claim 6, wherein the registered image of the pointing object is an image of the pointing object taken from a plurality of angles.   The position specifying device according to any one of claims 1 to 7, further comprising a laser irradiation unit that irradiates a laser in a photographing direction of the image acquisition unit.   The position specifying device according to claim 1, further comprising a display unit that displays an image acquired by the image acquisition unit.   The position according to claim 9, wherein the display unit displays each of the candidates when there are a plurality of candidates for the pointing object, and prompts the user to select the pointing object from the displayed candidates. Specific device. A distance measuring unit for measuring the distance to the pointing object;
The position specifying device according to any one of claims 1 to 10, wherein the pointing target specifying unit specifies the pointing target using also the distance measured by the distance measuring unit.   An operation instruction device comprising an operation instruction unit that issues an operation instruction to the pointing object based on position information of the pointing object specified by the position specifying device according to any one of claims 1 to 11.   A self-propelled robot that performs an operation on the specified object based on an instruction from the operation instruction device according to claim 12.

Images