JP2005088146A - Object processing system, object processing method and robot - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To simply identify an object in working environments by a picked-up image. <P>SOLUTION: This system includes an information server 101 in which object recognition information OBS needed for recognizing images of objects 501-506 is stored, a robot 1 for identifying the objects 501-506 based on the object processing information OBS stored in the server 101, and object identifying devices 201-206 for monitoring environments of the robot 1. The robot 1 includes a tag reader/writer 4 for reading information from tags 501a-506a attached to the objects 501-506, an address processing part and a network processing part for reading the object recognition information OBS from the information server 101 based on object ID information OB read by the tag reader/writer 4, and a control part 5 for identifying the object by the use of the object recognition information OBS read by the address processing part or the like, picked-up image information obtained by an imaging part 9, and object identification results of the object identifying devices 201-206. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a robot that autonomously performs processing such as operation and work on an object existing in the work environment in a given work environment, and an object processing system and an object processing method that use the robot.

There is a system in which an imaging unit is provided in a robot and the robot is allowed to work. This is a case where the robot works autonomously by robot vision.
In this system, a robot recognizes an object to be worked in a work environment from an image obtained from an imaging unit based on information such as a shape and specifies the object.
Here, as one method for allowing a robot to autonomously perform work in a given work environment, information about an object to be a work target existing in the work environment is given to the robot as knowledge, and the robot uses the knowledge to work. There is a way to build a plan and perform work autonomously.

  For example, as shown in FIG. 12, a case where the robot 600 autonomously works in a working environment where six various objects 501, 502, 503, 504, 505, and 506 exist will be described as an example. In FIG. 12, the objects 501 to 506 existing in the work environment are shown as two-dimensional plane objects. However, the work environment is actually a three-dimensional space, and the objects 501 to 506 existing in the work environment are also three-dimensional space. Is a solid object.

Here, consider a case where the robot 600 autonomously moves from the initial position P1 to the target position P2.
FIG. 13 shows human work for causing the robot to realize such movement.
First, a work environment model WM including information on each object 501 to 506 is prepared.
The work environment model WM is information indicating the structure of the actual work environment W in which the robot 600 works, and the work environment model WM includes information on each object 501 to 506 existing in the work environment W.

  There are various contents of information on the objects 501 to 506, for example, information on the positions (for example, absolute positions) of the objects 501 to 506 and information on geometric shapes. The designer of the robot 600 describes such information as a work environment model WM, and the work environment model WM is built in the control unit (computer) 601 of the robot 600. For example, the work environment model WM is constructed in the control unit (computer) 601 as environment data.

The control unit 601 stores information or a program for operating the robot 600 in addition to such a work environment model WM.
For example, information such as the size of the robot 600 is stored as information for operating the robot 600. In addition, a work plan algorithm is stored as a control program for operating the robot 600. The work plan algorithm is an algorithm for creating a work plan that is a work procedure of the robot 600 based on the work environment model WM. Information or a program for operating the robot 600 is determined or designed by the designer of the robot 600 and stored in the control unit 601 in advance.

The control unit 601 controls the drive unit 602 of the robot 600. The drive unit 602 is composed of, for example, moving means for moving the robot 600.
In such a robot 600, when a work command (or operation command) is given, the control unit 601 refers to the size of the robot 600, the position of the object of the work environment model WM, and the geometric shape by the work plan algorithm. Thus, the mathematical calculation process for checking the interference between the robot 600 and the object is performed, and the path from the initial position P1 to the target position P2 is determined. At this time, the robot 600 performs recognition processing using the image captured by the imaging unit and information on the geometric shape (for example, a template), performs template matching, for example, and identifies an object.

  Then, the control unit 601 makes a work plan so that the robot 600 actually operates along the determined route. Here, the work plan includes information (control signal group) indicating how the drive unit 602 should be controlled in order for the robot 600 to realize the work. The control unit 601 controls the drive unit 602 by outputting a control signal to the drive unit 602 based on this work plan. As a result, the robot 600 autonomously moves from the initial position P1 to the target position P2 while avoiding contact with the objects 501 to 506.

  As described above, there has conventionally been a method of constructing a work environment model in a computer (control unit) that controls a robot, making the robot make a work plan with reference to the work environment model, and causing the robot to work autonomously. Many researchers have been researching such methods. Such a method for autonomously working such a robot is called a model-based autonomous control technology of the robot.

  Such a method is not necessarily used only for moving the robot. For example, the present invention can be applied to autonomously operating a robot arm or a robot manipulator used for performing various operations on an object existing in a work environment. For example, when applied to a robot arm manipulator, the object information (work environment model) in the work environment includes the position of the object (position optimal for grasping) in addition to the position and geometric shape of the object. Inclusion is performed.

However, there is a limit to the ability of the robot to recognize an object in the work environment, and it does not reach an object recognition ability equivalent to that of a human. In other words, in the above-described example, it is impossible to specify an object in the work environment unless information on a geometric shape (for example, a template) used in the template matching can be specified. As described above, if the robot cannot accurately identify the object existing in the work environment, the robot cannot appropriately operate or work on the object.
Therefore, the present invention has been made in view of such circumstances, and an object thereof is to provide a robot, an object processing system, and an object processing method that can easily recognize (specify) an object in a work environment from a captured image.

  An object processing system according to a first aspect of the present invention is based on an information server storing object recognition information necessary for image recognition of an object, and object processing information stored in the information server. A robot that identifies the object; and a monitoring device that monitors an environment of the robot. In this object processing system, the monitoring device obtains monitoring information by monitoring the environment of the robot, and the robot has first imaging means for imaging the surroundings and the information for object recognition from the information server. A first tag reader that reads the information for reading from the non-contact type tag attached to the object by specific contact, and the information server based on the reading information read by the first tag reader The object is identified by using first communication means for reading the object recognition information from, the object recognition information read by the first communication means and the captured image information obtained by the first imaging means. And an object specifying unit that also uses the monitoring information obtained by the monitoring unit for specifying the object.

  An object processing system according to a second aspect of the present invention is the object processing system according to the first aspect of the present invention, wherein the monitoring device is attached to a second imaging means for imaging the surroundings and the object. A second tag reader that reads the reading information from the non-contact type tag that is in a specific contact; and a second tag reader that reads the object recognition information from the information server based on the reading information read by the second tag reader. The object is identified using the communication means, the object recognition information read by the second communication means and the captured image information obtained by the second imaging means, and the identification result is obtained as the monitoring information. And transmitting means for transmitting to the robot.

  An object processing system according to a third aspect of the present invention is the object processing system according to the first or second aspect of the present invention, wherein the robot writes a tag writer that writes information into the tag and the object specifying means specifies the object processing system. Object information writing means for writing information on the object to the tag by the tag writer, and the tag reader reads information on the object written on the tag by the object information writing means.

  According to a fourth aspect of the present invention, there is provided an object processing method for storing object recognition information necessary for image recognition of an object in an information server and reading the object recognition information from the information server. Information is stored in a non-contact tag attached to the object, the monitoring environment is obtained by monitoring the environment of the robot by a monitoring device, and the robot acquires surrounding captured image information by an imaging means. In addition, the reading information is read from the non-contact type tag by specific contact using a tag reader, the object processing information is read from the information server based on the read reading information, and the read object recognition information is read. And the captured image information are used to specify the object, and the monitoring information is also used to specify the object.

An object processing method according to a fifth aspect of the present invention is the object processing method according to the fourth aspect of the present invention, wherein the robot writes information on the identified object to the tag by a tag writer, When information on the object is written on the tag, the information on the object is read by the tag reader.
According to a sixth aspect of the present invention, there is provided a robot according to the object processing information necessary for recognizing an object stored in the information server and the monitoring information obtained by the monitoring device monitoring the environment of the robot. A robot for identifying the object. The robot includes an imaging unit that images the surroundings, a tag reader that reads read information for reading the object recognition information from the information server from a non-contact tag attached to the object, and Based on the reading information read by the tag reader, communication means for reading the object recognition information from the information server, the object recognition information read by the communication means, and the captured image information obtained by the imaging means. And an object specifying means for specifying the object and using the monitoring information for specifying the object.

  In the first, fourth, and sixth aspects of the invention, the information for object recognition necessary for image recognition of the object is stored in the information server, and the reading information for reading the object recognition information from the information server is stored. It is premised that the information is stored in a non-contact tag attached to the object. Then, the robot obtains the surrounding captured image information by the imaging means (first imaging means), reads the information for reading from the non-contact type tag by the tag reader (first tag reader), and reads the information. Based on the reading information, the object processing information is read from the information server, and the object is specified using the read object recognition information and captured image information. For identifying the object, monitoring information obtained by a monitoring device that monitors the environment of the robot is also used.

That is, only the object recognition information suitable for specifying the object based on the information of the tag attached to the object by the robot itself is obtained from the information server, and the object is obtained using the object recognition information and the captured image information. Processing to identify is performed. The monitoring information obtained by the monitoring device that monitors the environment of the robot is also used for specifying the object. For example, the environment of the robot includes an environment including an object specified by the robot.

According to the present invention, since the object specific process is performed using only the object recognition information suitable for the object specific process, the object can be specified easily and reliably. Then, by using the monitoring information, the accuracy of identifying the object can be increased.
According to the third and fifth aspects of the present invention, since the information on the specified object is written on the tag, the information on the specified object written on the tag when the robot tries to specify the object later. For example, by reading the position information of the object, the object can be specified without using the object processing information.

The best mode for carrying out the present invention will be described in detail with reference to the drawings.
In FIG. 1, there are six objects (three-dimensional objects) 501, 502, 503, 504, 505, and 506 in the work environment (work space) W, and the state of the robot 1 moving in the work environment W is shown. Show.
The robot 1 is configured to be able to perform operations or work on the objects 501 to 506 existing in the work environment W by providing the arm 3 on the upper part of the movable carriage 2. The robot 1 is connected to an information server 101 via an information communication network NET such as the Internet.

  The information server 101 manages information (hereinafter referred to as object recognition information) OBS for the robot 1 to recognize the objects 501 to 506. Examples of the object recognition information OBS include information on the geometric shapes and colors of the objects 501 to 506. Then, initial writing or uploading of the object recognition information OBS to the information server 101 (initial value writing or uploading) is performed by the manufacturer of the objects 501 to 506, the company entrusted by the manufacturer, or the like. Is called.

The installation location of the information server 101 is an accessible location on the information communication network NET using, for example, the Internet protocol, and is an arbitrary location on the information communication network NET.
For example, the information server 101 is managed by a ubiquitous knowledge management company that is an information management company such as a provider. In this case, the object recognition information OBS is managed by the ubiquitous knowledge management company, and written or uploaded to the information server 101 by the ubiquitous knowledge management company.

  For example, when there are many objects (products), if the creation of the object recognition information OBS is left to the company that manufactures the objects (products), the standards for the object operation information OB can be unified. The usefulness of the object operation information OBS is low. If the standard of the object operation information OB is not unified, for example, the creation of software that handles the object operation information OBS needs to be performed under a completely different method for each object operation information OBS. For this reason, the ubiquitous knowledge management company manages the object recognition information OBS existing for each object (product) as the one of a certain standard and manages it by the information server 101.

The objects 501 to 506 are objects handled by the robot 1 in the work environment W. As shown in FIG. 2, tags 501 a to 506 a are attached to the objects 501 to 506. Each of the objects 501 to 506 has a unique shape, weight, and the like. That is, each object 50
1 to 506 have characteristics indicated by the object recognition information OBS.
Examples of the tags 501a to 506a include wireless IC tags such as RFID (Radio Frequency Identification). RFID has a size of about several centimeters excellent in environmental resistance, and has a structure capable of electronically storing data and communicating with a reader by radio waves or electromagnetic waves.

  Information (hereinafter referred to as object ID information) OB for identifying the objects 501 to 506 to which the tags 501a to 506a are attached is written in the tags 501a to 506a. The object ID information OB includes address information of the information server 101 that manages the object recognition information OBS. The address information includes information for specifying the information server 101 on the information communication network NET (hereinafter referred to as server address information) and a specific memory in which the object recognition information OBS is stored in the specified information server. It includes information on areas (for example, IP (Internet Protocol)) information (hereinafter referred to as data address information). The object ID information OB includes information that is an appearance characteristic of the object and can be represented with a small amount of data, such as color information indicating the color of the object, information on a two-dimensional shape (planar shape), and the like. include. This object ID information constitutes reading information for reading the object recognition information OBS from the information server 101 into the robot 1.

The robot 1 includes a tag reader / writer 4, a control unit 5, a drive unit 6, an address processing unit 7, a network processing unit 8, and an imaging unit 9, as shown in FIGS.
The tag reader / writer 4 reads and writes information from and to the tags 501a to 506a of the objects 501 to 506. As shown in FIG. 1, the tag reader / writer 4 is attached to the arm 3 together with the imaging unit 9. The tag reader / writer 4 is configured to read / write information from / to the tags 501a to 506a wirelessly. There are various methods for reading and writing information with respect to the tags 501a to 506a. For example, the tag reader / writer 4 uses a standard function provided in a general IC tag technology to provide a certain amount of information. The information is read from and written to the tags 501a to 506a existing in the area. Therefore, when there are a plurality of tags (objects) in a certain area from the tag reader / writer 4, the tag reader / writer 4 automatically reads information from all the tags.

The address processing unit 7 reads address information from the object ID information read from the tags 501 a to 506 a by the tag reader / writer 4.
The network processing unit 8 accesses the information server 101 via the information communication network NET based on the server address information of the address information read by the address processing unit 7. Then, based on the data address information of the address information, the network processing unit 8 reads the object recognition information OBS stored in the predetermined storage area of the accessed information server via the information communication network NET. Then, the network processing unit 8 outputs the object recognition information OBS to the control unit 5.

On the other hand, the imaging unit 9 is for imaging a work environment and is configured to be stereoscopically viewable. For example, the imaging unit 9 includes two CCD (Charge Coupled Device) sensors as a stereotype. The imaging unit 9 outputs the captured image to the control unit 5.
The control unit 5 is configured to control various processes and the robot 1. Specifically, processing by the control unit 5 and control of the robot 1 are as follows.

The control unit 5 uses the captured image from the imaging unit 9 and the geometric shape information of the object recognition information OBS to perform object recognition processing, for example, template matching (in this case, the geometric shape of the object recognition information OBS). The information forms a template.) By which the object in the captured image is specified. Here, since various objects exist in the work environment, in specifying the object, which of the various objects is the object of the captured image is specified. Usually, an object in a captured image is a target of operation or work, and thus the specified object is a target of operation or work.

As described above, the object recognition information OBS is acquired based on the object ID information OB in the tag, and the object ID information OB acquired from the tag is the robot 1 (specifically, tag reader / writer). This is the object ID information OB obtained from the tag of the object (specifically tag) within the certain range when the tag (object) within the certain range is located from 4).
For this reason, there is an object imaged by the imaging unit 9, and only the object (specifically, tag) exists within a certain range from the robot 1 (specifically, tag reader / writer 4). First, the object recognition information OBS is obtained from the information server 101 based on the object ID information OB in the tag of the object. In this case, the control unit 5 can identify an object in the captured image by recognition processing using the captured image from the imaging unit 9 and the geometric shape information of the sole object recognition information OBS.

  On the other hand, one object is imaged by the imaging unit 9, but there are a plurality of objects other than the object (specifically a tag) being imaged within a certain range from the robot 1 (specifically, the tag reader / writer 4). When an object (specifically, a tag) exists, a plurality of pieces of object recognition information OBS are information based on the object ID information OB in the tags of the plurality of objects including the imaged object. Obtained from the server 101. In this case, the control unit 5 identifies an object in the captured image by recognition processing using the captured image from the imaging unit 9 and the plurality of object recognition information OBS. That is, the control unit 5 sequentially matches the plurality of object recognition information OBS with the captured image, and finally specifies the object in the captured image.

As described above, the control unit 5 identifies the object in the captured image by the recognition process using the captured image from the imaging unit 9 and the object recognition information OBS.
Further, as described above, the object ID information includes information that is an appearance characteristic of the object and can be represented with a small amount of data, for example, color information indicating the color of the object, a two-dimensional shape (planar shape). Information is included. For this reason, the control unit 5 performs a recognition process using the color information obtained from the object ID information OB and the captured image from the imaging unit 9 prior to the above-described recognition process. When the object can be specified by the recognition process using the color information obtained from the object ID information OB, the control unit 5 does not perform the recognition process using the captured image and the object recognition information OBS. Like that. As will be described later, since the object recognition information OBS is information necessary for creating the work environment model WM, the object recognition information OBS is read from the information server based on the object ID information OB. To do.

Then, in order to perform operations and work on the objects 501 to 506 specified by the object recognition information OBS or color information, the control unit 5 includes a control program PG for controlling the robot 1 and work as shown in FIG. An environmental model WM is stored.
The work environment model WM is constructed as information necessary for operating an object. The object recognition information OBS includes the geometric shape of the object as described above, and a work environment model WM for operating the object is constructed based on the geometric shape in the object recognition information OBS. ing. That is, the object recognition information OBS constitutes information for recognizing an object, and also constitutes information for operating the recognized object (object operation information). For this reason, on the premise that the object recognition information OBS is used as information for manipulating the object in this way, the object recognition information OBS has a weight in addition to information such as a geometric shape. Other information unique to the object, such as information indicating the operation content or work content of the object may be included. Here, the operation content is an operation of gripping an object and the like, and the work content is an operation of transporting an object to a predetermined place. For example, the operation information for gripping an object includes information on how to hold the object and where it is held.

Here, the work environment model WM held by the control unit 5 is made up of data constructed to some extent in advance, and may be completed or updated based on the object recognition information OBS.
The control program PG is constructed by a work plan algorithm. The work plan algorithm includes an algorithm for creating a work plan indicating a work procedure by the robot 1 based on the work environment model WM. The program PG to be controlled is information or a program for operating the robot 1, is determined or designed by the designer of the robot 1, and is stored in the control unit 5 in advance.

  The control unit 5 creates a work plan for the robot 1 based on the work environment model WM using the control program PG. Here, the creation of the work plan of the robot 1 based on the work environment model WM by the control program PG is performed using, for example, a conventional work plan creation technique. That is, the control unit 5 refers to the size of the robot 1 and the work environment model WM, performs mathematical arithmetic processing such as checking the interference between the robot 1 and the objects 501 to 506, and creates a work plan. The created work plan includes information (behavior control) indicating how to control the drive unit 6 in order to realize the work. For example, the control unit 5 determines an operation position (position, etc.) in the objects 501 to 506 based on the geometric shape and weight. Here, the drive part 6 is comprised by the mobile trolley 2, the arm 3, etc. FIG.

The control unit 5 controls the drive unit 6 by outputting a control signal to the drive unit 6 based on the work plan.
Thereby, the robot 1 moves by an autonomous action or performs an operation or work on the objects 501 to 506. Specifically, the robot 1 holds the objects 501 to 506 and conveys them to a predetermined place.
In some cases, the robot 1 performs a process of updating the object recognition information OBS.
Although the robot 1 performs autonomous behavior based on the work plan, the robot 1 may perform an operation in which an error occurs between the actual objects 501 to 506. For example, the unique information of the object may change. For example, the shape and weight of the object change with time. Even when the unique information of the object changes in this way, if the work plan is created based on the object recognition information OBS obtained from the information server 101, the robot 1 performs appropriate movement or the objects 501 to 501. An appropriate work cannot be performed on the object 506, that is, an operation with an error occurs between the actual objects 501 to 506.

  For this reason, when the robot 1 detects an operation that causes an error with the real objects 501 to 506, that is, when it detects a change in the unique information of the objects 501 to 506 such as the shapes of the objects 501 to 506. The object recognition information OBS is updated. In this case, the network processing unit 8 accesses the information server 101 and updates the object recognition information OBS stored in the information server 101. In this case, it is preferable that the information is updated under a certain restriction such as limiting the persons who can update the information, making only a predetermined information portion in the object operation information OB. At the same time, the control unit 5 updates the object recognition information OBS and the work environment model WM held by itself.

Further, the robot 1 may write information on the object specified by the recognition processing, for example, position information of the object, on the tag of the object.
As a result, when the robot 1 performs an operation or an operation on an object, the position information written in the tag can be used. Here, for example, it may be possible to store the position information of the object in the tag from the beginning. However, when the object moves, the position of the actual object (latest position) is different from the position information in the tag. For this reason, by writing the position information of the object identified by the recognition process in the tag, the robot 1 identifies the object based on the latest position information, and performs operations and work on the object. It can be carried out. In this case, it is a premise that the robot 1 includes a means capable of recognizing the absolute position, and the robot can specify the position of the object from the position information by the means.

As for the writing of the position information to the tag, the tag reader / writer 4 as the object information writing means writes the object position information, which is the object information specified by the control unit 5 as the object specifying means, into the tag. It can be said.
For example, as the method of writing the position information on the tag, it is added to the object ID information OB stored in the tag. As a result, the robot 1 goes to read the object ID information OB in the tag by the tag reader / writer 4 as a normal operation. At this time, the robot 1 detects that position information is added to the object ID information OB. Therefore, this position information is preferentially handled. That is, the robot 1 specifies an object based on the position information without performing recognition processing using the object recognition information OBS.

  Further, as described above, the distance range in which the tag reader / writer 4 can read information from the tags 501a to 506a is constant, and the imaging range by the imaging unit 9 is constant, so that the robot 1 can detect the objects 501 to 506. When operating or working on the robot 1, the robot 1 needs to approach the objects 501 to 506 to some extent. For this reason, the robot 1 preliminarily stores information on a rough travel route in the work environment W as basic operation information, and the robot 1 stores the information in the work environment W based on the travel route information. To move. For example, the travel route information is stored in advance in the work environment model WM. Note that the basic motion information held in advance by the robot 1 is not limited to the travel route information, and may be information indicating other basic motions.

Next, a series of processing procedures will be described with reference to FIG.
First, necessary data is created (step S1). Specifically, an object manufacturer or the like creates object recognition information OBS, stores the created object recognition information OBS in the information server 101, and stores the created object recognition information OBS in the object recognition information OBS stored in the information server 101. Object ID information OB (address information) for access via the information communication network NET is written (stored) in the tags 501a to 506a of the objects 501 to 506. The designer of the robot 1 creates a work plan algorithm (control program PG), and writes (stores) the created work plan algorithm (control program PG) in the control unit 5.

Subsequently, the robot 1 reads the address information of the object ID information OB written in the tags 501a to 506a of the objects 501 to 506 by the tag reader / writer 4 (step S2). At this time, a tag to be read of the object ID information OB is a tag of an object existing in a certain area from the robot 1.
Subsequently, the robot 1 reads the object recognition information OBS from the information server 101 via the information communication network NET by the address processing unit 7 and the network processing unit 8 based on the address information (step S3).

Subsequently, the robot 1 specifies a target object (step S4). FIG. 5 shows the processing contents.
First, the control unit 5 roughly positions the main body of the robot 1 based on the captured image obtained by the imaging unit 9 (step S11). For example, the control unit 5 controls the moving carriage 2 to move the robot 1 so that the imaging unit 9 is focused on the target object.
Subsequently, the control unit 5 drives the arm 3 of the drive unit 6 so that the tag reader / writer 4 is positioned near the target object (step S12).
Subsequently, the control unit 5 identifies a target object based on the object recognition information OBS read from the information server 101 (step S13). Specifically, the control unit 5 identifies an object by recognition processing using the captured image from the imaging unit 9 and the geometric shape information of the object recognition information OBS.

As described above, the robot 1 approaches the target object, and finally identifies what the object is by the recognition process using the captured image obtained from the imaging unit 9 and the object recognition information OBS.
FIG. 6 shows a state of the robot 1 when such processing is performed. As shown in FIG. 6, the robot 1 reads the object recognition information OBS from the information server 101 based on the address information in the tags 501a to 506a of the objects 501 to 506, and based on the read object recognition information OBS. To identify the object.

After specifying the object in this way, the robot 1 creates or updates the work environment model WM based on the object recognition information OBS by the control unit 5 (step S5).
Subsequently, the robot 1 creates a work plan by the control unit 5 with reference to the work environment model WM based on the control program (work plan algorithm) PG (step S6). Then, the robot 1 acts autonomously based on the work plan (step S7). Here, the robot 1 moves by autonomous behavior or performs an operation or work on an object.

Further, the robot 1 determines whether or not the object recognition information OBS needs to be updated by the control unit 5 (step S8). That is, the control unit 5 determines whether the robot 1 has performed an operation in which an error has occurred with an actual object.
Here, the control unit 5 updates the object recognition information OBS in the information server 101 on the assumption that the object recognition information OBS needs to be updated when the robot 1 performs an operation in which an error occurs with a real object. And the work environment model is updated (step S8). For updating the object recognition information OBS in the information server 101, the network processing unit 8 accesses the information server 101 and updates the object recognition information OBS stored in a predetermined storage area.

On the other hand, when the robot 1 does not perform an operation in which an error has occurred with the actual objects 501 to 506, the control unit 5 ends the process regarding autonomous behavior, assuming that it is not necessary to update the object recognition information OBS.
Here, as an example, consider a case where there are a plurality of objects within a certain range from the robot 1.
In this case, the tag reader / writer 4 reads the object ID information OB from each of the tags of the plurality of objects (step S2). In this case, among the plurality of objects, there are objects that are imaged by the imaging unit 9, and there are also objects that are not imaged by the imaging unit 9 but only the object ID information of the tag is read. is there.

  Then, the robot 1 reads a plurality of object recognition information OBS corresponding to each object ID information OB from the information server 101 via the information communication network NET (step S3). Then, the robot 1 identifies an object by recognition processing using each object recognition information OBS (step S4). After identifying the object, the robot 1 creates or updates the work environment model WM based on the object recognition information OBS (step S5), and then creates a work plan by referring to the work environment model WM (step S5). S6), act autonomously based on the work plan (step S7).

The effect by the above process or operation | movement is demonstrated.
As described above, the robot 1 reads the address information of the object ID information OB written in the tags 501a to 506a by the tag reader / writer 4, and reads the object recognition information OBS from the information server 101 based on the address information. The object is identified by the recognition process using the object recognition information OBS and the captured image of the imaging unit 9. As described above, the robot 1 identifies an object using the optimal object recognition information OBS.

In the conventional template matching, it takes time to specify a template such as the object recognition information OBS, and it is not certain whether the template specified as such is optimal.
In addition, for example, even if an object is specified only by information stored in a tag, the tag reader / writer only reads information (specific information such as the shape of the object) stored in a tag existing in a certain range. The position of the object attached to the tag (relative position or direction relative to the robot) cannot be specified. On the other hand, even if an object is specified only by a captured image, it is difficult to specify the object for the reasons described above.

However, the object recognition information OBS can be read from the information server 101 by the number corresponding to the object (tag) within a certain range from the tag reader / writer 4 (robot 1), and the object recognition information OBS, the captured image information, It becomes possible to specify the object simply by matching.
Thus, the robot 1 can specify an object reliably by a simple process. As a result, the robot 1 can perform an optimal operation and work on the object.
Further, as described above, the robot 1 creates the work environment model MW based on the object recognition information (object operation information) OBS read from the information server 101, and also creates a work plan based on the work environment model MW. Is generated. And the robot 1 is performing the desired operation and operation | work requested | required of the said objects 501-506 with respect to the objects 501-506 based on the work plan.

Here, the object recognition information (object operation information) OBS for creating the work environment model MW is created by manufacturing the objects 501 to 506 and the objects 501 to 501 such as a business operator entrusted by the manufacturer. It is carried out by those who are involved in the manufacture of 506 and the like.
Conventionally, it is necessary to store information corresponding to such object recognition information (object operation information) OBS in the robot in advance when storing a program for defining behavior control of the robot, The information is created by the person who designs the behavior control of the robot such as the robot designer.

However, there are an infinite number of objects handled by the robot, and when it is considered to be processed by the robot, the robot designers observe the robot's work environment, interpret it, In addition, it is difficult to create work environment information based on the determination.
On the other hand, the object recognition information (object operation information) OBS is created by the manufacturer of the objects 501 to 506 and the person involved in the manufacture of the objects 501 to 506 such as a business contracted by the manufacturer. Has been done by. That is, a person who knows details about the objects 501 to 506 and has expert knowledge creates the object operation information OB.

  As a result, the manufacturer of the objects 501 to 506 creates the object recognition information (object operation information) OBS on behalf of the robot designer or the like, thereby reducing the burden on the robot designer or the like. Further, the object recognition information (object operation information) that accurately shows the unique information of the object by creating the object recognition information (object operation information) OBS by a person who knows details about the object and has expertise. ) OBS can be created. As a result, the robot 1 can accurately process the object such as the robot 1 accurately grasping the object.

Next, a system will be described in which a robot identifies objects existing in a work environment with reference to object identification results of a plurality of object identification devices.
FIG. 7 shows the configuration of the system.
In this system, a plurality of (six in this example) object specifying devices 201, 202, 203, 204, 205, 206 are connected on the information communication network NET, and the robot 1 is connected to these object specifying devices 201, The object identification results 202, 203, 204, 205, and 206 are also obtained, and the objects 501 to 506 existing in the work environment are finally identified.

The object identifying devices 201 to 206 are monitoring devices that monitor the environment of the robot 1 and obtain monitoring information. As shown in FIGS. 7 and 8, the tag reader / writer 211, the imaging unit 212, the address processing unit 213, the network A processing unit 214 and a processing unit 215 are provided. The object specifying devices 201 to 206 are fixedly set at predetermined positions in the work environment.
The tag reader / writer 211, the imaging unit 212, the address processing unit 213, and the network processing unit 214 have the same functions as those provided in the robot 1.

That is, the tag reader / writer 211 reads and writes information on the tag of the object wirelessly. At this time, the tag reader / writer 211 reads / writes information from / to the tags 501a to 506a (objects 501 to 506) existing in a certain area.
The address processing unit 213 reads address information for accessing the information server 101 from the object ID information OB read from the tags 501a to 506a by the tag reader / writer 211.

  The network processing unit 214 accesses the information server 101 via the information communication network NET based on the server address information of the address information read by the address processing unit 211. Then, based on the address information, the network processing unit 214 reads the object manipulation information OBS stored in the predetermined storage area of the accessed information server 101 via the information communication network NET. The network processing unit 214 outputs the object recognition information OBS read from the information server 101 to the processing unit 215.

The network processing unit 214 is also configured to transmit various information to the robot 1 via the information communication network NET.
Here, the address processing unit 213 and the network processing unit 214 constitute communication means (second communication means) for reading the object recognition information OBS from the information server 101 based on the object ID information OB read by the tag reader / writer 211. doing.
The imaging unit 212 is for imaging the work environment of the robot 2 and is configured to be stereoscopically viewable. The imaging unit 212 outputs the captured image to the processing unit 215.
The processing unit 215 performs object recognition processing using the captured image from the imaging unit 212 and the geometric shape information of the object recognition information OBS, for example, template matching (in this case, the geometric shape information of the object recognition information OBS). Is a template.) Identifies the objects 501 to 506.

  Here, since the object ID information OB includes color information and two-dimensional shape information, the processing unit 215 performs processing similar to that of the control unit 5 of the robot 1 using this. That is, the processing unit 215 performs recognition processing using color information obtained from the object ID information OB and the captured image from the imaging unit 212 prior to the above-described recognition processing. When the object can be identified by the recognition process using the color information obtained from the object ID information OB, the processing unit 215 does not perform the recognition process using the captured image and the object recognition information OBS. To do.

Then, the processing unit 215 transmits an object identification result, which is a monitoring result obtained based on the object recognition information OBS or the color information of the object ID information, to the robot 1 via the information communication network NET. Specifically, the processing unit 215 outputs the object specifying result to the network processing unit 214, and the network processing unit 214 transmits the object specifying result to the robot 1 via the information communication network NET. Here, the information communication network NET that connects the object specifying devices 201 to 206 and the robot 1 so as to be communicable is constituted by, for example, a LAN (Local Area Network). Further, the object specifying result is, for example, position information of the object, and the processing unit 215 attaches the object ID information to the position information of the object and transmits it to the robot 1 as the object specifying result.

Here, the processing unit 215 and the network processing unit 214 specify an object using the object recognition information OBS read by the function of the communication unit and the captured image obtained by the imaging unit 212, and specify the object. A transmission means for transmitting the result to the robot is configured.
The robot 1 receives the object specifying result from the object specifying devices 201 to 206 by the network processing unit 214.

  In the robot 1, the control unit 5 performs an object recognition process using the geometric shape information of the object recognition information OBS obtained from the information server 101 and the captured image from the imaging unit 9, and the object 501 to be a work target. ˜506 is specified. On the other hand, the control unit 5 specifies the objects 501 to 506 based on the object specifying results of the object specifying devices 201 to 206. That is, the control unit 5 uniquely specifies the objects 501 to 506 and also specifies the objects 501 to 506 based on the object specifying results of the object specifying devices 201 to 206.

Next, a series of processing procedures will be shown.
The processing of the robot 1 and the processing between the robot 1 and the information server 101 are the same as the processing contents shown in FIG.
On the other hand, the object identifying devices 201 to 206 perform processing as shown in FIG.
First, the object identifying devices 201 to 206 read the address information of the object ID information OB written in the tags 501a to 506a of the objects 501 to 506 by the tag reader / writer 211 (step S21). At this time, a tag to be read of the object ID information OB is a tag of an object existing in a certain area from the robot 1.

Subsequently, the object identifying devices 201 to 206 read the object recognition information OB from the information server 101 via the information communication network NET by the address processing unit 213 and the network processing unit 214 based on the address information (step S22). ).
Subsequently, the object specifying devices 201 to 206 specify an object based on the read object recognition information OBS by the processing unit 215 (step S23), and transmit the object specifying result to the robot 1 (step S24). ).

Corresponding to the processing of the object identification devices 201 to 206, the robot 1 performs autonomous behavior according to the processing procedure shown in FIG.
In the processing procedure when the robot performs autonomous action, the processing of step S31 is performed instead of the processing of step S13, as can be seen by comparing FIG. 5 and FIG.
That is, as step S31, the control unit 5 of the robot 1 specifies an object based on the object recognition information OBS directly read from the information server 101 and the object specifying results of the object specifying devices 201 to 206. That is, the robot 1 uniquely identifies the objects 501 to 506 and also identifies the objects 501 to 506 based on the object identification results of the object identifying apparatuses 201 to 206.

FIG. 11 shows a state of the robot 1 when performing such processing. As shown in FIG. 11, the robot 1 reads the object recognition information OBS from the information server 101 based on the address information of the tags 501a to 506a of the objects 501 to 506, and the read object recognition information OBS and the object identification. The object is specified based on the object specifying result of the devices 201 to 206.
The effect by the above process or operation | movement is demonstrated.
As described above, the robot 1 reads the object recognition information OBS from the information server 101 based on the address information of the tags 501a to 506a of the objects 501 to 506, and the read object recognition information OBS and the object specifying device 201. The target object is identified based on the object identification results of ˜206. That is, the robot 1 finally specifies an object based on a plurality of object specifying results including its own object specifying result. Thereby, the robot 1 can specify an object reliably. As a result, the robot 1 can perform an optimal operation and work on the object.

  For example, the imaging range by the imaging unit 9 is limited. For example, if the robot 1 moves or moves the arm 3, the imaging unit 9 causes the object to be focused by the imaging unit 9 even though the imaging unit 9 has initially captured the object reliably. In some cases, it is impossible to reliably capture an image. Even in such a case, the robot 1 can specify an object by using the object specifying results of the object specifying apparatuses 201 to 206. In particular, when the arm 3 includes the imaging unit 9 as in the present embodiment, the operability of the imaging unit 9 is increased or the imaging range is widened, but the arm 3 is used to operate an object. If it moves, the imaging condition by the imaging part 9 will change. Even in such a case, the object can be reliably specified by using the object specifying results of the object specifying apparatuses 201 to 206.

In addition, the robot 1 can specify objects other than those captured by the imaging unit 9, that is, all objects existing in the work environment, based on the object specifying information of the object specifying apparatuses 201 to 206. .
The embodiment of the present invention has been described above. However, the present invention is not limited to being realized as the above-described embodiment.
That is, in the above-described embodiment, the case where the processing means is the arm 3 as the operation means or the mobile carriage 2 as the movement means is described, and further, the operation content and the processing content that is the work content are specifically described. did. However, it goes without saying that the processing means and processing contents are not limited to this.
In the above-described embodiment, the case where the tags 501a to 506a are RFIDs has been described. However, it is needless to say that the present invention is not limited to this, and the tags 501a to 506a only need to be capable of reading and writing information wirelessly.

The tag reader / writer 4 may be anything as long as it can read / write information from / to the tags 501a to 506a. In a scene where it is not necessary to write information to the tags 501a to 506a, the robot 1 may be provided with a read-only tag reader instead of the tag reader / writer 4.
In the above-described embodiment, the case has been described in which the object operation information OB includes information on the geometric shape, color, and weight of the object. However, it goes without saying that the present invention is not limited to this. That is, for example, the object operation information OB may include material information.

  Further, in consideration of the aspect that the object recognition information OBS indicates information necessary for manipulating the object (object operation information), the object recognition information OBS is used for what purpose the object is. It may indicate what the object manufacturer expects to be handled by the robot. The form of the object recognition information OBS may be any form that can be described as numerical information, digital information, or language information. This is because it is possible to write the object recognition information OBS in the tags 501a to 506a or the information servers 101 to 106 by using the current information engineering and computer engineering techniques.

In the above-described embodiment, the case has been described in which the shape information of the object recognition information OBS used for object recognition is three-dimensional geometric shape information. However, the present invention is not limited to this. In the case of a system that obtains an object identification result using the object identification devices 201 to 206 described above, the shape information of the object recognition information OBS may be two-dimensional shape information. In this case, for one object, a plurality of types of two-dimensional shape information obtained by viewing the object from each direction are prepared. As a result, the object specifying devices 201 to 206 are three-dimensionally arranged in the work environment, so that the object specifying devices 201 to 206 specify the object based on the two-dimensional shape information, and the robot 1 also The object can be specified by the two-dimensional shape information. In the case of two-dimensional shape information, it is considered that an object specifying device that cannot specify an object will come out, but eventually, the robot 1 has a plurality of objects obtained from each of the object specifying devices 201 to 206. The identification result is evaluated by probability calculation or the like to identify the object.

  For example, it is considered that the amount of data can be reduced by using the two-dimensional shape information rather than the three-dimensional shape information. For this reason, when the shape information of the object recognition information OBS is two-dimensional shape information, the information server 101 can manage the object recognition information OBS with a reduced load. In addition, the load of recognition processing in the robot 1 and the object identifying devices 201 to 206 can be reduced.

  In the above-described embodiment, the case where the object recognition information OBS includes information necessary for recognizing the object and information necessary for manipulating the object (object operation information) has been described. . However, the present invention is not limited to this. Apart from the object recognition information OBS that is information necessary for recognizing an object, object operation information that is information necessary for operating the object is prepared. Also good. In this case, for example, the object operation information is written in an information server or tag. For example, when the information is written on the tag, the robot 1 reads the object operation information from the tag by the tag reader / writer 4 and creates (completes or updates) the work environment model based on the object operation information.

  Further, in the above-described embodiment, the case where the object specifying devices 201 to 206 are fixedly installed in the work environment W has been described. However, it is not limited to this. That is, the object identifying devices 201 to 206 may be moved in the work environment W. For example, the object identifying devices 201 to 206 may be provided in another robot 1. That is, the tag reader / writer 4 and the imaging unit 9 of the other robot 1 constitute the tag reader / writer 211 and the imaging unit 212 of the object identifying devices 201 to 206.

  The object specifying devices 201 to 206 are not limited to the tag reader / writer 4 and the imaging unit 9. For example, the object identifying devices 201 to 206 may include one of the tag reader / writer 4 and the imaging unit 9, or other detection means. As described above, the object specifying devices 201 to 206 only have to be configured to obtain information that supplements the object specification performed by the robot 1.

For example, consider a case where the object identifying devices 201 to 206 include only the tag reader / writer 4. The tag reader / writer 4 can read information in a tag existing within a certain range. For this reason, if the object specifying devices 201 to 206 are arranged at a predetermined position, the data readable area by the tag reader / writer 4 of the object specifying devices 201 to 206 is a fixed region. When the tag reader / writer 4 of the object specifying devices 201 to 206 can acquire information, the position of the tag (object) from which the information is acquired can be specified to some extent. Therefore, the object specifying devices 201 to 206 transmit the position information of the object and the object ID information OB of the object as the object specifying information to the robot 1, so that the robot 1 can detect the position of the object according to the object specifying information. Or an object can be specified.
In the above-described embodiment, the number of objects existing in the work environment and the number of object specifying devices have been described as specific, but it goes without saying that the present invention is not limited to this.

It is a figure which shows the mode of the robot which performs operation or work with respect to an object in a working environment. It is a figure which shows the object with which the tag was mounted | worn. It is a block diagram which shows the structure of the said robot. It is a flowchart which shows the system various processes including the process by the said robot. It is a flowchart which shows the process in which the said robot pinpoints the position of the object of work object. It is a figure which shows the mode of the robot at the time of pinpointing the position of the object of work object. It is a figure which shows the system in which the said robot got the recognition result from the apparatus for object identification. It is a block diagram which shows the structure of the said apparatus for object identification. It is a flowchart which shows the process of the said apparatus for object identification. 5 is a flowchart showing a process in which the robot specifies a position of an object to be worked in a system in which the robot obtains a recognition result from an object specifying device. It is a figure which shows the mode of the robot at the time of specifying the position of the object of work object in the system which the said robot acquired the recognition result from the apparatus for object specification. It is the figure used for description of the operation | work by the autonomous behavior of the robot in a certain work environment in the past. It is the figure used for description of the method of storing the data and program which a robot uses conventionally in the said robot.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Robot 2 Mobile trolley 3 Arm 4 Tag reader / writer 5 Control part 6 Drive part 7 Address book process part 8 Network processing part 9 Imaging part 101 Information server 201-206 Object identification apparatus 211 Tag reader / writer 212 Imaging part 213 Address processing part 214 Network Processing unit 215 Processing unit 501 to 506 Object 501a to 506a Tag 510 Product (object)
510a Tag OB Object ID information OBS Object recognition information PG Control program W Work environment WM Work environment model

Claims (6)

An information server that stores object recognition information necessary for image recognition of an object, a robot that identifies the object based on object processing information stored in the information server, and an environment of the robot A monitoring device for monitoring,
The monitoring device obtains monitoring information by monitoring the environment of the robot,
The robot reads a first imaging means for imaging the surroundings and reading information for reading the object recognition information from the information server from a non-contact tag attached to the object by specific contact. 1 tag reader, first communication means for reading the object recognition information from the information server based on the reading information read by the first tag reader, and the object read by the first communication means An object identifying unit that identifies the object using the recognition information and the captured image information obtained by the first imaging unit, and also uses the monitoring information obtained by the monitoring device to identify the object. Characteristic object processing system.   The monitoring device includes: a second imaging unit that images the surroundings; a second tag reader that reads the reading information from a non-contact tag attached to the object by specific contact; and the second tag reader reads Based on the reading information, the second communication means for reading the object recognition information from the information server, the object recognition information read by the second communication means, and the second imaging means are obtained. The object processing system according to claim 1, further comprising: a transmission unit that identifies the object using the captured image information and transmits the identification result to the robot as the monitoring information. The robot includes: a tag writer that writes information to the tag; and an object information writing unit that writes information on an object specified by the object specifying unit to the tag by the tag writer.
The object processing system according to claim 1, wherein the tag reader reads information on the object written on the tag by the object information writing unit. Non-contact type tag in which information for object recognition necessary for image recognition of an object is stored in an information server, and reading information for reading the object recognition information from the information server is attached to the object And monitoring information is obtained by monitoring the robot environment with a monitoring device,
The robot acquires surrounding captured image information by an imaging unit, reads the information for reading from the non-contact type tag by specific contact using a tag reader, and reads the information from the information server based on the read information for reading. An object processing method comprising: reading object processing information, specifying the object using the read object recognition information and the captured image information, and using the monitoring information for specifying the object. The robot writes information of the identified object on the tag by a tag writer,
5. The object processing method according to claim 4, wherein when the object information is written in the tag, the robot reads the object information by the tag reader. An object processing information necessary for recognizing an object stored in an information server and a monitoring device that identifies the object based on monitoring information obtained by monitoring the environment of the robot;
An imaging means for imaging the surroundings, a tag reader for reading information for reading the object recognition information from the information server from a non-contact type tag attached to the object, and the tag reader Based on the reading information, the object using the communication means for reading the object recognition information from the information server, the object recognition information read by the communication means, and the captured image information obtained by the imaging means. And an object specifying unit that uses the monitoring information for specifying the object.

Images