JP2016206066A - Object recognition apparatus and object recognition method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform object recognition at high speed.SOLUTION: A plurality of regions are preset in a robot working environment. An object recognition apparatus comprises: storage means for storing an object category of each set region, identification information of an object belonging to the object category, and shape information of the object in association with one another; distance information acquisition means for acquiring distance information of an object to be recognized; region specification means for specifying a region where an object is present; candidate detection means for detecting an object in a region on the basis of the distance information of an object acquired by the distance information acquisition means; and recognition means for recognizing an object candidate detected by the candidate detection means, with use of the information stored in the storage means, by comparing with the shape information of an object belonging to the object category of the region specified by the region specification means.SELECTED DRAWING: Figure 3

Description

  The present invention relates to an object recognition device and an object recognition method for recognizing an object in a work environment of a robot.

  There is known an object recognition device that recognizes an object by acquiring shape information of an object existing in a work environment and collating the acquired shape information with shape information stored in advance (Patent Literature). 1).

JP 2010-060451 A

  The target object existing in the work environment is set as an object candidate, and the shape information of the object candidate is acquired and collated. For this reason, as the number of object candidates to be collated increases, it takes much time for object recognition.

  The present invention has been made to solve such problems, and has as its main object to provide an object recognition apparatus and an object recognition method capable of performing object recognition at high speed.

One aspect of the present invention for achieving the above object is an object recognition apparatus in which a plurality of areas are set in advance in a robot work environment and recognize objects in the respective areas. Storage means for storing the object category, identification information of the object belonging to the object category, and shape information of the object in association with each other, distance information acquisition means for acquiring distance information of the object to be recognized, Area specifying means for specifying an area where the object exists, candidate detection means for detecting an object in the area as an object candidate based on distance information of the object acquired by the distance information acquiring means, and candidate detection The object candidate detected by the means is compared with the shape information of the object belonging to the object category of the area specified by the area specifying means using the information stored by the storage means. Ri comprises recognizing means for recognizing, and it is an object recognition apparatus according to claim.
In this aspect, the recognizing unit compares the object candidate detected by the candidate detecting unit with the shape information of the object belonging to the object category of the region specified by the region specifying unit, and then the object in the region You may recognize by comparing with the shape information of the object which does not belong to a category.
In this aspect, the storage unit stores the set area and the position information of the robot when the distance information is acquired in association with each other, and the distance information acquisition unit stores the distance information in the area. You may acquire the distance information of the said recognition target object in a corresponding position.
In this aspect, mark information indicating a position of the area is set in the area in the work environment, and the area specifying unit may specify the area based on the mark information.
One aspect of the present invention for achieving the above object is an object recognition method in which a plurality of areas are set in advance in a work environment of a robot, and an object in each area is recognized. Storing the object category, the identification information of the object belonging to the object category, and the shape information of the object in association with each other, obtaining the distance information of the object to be recognized,
Identifying a region where the object exists, detecting an object in the region as an object candidate based on the acquired distance information of the object, and storing the detected object candidate And a step of recognizing the information by comparing it with shape information of an object belonging to the object category of the specified region using the information.

  ADVANTAGE OF THE INVENTION According to this invention, the object recognition apparatus and object recognition method which can perform object recognition at high speed can be provided.

1 is a perspective view illustrating a schematic configuration of a robot according to an embodiment. 1 is a block diagram illustrating a schematic system configuration of a robot according to an embodiment. 1 is a block diagram illustrating a schematic system configuration of an object recognition apparatus according to an embodiment. It is a figure which shows an example of each area | region set in the work environment. It is a figure which shows an example of table information. It is a figure which shows an example of the mark information set to each area | region in a work environment. It is a flowchart which shows an example of the processing flow of the object recognition method which concerns on embodiment.

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a perspective view showing a schematic configuration of a robot according to an embodiment of the present invention. The robot 1 according to the present embodiment is configured as an autonomous work robot that performs work autonomously, for example. The object recognition apparatus according to the present embodiment is mounted on the autonomous work robot as described above and recognizes an object.
FIG. 2 is a block diagram showing a schematic system configuration of the robot according to the embodiment of the present invention. The robot 1 according to this embodiment includes a robot body 2, a moving device 3 that moves the robot body 2, a gripping operation device 4 that grips and moves an object, and a control device that controls the moving device 3 and the gripping operation device 4. 5, a distance sensor 6, and an object recognition device 7.

The moving device 3 rotates a plurality of wheels by driving a motor or the like, for example, according to a control signal from the control device 5, and moves the robot body 2 to a desired position.
The gripping operation device 4 includes, for example, a gripping part 41 that grips an object, a plurality of links 43 that are connected via joint parts 42 such as a wrist joint, an elbow joint, and a shoulder joint, and a motor that drives each joint part 42. It is comprised as an articulated arm which consists of a plurality of actuators.

  The control device 5 causes the robot 1 to perform work by controlling the moving device 3 and the gripping operation device 4 based on the operation signal input via the operation terminal 8. The operation terminal 8 is a portable terminal such as a tablet terminal, a PC (Personal Computer), or a smartphone. The operation terminal 8 is provided with a touch panel and a mechanical switch. The user can input operation information for operating the robot 1 to the operation terminal 8 by operating these touch panels and mechanical switches. The operation terminal 8 may be provided with a microphone, and operation information may be input to the operation terminal 8 through the microphone.

  The operation terminal 8 and the control device 5 are connected by wireless or wired communication, and exchange data with each other. The operation terminal 8 outputs an operation signal corresponding to the work information input by the user to the control device 5. The operation information may be directly input to the robot 1 without using the operation terminal 8.

  The control device 5 and the object recognition device 7 are, for example, a CPU (Central Processing Unit) 5a, 7a that performs arithmetic processing or the like, a ROM (Read Only Memory) or a RAM (RAM) that stores arithmetic programs executed by the CPUs 5a, 7a, and the like. Random Access Memory (Random Access Memory) is composed of memories 5b and 7b, and a microcomputer composed of interface units (I / F) 5c and 7c for inputting / outputting signals to / from the outside. The CPU 5a, 7a memories 5b, 7b and the interface units 5c, 7c are connected to each other via a data bus or the like.

  The distance sensor 6 detects distance information of an object in the work environment. For example, the distance sensor 6 is mounted on the head of the robot 1 or the like, but is not limited thereto. The distance sensor 6 may be installed in a work environment (for example, a wall or a ceiling) in which the robot 1 operates. Thereby, it is possible to detect the distance information of the object with a wide field of view without moving the robot 1. Therefore, it is possible to specify an area in the work environment described later at high speed.

  A plurality of distance sensors 6 may be provided in the robot 1 and the operating environment. The distance sensor 6 is a sensor such as a camera (RGB-D camera, stereo camera), a laser range finder, an ultrasonic sensor, or a three-dimensional distance sensor.

  By the way, there are various objects in the work environment of the robot, and their positions and postures are also various. Conventionally, such a wide variety of objects are used as object candidates, shape information of the object candidates is acquired, and each object is recognized by collating with preset shape information. For this reason, when the number of object candidates to be collated increases, there arises a problem that much time is required for object recognition.

  On the other hand, the object recognition device 7 according to the present embodiment associates the set object category of each region, the identification information of the object belonging to the object category, and the shape information of the object. Based on the distance information of the object, the object in the specified area is detected as an object candidate, and the detected object candidate belongs to the object category of the specified area using the stored table information It recognizes by comparing with the shape information of the object.

  Thereby, it can be estimated that the object candidate detected in each specified area belongs to the object category of the area. Therefore, it is possible to easily recognize object candidates detected by narrowing down within each area by simply collating with object information of the object category of each area. That is, object recognition can be performed at high speed. Moreover, since misrecognition can be reduced, the estimation accuracy of the position and orientation of each object can also be improved.

  FIG. 3 is a block diagram illustrating a schematic system configuration of the object recognition apparatus according to the present embodiment. The object recognition device 7 according to the present embodiment includes a distance information acquisition unit 71, a storage unit 72, a region specifying unit 73, a candidate detection unit 74, and an object recognition unit 75.

  The distance information acquisition unit 71 is a specific example of a distance information acquisition unit. The distance information acquisition unit 71 uses the distance sensor 6 to acquire object distance information (including image information). The distance information is acquired by position information (x, y, z) of an object in the field of view (or scan range) of the distance sensor 6 viewed from the coordinate system of the sensor, for example. The distance information is acquired by a known method such as acquisition by a monocular camera. The distance information acquisition unit 71 outputs the acquired distance information of the object to the control device 5, the region specifying unit 73, the candidate detection unit 74, and the object recognition unit 75.

  The storage unit 72 is a specific example of storage means. The memory | storage part 72 is comprised by the memory 7b, for example. A plurality of areas are set in advance in the work environment of the robot 1. In the work environment, for example, as shown in FIG. 4, a dish area for placing a dish on a table, a glass area for placing glass, and a waste area for placing waste are set. Yes. The storage unit 72 includes, for example, the object category of each set area, the identification information of the object belonging to the object category, and the shape information of the object, and object information necessary for performing work on the object Are stored in table information in association with each other.

  Object information includes, for example, object shape information (size, weight, shape, features, etc.), object position and orientation, object gripping method, object gripping position, object gripping force, work content for the object, etc. It contains the information necessary to work on it.

For example, as shown in FIG. 5, the storage unit 72 stores object categories (dish category, glass category, waste category) of each area (dish area, glass area, waste area) and each object belonging to each object category. Table information in which object information for (large plate, small plate, square plate, mug, small glass, waste in general) is associated with each other is stored.
The table information stored in the storage unit 72 is configured such that the user can arbitrarily set and change the information via the operation terminal 8 or the like, for example.

  The area specifying unit 73 is a specific example of the area specifying means. The area specifying unit 73 specifies each area where the object is located based on the distance information acquired by the distance information acquiring unit 71 and the environmental map information stored in advance in the storage unit 72. The environment map information is information indicating a map (position information of obstacles and moving paths) in the work environment where the robot 1 performs work. In the environmental map information, the plurality of areas described above are set in advance. In the example illustrated in FIGS. 4 and 5, each region and object category on the table is described, but the present invention is not limited to this. For example, each region and object category may be set for an arbitrary object such as a floor, a desk, or a low table.

The object information of the table information may include the position of the robot 1 that is optimal for acquiring the distance information of the recognition target object corresponding to each region. Here, the optimal position of the robot 1 is, for example, a position where the entire table can be looked over as close as possible to the object when the object is placed on the table, or a position where the recognition accuracy of the object is stable. When the distance information acquisition unit 71 acquires the distance information, the control device 5 controls the moving device 3 based on the object information of the table information, and moves the robot 1 to the optimal position corresponding to each region. Thereby, the position of the robot 1 when the distance information acquisition unit 71 acquires the distance information of the object in each area is constant for each area. For this reason, the change of the illumination condition of an object becomes small and the recognition accuracy of an object improves.
In addition, each area | region of the environmental map information memorize | stored in the memory | storage part 72 is comprised so that a user can arbitrarily set and change via the operation terminal 8 etc., for example.

  The area specifying unit 73 may specify each area based on the image information of each area acquired by the distance information acquiring unit 71. This image information is, for example, mark information indicating the position of each region, such as visualized information such as lines that divide each region, coloring, and markers (identifiers) provided in each region. (FIG. 6). The area specifying unit 73 can recognize a marker based on the image information acquired by the distance information acquiring unit 71, and can specify a predetermined shape (rectangular, circular, etc.) range from the marker as an area. Thereby, the area specifying unit 73 can easily specify each area, and the time for object recognition can be further shortened.

  The candidate detection unit 74 is a specific example of candidate detection means. The candidate detection unit 74 detects each object in each region specified by the region specifying unit 73 as an object candidate based on the distance information of the object acquired by the distance information acquisition unit 71. For example, based on the distance information of the object acquired by the distance information acquisition unit 71, the candidate detection unit 74 assumes that the table is a plane and cuts out a portion protruding from the plane as an object. Then, the candidate detecting unit 74 detects the object candidate by calculating the position of the centroid of the clipped object and determining whether the position of the centroid is within each region. The candidate detection unit 74 outputs the detected object candidate to the object recognition unit 75.

  The object recognition unit 75 is a specific example of a recognition unit. The object recognizing unit 75 uses the object information detected by the candidate detecting unit 74 and the shape information of the objects belonging to the object category of the area specified by the area specifying unit 73 using the table information stored in the storage unit 72. Recognize by comparing.

  For example, the object recognition unit 75 calculates the feature amount of the object candidate detected by the candidate detection unit 74 based on the distance information of the object acquired by the distance information acquisition unit 71. Then, the object recognizing unit 75 calculates the model information (shape information) of the calculated feature amount of the object candidate and the feature amount of the object belonging to the object category of the area specified by the area specifying unit 73 stored in the storage unit 72 in advance. ), And the object candidate is recognized.

  Normally, each object is sorted into each area, and objects belonging to the object category of each area are arranged in each area. However, it is assumed that an object that does not belong to the object category of each area is mixed in each area.

  On the other hand, the object recognizing unit 75 compares the object candidate detected by the candidate detecting unit 74 with the shape information of the object belonging to the object category of the area specified by the area specifying unit 73, and then the object in the area It recognizes by comparing with the shape information of the object which does not belong to a category. As a result, an object that does not belong to the object category of each area (such as an object that is not sorted into each area) can be recognized as necessary.

FIG. 7 is a flowchart illustrating an example of a processing flow of the object recognition method according to the present embodiment.
The user sets each area (for example, a dish area, a glass area, a waste area, etc. on the table) in the environmental map information in the storage unit 72 via the operation terminal 8 (step S101). The user sets the object category, the object belonging to the object category, and the object information for each area of the table information in the storage unit 72 via the operation terminal 8 (step S102).

  The user gives a work instruction (for example, work position (living table) and work content (cleaning work)) of the robot 1 to the control device 5 via the operation terminal 8 (step S103). The control device 5 moves based on the distance information acquired by the distance information acquisition unit 71 and the environmental map information stored in advance in the storage unit 72 according to the operation signal of the work instruction from the operation terminal 8. The apparatus 3 is controlled to move the robot 1 to the work position.

The distance information acquisition unit 71 acquires the image information of the work position (step S104), and outputs it to the region specifying unit 73, the candidate detection unit 74, and the object recognition unit 75.
The area specifying unit 73 specifies each area of the work position based on the image information acquired by the distance information acquiring unit 71 (step S105).

  The candidate detecting unit 74 detects each object in each region specified by the region specifying unit 73 as an object candidate based on the distance information of the object acquired by the distance information acquiring unit 71 (step S106).

  The object recognizing unit 75 selects each object candidate in each region detected by the candidate detecting unit 74 based on the distance information of the object acquired by the distance information acquiring unit 71 and the table information in the storage unit 72. Recognize (step S107). For example, the control device 5 controls the moving device 3 and the gripping operation device 4 of the robot 1 based on the object recognized by the object recognition unit 75 and the table information in the storage unit 72, and executes the work. In this case, along with the recognition of the object by the object recognition unit 75, the control device 5 uses the object information (shape information, etc.) of the object stored in the storage unit 72 in consideration of the constraints of the robot 1, It can be determined whether or not it can be held with a single arm and can be held with the current ability. Therefore, the working ability of the robot 1 can be efficiently extracted.

  As described above, in the object recognition apparatus 7 according to the present embodiment, the object category of each region set in the storage unit 72, the identification information of the object belonging to the object category, and the shape information of the object Table information in which object information necessary for performing work is associated with each other is stored. The area specifying unit 73 specifies an area where the object exists. The candidate detection unit 74 detects an object in the region as an object candidate based on the distance information of the object acquired by the distance information acquisition unit 71. The object recognition unit 75 compares the object candidate detected by the candidate detection unit 74 using the table information stored in the storage unit 72 with the shape information of the object belonging to the object category of the area specified by the area specifying unit 73. To recognize. Thereby, it can be estimated that the object candidate detected in each specified area belongs to the object category of the area. Therefore, it is possible to easily recognize object candidates detected by narrowing down within each area by simply collating with object information of the object category of each area. That is, object recognition can be performed at high speed.

Note that the present invention is not limited to the above-described embodiment, and can be changed as appropriate without departing from the spirit of the present invention.
Further, the present invention can be realized by causing the CPU 7a to execute a computer program, for example, the processing shown in FIG.
The program may be stored using various types of non-transitory computer readable media and supplied to a computer. Non-transitory computer readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (for example, flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (for example, magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / W and semiconductor memory (for example, mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (random access memory)) are included.
The program may also be supplied to the computer by various types of transitory computer readable media. Examples of transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.

  DESCRIPTION OF SYMBOLS 1 Robot, 2 Robot main body, 3 Moving apparatus, 4 Grip operation apparatus, 5 Control apparatus, 6 Distance sensor, 7 Object recognition apparatus, 8 Operation terminal

Claims (5)

A plurality of areas are set in advance in the work environment of the robot, and an object recognition device for recognizing an object in each area,
Storage means for storing the object category of each set region, identification information of an object belonging to the object category, and shape information of the object in association with each other;
Distance information acquisition means for acquiring distance information of an object to be recognized;
Area specifying means for specifying an area in which the object exists;
Candidate detection means for detecting an object in the region as an object candidate based on distance information of the object acquired by the distance information acquisition means;
Recognition by recognizing the object candidate detected by the candidate detection means by using the information stored in the storage means and comparing it with the shape information of the object belonging to the object category of the area specified by the area specifying means. Means,
An object recognition apparatus comprising: The object recognition device according to claim 1,
The recognizing unit compares the object candidate detected by the candidate detecting unit with the shape information of the object belonging to the object category of the region specified by the region specifying unit, and then the object not belonging to the object category of the region Recognize by comparing with the shape information of
An object recognition apparatus characterized by that. The object recognition device according to claim 1 or 2,
The storage means stores the set area and the position information of the robot at the time of acquiring the distance information in association with each other,
The distance information acquisition means acquires distance information of the recognition target object at a position corresponding to the region;
An object recognition apparatus characterized by that. The object recognition device according to any one of claims 1 to 3,
In the area within the work environment, mark information indicating the position of the area is set,
The area specifying means specifies the area based on the landmark information.
An object recognition apparatus characterized by that. A plurality of areas are set in advance in the work environment of the robot, and an object recognition method for recognizing an object in each area,
Storing the object category of each set region, identification information of an object belonging to the object category, and shape information of the object in association with each other;
Obtaining distance information of an object to be recognized;
Identifying a region where the object is present;
Detecting an object in the region as an object candidate based on the acquired distance information of the object;
Recognizing the detected object candidate by comparing it with shape information of an object belonging to the object category of the identified region using the stored information;
An object recognition method comprising:

Images