JPH1083455A - Object recognizing device and method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To increase a learning capacity of a neural network. SOLUTION: The color image data of an object which is fetched from a television camera is quantized in the space of hues and saturation. That is, when color image data which is defined by the hue and saturation of the color image data exists within the range of an area A1, the data is quantized as red data. Similarly, when color image data exists within the range of an area A2 or an area A3, it is defined as green or blue quantization data respectively. In this way, after data is quantized to a small number of quantization data, each quantization data is separately stored in a neural network.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus and method for recognizing an object, and more particularly, to an apparatus and method for recognizing an object to be recognized quickly and reliably.

[0002]

2. Description of the Related Art In a factory, a mobile robot is sometimes used when carrying articles. Usually, a guide line is buried in the ground or a floor in a building for moving the mobile robot, and the mobile robot is guided to a predetermined destination by tracing the guide line. In this way, the mobile robot can be moved to a predetermined destination without requiring a driver.

However, according to such a method, the mobile robot can always travel only on a fixed traveling route, and cannot reach the destination if there is an unintended obstacle on the way. was there.

Therefore, when an obstacle is present on the way, it is possible to temporarily move away from the guide line, avoid the obstacle, and then run the vehicle again along the guide line. It has been disclosed.

[0005] However, such a conventional method is as follows.
The guide line basically follows the guide line, so the guide line must be provided, and when the layout of the equipment in the factory is changed, the guide line must also be changed. Was not.

[0006] Therefore, a method has been proposed in which a mobile robot receives, for example, radio waves emitted from a destination and moves in a direction in which the radio waves are transmitted.

However, detecting the absolute position of the moving space in this way not only limits the range in which the robot can move, but also increases the size of the entire system and increases the cost.

Therefore, for example, landmarks (marks) are arranged at predetermined positions, and based on the landmarks,
It is conceivable to move the robot autonomously. With this configuration, the robot can capture landmarks with, for example, a television camera and recognize the landmarks. Therefore, it is possible to relatively easily move the robot along a free path. .

By the way, in order to autonomously move the robot with reference to the landmark, it is necessary to recognize the landmark. Conventionally, landmarks have been determined in advance to have a predetermined shape so that the landmarks can be easily recognized. Then, whether or not the landmark is a fixed shape is determined by processing the output of the television camera.

[0010]

However, in order to recognize whether or not the imaged shape is equal to a predetermined shape, correlation storage or the like is used.
There is a problem in that the learning capacity is limited when learning is performed using a pixel image. In addition, there is a problem that it is difficult to identify landmarks having similar shapes.

The present invention has been made in view of such a situation, and is capable of storing a larger number of reference objects (landmarks) to be recognized. It is to be able to recognize.

[0012]

An object recognizing device according to claim 1 is an image recognizing device for capturing an image of a reference object to be recognized and outputting corresponding color image data; Quantizing means for individually quantizing the color image data as data belonging to any one of a finite number of areas in a predetermined color space; and quantized data individually quantized by the quantizing means in advance. A storage means for individually learning and storing for each area, and quantized data for each area individually stored in advance in the storage means,
The comparing means for individually comparing the quantized data for each area output from the quantizing means for each area, and the exclusive means based on the comparison result individually compared for each area by the comparing means. Determining means for determining a specific reference object.

[0013] In the object recognition method according to the present invention, the color image data of the reference object obtained by imaging the reference object to be recognized is stored in a predetermined color space in any of a limited number of regions. Individually quantized as data belonging to, individually learned and quantized data previously individually learned and stored for each region, and quantized data for each region previously individually stored and captured. The quantized data of each region obtained by the above is individually compared for each region, and a specific reference object is exclusively determined based on the comparison result individually compared for each region. And

In the object recognition device according to the first aspect and the object recognition method according to the eighth aspect, the color image data of the object is quantized into quantization data in a hue and saturation space. The quantized data of the object to be recognized is individually stored in advance for each quantized data. The quantized data of the imaged object is individually compared with the previously stored quantized data for each corresponding quantized data.

[0015]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of the present invention will be described below. In order to clarify the correspondence between each means of the invention described in the claims and the following embodiments, each means is described. When the features of the present invention are described by adding the corresponding embodiment (however, an example) in parentheses after the parentheses, the result is as follows. However, of course, this description does not mean that each means is limited to those described.

The object recognizing device according to the first aspect of the present invention captures a reference object to be recognized and outputs corresponding color image data (for example, the television camera 12 in FIG. 2).
And quantizing means for individually quantizing the color image data of the reference object output by the imaging means as data belonging to any one of a finite number of areas in a predetermined color space in advance (for example, FIG. And a storage unit (e.g., the correlation storage net 41 in FIG. 5) that learns and stores the quantized data individually quantized by the quantization unit separately for each region. Comparison means for individually comparing the quantized data for each area previously stored individually in the storage means and the quantized data for each area output from the quantization means for each area (for example, FIG. 5 for determining a specific reference object exclusively based on the comparison result individually compared for each region by the comparing means (for example, the winner-take-all type neural network shown in FIG. 5). Net Characterized in that it comprises a workpiece 42) and.

FIG. 1 shows the external configuration of a mobile robot that autonomously moves along a route, to which the object recognition device of the present invention is applied. In this embodiment, the robot 11
A television camera 12 is attached to the upper part of the camera to capture images of the surroundings. Wheels 13 are attached to the lower side of the robot 11 so that the robot 13 can move to an arbitrary position. Also, on the side of the robot 11,
The display 14 is attached, and predetermined characters and images are displayed.

FIG. 2 shows an example of the internal configuration of the robot 11. The television camera 12 captures the surrounding video as a color image, and outputs the captured color image data to the control circuit 24 and the quantization circuit 25. The quantization circuit 25 quantizes the input color image data and outputs it to the neural network recognition device 23. The neural network recognition device 23 recognizes a landmark to be described later from the color image data input from the quantization circuit 25, and outputs a recognition result to the control circuit 24. For example, the control circuit 24 including a microcomputer notifies the neural network recognition device 23 of the moving direction of the robot,
CR of the prediction of the next landmark supplied from
The data is output to and displayed on a display 14 composed of a T, LCD, or the like.

The control circuit 24 drives the motor 21 to direct the television camera 12 in a predetermined direction. Further, the control circuit 24 drives the motor 22, rotates the wheels 13, and moves the robot 11 to a predetermined position.

FIG. 3 shows the moving space of the robot 11 in a plan view. In this embodiment, the obstacle 10
Landmarks 1 to 5 for the robot 11 to recognize the current position are arranged around A and the obstacle 10B. In the case of this embodiment, the robot 11
Are landmark 1, landmark 2, landmark 5
Move around the obstacle 10A in the counterclockwise direction along the path of the landmark 1, the landmark 2, the landmark 3, the landmark 4, the landmark 4, and the landmark 5 to move the obstacle 10A and the obstacle 10B. It is assumed that it moves around in the counterclockwise direction.

In this case, the control circuit 24 executes the processing shown in FIG. First, in step S1, the control circuit 24 determines whether a landmark has been found.
That is, the television camera 12 captures the surrounding image,
The color image data obtained as a result of imaging is quantized by the quantization circuit 2
5 to the neural network recognition device 23. When the landmark is recognized, the neural network recognition device 23 outputs the recognition result to the control circuit 24 as described later. The control circuit 24 monitors the output of the neural network recognizing device 23 to determine whether or not a landmark has been found. If the landmark has not been found yet, the process proceeds to step S2 and the motor 2
1 to rotate the television camera 12 in a predetermined direction or to drive the motor 22 so that the robot 1 can move the wheels around the obstacles 10A and 10B in FIG. 13 is rotated. This step S1,
The process of S2 is repeatedly executed until it is determined in step S1 that a landmark has been found.

If it is determined in step S1 that a landmark has been found, the process proceeds to step S3, where the control circuit 24 determines whether an obstacle exists in the direction of the found landmark. That is, the control circuit 24
The presence or absence of an obstacle is determined from the output of the television camera 12, and if it is determined that an obstacle exists, the process proceeds to step S
Then, the process proceeds to step 4 in which the wheel 13 is rotated rightward. At this time, the control circuit 24 drives the motor 22 to rotate the wheels 13 clockwise (clockwise).

Thereafter, the flow returns to step S3, where the robot 1
It is determined again whether or not an obstacle exists in the direction in which 1 has newly pointed. If it is determined that an obstacle exists, the process proceeds to step S4 again, and a process of rotating the robot 11 further clockwise is performed. If it is determined in step S3 that there is no obstacle, the process proceeds to step S5, and the control circuit 24 executes a process of moving the robot 11 in the direction of the landmark found in step S1. That is, at this time, the control circuit 24 drives the motor 22, rotates the wheel 13, and moves the robot 11 in the direction of the landmark.

Then, the process proceeds to a step S6, wherein the control circuit 24
Indicates whether or not a landmark has been reached
2 and the output of the neural network recognition device 23. That is, when a signal indicating that a landmark has been detected is input from the neural network recognition device 23 and the TV camera 12 is outputting a sufficiently large landmark image, the control circuit 24 Is determined to have been reached.
If the landmark has not been reached yet, the process returns to step S3, and the subsequent processing is repeatedly executed.
If it is determined that the landmark has been reached, step S
Returning to step 1, a new landmark is found, and the same processing as described above is executed toward the new landmark.

As described above, the robot 11 travels toward the landmark 1 so as not to collide with the obstacle 10A, and travels from the landmark 1 toward the landmark 2 when it reaches the landmark 1. . When the vehicle reaches the landmark 2, the vehicle travels further toward the landmark 5. When the vehicle reaches the landmark 5, the vehicle travels toward the landmark 1 from there.

Alternatively, when the robot 11 reaches the landmark 2, the robot 11
After moving in the direction of the landmark 3 and reaching the landmark 3, proceed to the landmark 4 from there, and proceed to the landmark 5 when reaching the landmark 4.

Which of the two routes the robot 11 travels can be programmed in advance by the control circuit 24.

Here, the configuration of the neural network recognition device 23 will be described. As shown in FIG. 5, the neural network recognizing device 23 includes an associative memory net 41 composed of a Hopfield type neural network, and a winner-take-all type neural network 42. , And a recurrent type neural network 43.

Incidentally, a phase connection type (Interconnection type)
The Hopfield-type neural network, which is a type of neural network, has been proposed for applications to optimization problems typified by the salesman traveling problem, or to associative memories that store and recall complete patterns from incomplete input patterns. ing.

For details of the application of the recurrent neural network to the learning surface of a mobile robot, see the document “Structure of Cognition and Autonomy in Robots: From the Perspective of Dynamic Forms”, Journal of the Robotics Society of Japan, Vol. .4.pp.4
74-477, 1996, and J. Tani: "Model-Based Learning fo
r Mobile Robot Navigation from the Dynamical Syste
ms Perspective. "IEEE Trans.System, Man, and Cybernet
ics Part B: Cybernetics, Vol. 26. No. 3, June 1996.

The color image data output from the television camera 12 is input to the quantization circuit 25 and quantized before being input to the correlation storage network 41. That is, as shown in FIG. 6, the quantization circuit 25 determines the hue (color type), saturation (color density from achromatic color to pure color), and hue and color among lightness, which are three elements of color. It has a space (table) defined by degrees, and among the color image data defined by the predetermined hue and saturation, all the color image data belonging to the range of the area A1 are quantized as, for example, red data. . Similarly, all the color image data belonging to the area A2 is quantized as green data, and all the color image data belonging to the area A3 is quantized as blue data. That is, the color image data is individually quantized as data belonging to any of the finite number of areas A1, A2, and A3.

It should be noted that the names of red, green and blue here are merely for convenience, and other names may be used. That is, these names are merely codes (names of quantized data) of each area.

Although the color image data existing in the space defined by the hue and the saturation exists infinitely,
In the case of this embodiment, this is quantized into three pieces of quantized data. In this way, by quantizing a large number of color image data into a sufficiently small number of quantized data,
Learning and recognition of objects by neural networks are possible. In this way, the quantized data obtained by quantizing the color image data by the quantization circuit 25 is supplied to the neural network recognition device 23. Therefore, the quantized data input to the neural network recognition device 23 is data represented by any one of the three data defined by the space shown in FIG.

As shown in FIG. 5, the correlation storage net 41
Has a number of fields corresponding to the number of quantization steps (three in this embodiment) shown in FIG.
The field 41R is a field corresponding to the red quantized data in the area A1 in FIG.
G is a field corresponding to the green quantized data in the area A2 in FIG. 6, and the field 41B is a field corresponding to the blue quantized data in the area A3 in FIG. The input pattern composed of the three pieces of quantized data output from the quantization circuit 25 is stored in the correlation storage net 41.
Are input (recalled) to the neurons in the corresponding fields.

That is, the internal state of each neuron is represented by U
Then, the following equation is established.

[0036]

(Equation 1)

Here, i represents the number of the neuron, and t
Represents a predetermined time. Thus, U _i ^{t + 1} represents the internal state of the ith neuron at time t + 1.

Here, k is a constant representing a damper,
α is also a predetermined constant.

W _ij represents a connection weight coefficient from the i-th neuron to the j-th neuron. a
_j ^t represents the output of the neuron at the j-th time t. This output is defined by the following equation.

[0040]

(Equation 2)

Here, logistic (A) indicates that A is multiplied by a sigmoid function. T represents a constant. That is, the above equation means that the result of multiplying the result of dividing the internal state of the neuron by the constant T by the sigmoid function is the output of the neuron.

As described above, the three quantized data are individually compared according to the firing state of each neuron of the correlation storage net 41, and the recall dynamics are performed. On the other hand, the learning dynamics are represented by the following equations. expressed.

[0043]

(Equation 3)

0.5 in the above equation functions as a threshold value. That is, the output of each neuron is an activity value between 0 and 1, but when the activity value is less than 0.5,
It has a function to make the connection weight coefficient negative and to make the connection weight coefficient positive when it is larger than 0.5.

Landmark 1 in neural network
When the landmark 5 is learned as a reference object serving as a reference for recognition, the result of the learning is represented by
_ij .

The winner-take-all type neural network 42 has at least as many neurons (5 in this embodiment as the number of landmarks to be recognized).
When each activity value is input from each field 41R, 41G, 41B of the correlation storage net 41, one of the five neurons that outputs the largest activity value is output. Learning is performed in which the output of the neuron is set to 1.0 and the outputs of the other four neurons are set to 0.0. As a result, one landmark is determined based on the output patterns that are individually compared in the three fields 41R, 41G, and 41B of the correlation storage net 41, and the only neuron corresponding to the landmark is fired. Will be.

As described above, in the winner-take-all type neural network 42, by taking all the winners,
Since only one neuron is fired exclusively, the subsequent recurrent neural network 4 that processes its output
3 can be simplified.

Recurrent neural network 43
Is basically composed of an input layer 51, an intermediate layer 52, and an output layer 53. The input layer 51 includes a pattern node 51A having five neurons corresponding to the winner-take-all type neural network 42 and a context node (context) having neurons holding the internal state of the recurrent type neural network 43.
node) 51B, and a direction node 51C having a neuron to which a direction to move next is instructed by the control circuit 24.

The output layer 53 has a pattern node 53A having neurons corresponding to five landmarks, and a context node 53B corresponding to the context node 51B in the input layer 51. Each neuron of the intermediate layer 52 connects each node of the input layer 51 and each node of the output layer 53. Also, the context node 5 of the output layer 53
The output of the 3B neuron is fed back to the context node 51B neuron of the input layer 51. This allows the context loop (co
ntext loop) is formed.

Recurrent neural network 43
Is the winner-take-all type neural network 4
From 2, when an input corresponding to one landmark is made to the pattern node 51 A of the input layer 51, the next appearing landmark is predicted and output from the output layer 53.

The neural network recognition device 23
When color image data is input from the quantization circuit 25,
The processing shown in the flowchart of FIG. 7 is executed.

First, in step S11, the process waits until a landmark is searched. In the case of this embodiment, the landmarks 1 to 5 are all colored in a predetermined color, and the neural network recognizing device 23 determines YES in step S11 when color image data is input. The process proceeds to step S12.

In step S12, the neural network recognizing device 23 executes a process of recognizing the landmark (current landmark) just searched. This recognition processing is executed in the correlation storage net 41 of the neural network recognition device 23.

When the current landmark recognition process in step S12 is completed, the process proceeds to step S13, where the process of narrowing down the recognition result obtained in step S12 is performed in the winner-take-all type neural network 42. That is, it is clear which of the five landmarks has been recognized. Then, step S1
Then, the process proceeds to step 4 in which the recurrent neural network 43 predicts a landmark appearing next to the current landmark. The predicted result is displayed on display 1
4 is displayed. The above processing is repeatedly executed each time a landmark is searched.

Now, landmarks 1 to 5
Is stored (learned) as a connection weight coefficient in the correlation storage net 41. In this state, for example, the correlation storage net 41
Then, when the pattern of the landmark 2 captured by the television camera 12 and quantized by the quantization circuit 25 is input,
In the fields 41R, 41G, and 41B, quantized red data, green data, and blue data of the landmark 2 fire at predetermined positions, respectively. The winner-take-all type neural network 42
A corresponding landmark is determined from the firing state of each field, and a neuron corresponding to one landmark is fired based on the determination result. In this case, the neuron corresponding to the landmark 2 fires.

Therefore, a neuron corresponding to the landmark 2 is fired at the pattern node 51A of the input layer 51 of the recurrent neural network 43, corresponding to the neuron of the winner-take-all neural network 42. At this time, the control circuit 24 determines whether the next direction to go is left or right,
A signal corresponding to the direction is sent to a direction node 51 of the input layer 51.
Input to C. In the embodiment of FIG. 5, the neuron corresponding to the left direction is fired. Therefore, the recurrent neural network 43 predicts a landmark arriving next to the landmark 2 and outputs the prediction result to the pattern node 53A of the output layer 53. FIG.
As shown in the figure, when the landmark 2 is detected and the next moving direction is the left direction,
The next landmark to appear is landmark 5. Therefore, in this case, as shown in FIG.
The neuron with the number 5 corresponding to the landmark 5 fires.

When receiving the data for predicting the next landmark from the neural network recognition device 23, the control circuit 24 outputs the number corresponding to the input to the display 14 for display. In this case, for example, the number 5 is displayed on the display 14. Thereby, the user can know that the landmark that appears next is the landmark 5.

Recurrent neural network 43
In the state where the neuron corresponding to the landmark 2 in the pattern node 51A of the input layer 51 of the
In the case where the neuron corresponding to the right direction is fired at the direction node 51C, as shown in FIG. At the pattern node 53A of the layer 53, the neuron of number 3 corresponding to the landmark 3 fires.

For example, if the landmark 4 is a landmark having a color similar to that of the landmark 1, it is unclear which of the landmark 1 and the landmark 4 has been recognized. However, in the case of this embodiment, since the context loop is provided in the recurrent neural network 43, the history is internally reflected, whereby the time-series appearance order of the landmarks 1 to 4, that is, State transitions are also identified.

As shown in FIG. 3, the landmark 4 appears next to the landmark 3, and the landmark 1 appears next to the landmark 5. In the recurrent neural network 43, the context nodes 51B and 53B make it possible to identify the current state, and if the landmark recognized immediately before is the landmark 3, the next input is performed. The recognized landmark is recognized as landmark 4 instead of landmark 1. Similarly, when the landmark recognized immediately before is the landmark 5, it can be recognized that the landmark predicted next is not the landmark 4 but the landmark 1.

The embodiment of the present invention has been described above by taking as an example a case where a mobile robot that autonomously moves along a route recognizes a landmark for recognizing a current position. It is also possible to adapt when recognizing a predetermined object.

[0062]

As described above, according to the object recognizing device of the first aspect and the object recognizing method of the eighth aspect, the stored quantized data and the color image data of the captured reference object are compared. Quantized data is individually compared in advance for each finite number of areas in a predetermined color space, so the learning capacity is large, and many reference objects can be stored and recognized. Becomes

[Brief description of the drawings]

FIG. 1 is a diagram showing an external configuration of a robot to which an object recognition device of the present invention is applied.

FIG. 2 is a block diagram showing an internal configuration of the embodiment of FIG.

FIG. 3 is a diagram illustrating a moving space according to the embodiment of FIG. 1;

FIG. 4 is a flowchart illustrating an operation of the control circuit of FIG. 2;

5 is a diagram showing a detailed configuration example of the neural network recognition device 23 of FIG.

FIG. 6 is a diagram for explaining the operation of the quantization circuit 25 of FIG. 2;

7 is a flowchart illustrating the operation of the neural network recognition device 23 of FIG.

[Explanation of symbols]

Reference Signs List 11 robot, 12 TV camera, 13 wheels, 14 display, 23 neural network recognition device, 24 control circuit, 25 quantization circuit, 41 correlation storage net, 42 winner-take all type neural network, 43 recurrent type neural network

Claims (9)

[Claims] 1. An image pickup means for picking up an image of a reference object to be recognized and outputting corresponding color image data, and converting the color image data of the reference object output by the image pickup means into a predetermined color space on a predetermined color space in advance. Quantizing means for individually quantizing as data belonging to any of the divided regions; and Quantized data individually quantized by the quantizing means in advance and individually learned for each of the regions. A storage unit for storing, the quantized data for each of the regions, which are individually stored in advance in the storage unit, and the quantized data for each of the regions output from the quantizing unit, for each of the regions. And a determination unit for exclusively determining a specific reference object based on a comparison result individually compared for each of the regions by the comparison unit. Body recognition device. 2. The predetermined color space according to claim 1, wherein the predetermined color space is a space defined by hue and saturation among hue, saturation, and brightness, which are three elements of color. Object recognition device. 3. The storage means stores quantized data corresponding to a reference object to be recognized as a connection weighting factor between neurons of a neural network for each of the regions. For each region, quantized data is compared according to the firing state of each neuron of the neural network, and the storage means and the comparing means constitute a correlation storage net for recognizing a reference object which is currently being imaged. The object recognition device according to claim 1, wherein: 4. The object recognition apparatus according to claim 3, wherein the correlation net is constituted by a Hopfield type neural network. 5. The object recognition apparatus according to claim 1, wherein said determination means is constituted by a winner-take-all type neural network for narrowing down a comparison result of said comparison means. 6. An internal history for learning a time-series appearance order of a reference object determined by said determination means and for predicting a next reference object to appear based on a determination result of said determination means. 2. The object recognition apparatus according to claim 1, further comprising a recurrent neural network having a context loop that reflects the information. 7. The object recognition device according to claim 1, wherein the reference object is used as a landmark for a mobile robot autonomously moving on a route to recognize a current position. 8. A method of individually quantizing color image data of a reference object obtained by imaging a reference object to be recognized as data belonging to any one of a finite number of regions in a predetermined color space. , Individually learning and storing the quantized data individually quantized in advance for each of the regions, and the quantized data for each of the regions previously stored individually, and the respective regions obtained by imaging. Quantized data for each of the regions is compared individually for each region, and based on a comparison result individually compared for each region,
An object recognition method characterized by exclusively determining a specific reference object. 9. A chronological order of appearance of a reference object determined exclusively based on a comparison result individually compared for each of the regions, and a next appearing order is determined based on the determination result. The method according to claim 8, wherein a reference object is predicted.