JP2009074995A - Mobile unit information processor, mobile unit information processing method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To generate landmark data for specifying the position of a mobile unit from a plurality of color images photographed within a moving range without arranging landmarks within the travel environment of the mobile unit in advance, and to estimate the position of the mobile unit from a photographed image for position estimation and the landmark data. <P>SOLUTION: The mobile unit information processor 1 comprises: a photographing section 2 for photographing a subject within a moving environment to acquire a color image; a storage section 4 for storing a plurality of photographed images of an object to be learned and positional information on the photographing position of the photographed image with them associated with each other; and a control section 8 which learns a self-organizing map on the basis of the color information of each pixel of the plurality of photographed images, provides a plurality of clusters and the color range of the clusters for classifying color information by the self-organizing map after learning, specifies a cluster into which the color information of the photographed image is classified for each photographed image, and generates landmark data with the specified cluster, the photographed image, and the positional information associated with each other. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a mobile information processing apparatus, a mobile information processing method, and a program.

  For example, as a method for a mobile robot to identify its own position, a landmark is set in advance at a specified position in the traveling environment of the moving body, and the self-position is determined by a captured image obtained by imaging the landmark. An estimation method is known.

Specifically, for example, a plurality of landmarks are arranged in advance in the traveling environment of the moving body, and the moving state amount when the moving body is moved by an arbitrary distance from the initial position, and the moving body are imaged. In addition, a landmark registration method is known in which the three-dimensional position coordinates of a landmark are calculated from the shift amounts of a plurality of landmark images (see, for example, Patent Document 1).
JP 2006-234453 A

However, in the technique described in Patent Document 1, the three-dimensional position coordinates of the arranged landmarks are automatically registered, and the self-position is estimated based on the registered landmark positions. It is necessary to place the mark at a specified position.
In addition, since the landmark is arranged in the traveling environment of the moving body, the beauty in the traveling environment is impaired.

  This invention makes it an example of a subject to cope with such a problem. That is, generating landmark data for specifying the position of a moving object from a plurality of color images photographed within the moving range without previously arranging the landmark in the traveling environment of the moving object, for position estimation It is an object of the present invention to estimate the position of the moving object from the captured image and landmark data.

In order to achieve such an object, the present invention comprises at least the configurations according to the following independent claims.
A mobile information processing apparatus according to the present invention is a mobile information processing apparatus that generates landmark data for performing position estimation using color images captured in a mobile environment, and includes a plurality of captured images to be learned, And a storage unit that associates and stores positional information related to the imaging position of the captured image, and learns a self-organizing map based on the color information of each pixel of the plurality of captured images, and the self-organizing map after learning A plurality of clusters for classifying the color information and a color range of the clusters, specifying the cluster in which the color information of the captured image is classified for each captured image, the identified cluster, And a control unit that generates the landmark data in which the captured image and the position information are associated with each other.

  Preferably, the control unit identifies the cluster into which color information of each pixel of the captured image for position estimation is classified based on the cluster and the color range, and identifies the identified cluster and the land. According to the mark data, the similarity of the captured image to be learned with respect to the captured image for position estimation is calculated, and the position estimation is performed based on the similarity and the position information associated with the captured image to be learned. It is characterized by performing the process regarding.

  Also, the information processing method of the mobile information processing apparatus according to the present invention stores a plurality of captured images that have been subjected to color imaging in a moving environment and positional information related to the imaging position of the captured images in association with each other, and moves An information processing method for a mobile information processing apparatus that generates landmark data for estimating a position from a color image captured in an environment, the self-organization based on color information of each pixel of the plurality of captured images A first step of defining a plurality of clusters for classifying the color information and a color range of the clusters based on a self-organizing map after learning, and the imaging for each captured image The cluster to which the color information of the image is classified is specified, and the landmark data that associates the specified cluster, the captured image, and the position information is generated. And having a second step.

  Preferably, based on the cluster and the color range, the cluster into which color information of each pixel of the captured image for position estimation is classified is identified, and according to the identified cluster and the landmark data Calculating a similarity of the captured image to be learned with respect to the captured image for position estimation, and performing processing related to the position estimation based on the similarity and position information associated with the captured image to be learned It has 3 steps.

  In addition, the program according to the present invention stores a plurality of captured images that have been color-captured in a moving environment and positional information related to the imaging position of the captured images in association with each other, and the color images that are captured in the moving environment A program for causing a mobile information processing apparatus that generates landmark data to perform position estimation by performing learning of a self-organizing map based on color information of each pixel of the plurality of captured images. A first step of defining a plurality of clusters for classifying the color information and a color range of the clusters by the self-organizing map after learning, and the color information of the captured image is classified for each captured image A second cluster that generates the landmark data that associates the identified cluster, the captured image, and the position information. And-up, characterized in that to be executed by the mobile information processing apparatus.

  Preferably, based on the cluster and the color range, the cluster into which color information of each pixel of the captured image for position estimation is classified is identified, and according to the identified cluster and the landmark data Calculating a similarity of the captured image to be learned with respect to the captured image for position estimation, and performing processing related to the position estimation based on the similarity and position information associated with the captured image to be learned It has 3 steps.

  According to the present invention, landmark data for specifying the position of a moving body is generated from a plurality of color images photographed within the moving range without arranging the landmark in advance in the traveling environment of the moving body. Can do. Further, according to the present invention, it is possible to estimate the position of the moving object from the captured image for position estimation and the landmark data.

  A mobile information processing apparatus according to the present invention is a mobile information processing apparatus that generates landmark data for performing position estimation using color images captured in a mobile environment, and includes a plurality of captured images to be learned, And a storage unit that associates and stores position information related to the imaging position of the captured image, and learns a self-organizing map based on the color information of each pixel of the plurality of captured images, and by the self-organizing map after learning, A plurality of clusters for classifying color information and a color range of the clusters are specified, a cluster in which the color information of the captured image is classified for each captured image, the identified cluster, captured image, and position And a control unit that generates landmark data associated with the information.

  According to the mobile information processing apparatus configured as described above, the control unit learns the self-organizing map based on the color information of each pixel of the plurality of captured images stored in the storage unit, and self-organizes after learning. A map defines a plurality of clusters for classifying color information and a color range of the clusters, identifies a cluster in which the color information of the captured image is classified for each captured image, and identifies the identified cluster and captured image In order to identify the position of the moving body from a plurality of images taken within the moving range without previously arranging the landmark in the traveling environment of the moving body. Landmark data can be generated.

  Preferably, the control unit identifies a cluster in which color information of each pixel of the captured image for position estimation is classified based on the cluster and the color range, and corresponds to the identified cluster and landmark data. Then, the similarity of the captured image to be learned with respect to the captured image for position estimation is calculated, and processing related to position estimation is performed based on the similarity and position information associated with the captured image to be learned. And

  According to the mobile information processing apparatus having the above configuration, the control unit identifies a cluster in which color information of each pixel of the captured image for position estimation is classified based on the cluster and the color range, and the identified cluster In addition, according to the landmark data, the similarity of the captured image to be learned with respect to the captured image for position estimation is calculated, and position estimation is performed based on the similarity and the position information associated with the captured image to be learned. Since the process is performed, the position of the moving object can be estimated from the captured image for position estimation and the landmark data.

  A mobile information processing apparatus, information processing method, and program according to the present invention will be described below with reference to the drawings.

[First Embodiment]
FIG. 1 is an overall functional block diagram of a mobile information processing apparatus according to the first embodiment of the present invention. FIG. 2 is a diagram for explaining a moving environment of the mobile information processing apparatus shown in FIG.

In the present embodiment, a mobile robot (also referred to as a robot) 100 that employs the mobile information processing apparatus 1 according to the present invention will be described.
As shown in FIG. 1, the mobile information processing apparatus 1 according to the present embodiment includes an imaging unit 2. For example, as shown in FIG. 2, the mobile information processing apparatus 1 moves within the mobile environment 200 and moves at a predetermined timing. A plurality of color images P are acquired by performing color imaging of the subject inside. The mobile information processing apparatus 1 stores the captured image P in the storage unit 4 in association with position information regarding the imaging position.
In addition, the mobile information processing apparatus 1 generates landmark data for estimating the position of the robot 100 in the mobile environment 200 based on the color image P captured in the mobile environment 200.
Specifically, the mobile information processing apparatus 1 according to the present embodiment captures a subject in the moving environment 200 to acquire a plurality of color images, and based on the color information of each pixel of the plurality of color images, The color distribution is classified into a plurality of clusters, the color range of each cluster is defined, the cluster into which the color information of the captured image is classified for each captured image, the cluster, the captured image, and the position information The landmark data associated with is generated.

  In addition, the mobile information processing apparatus 1 identifies a cluster in which color information of each pixel of the captured image for position estimation is classified based on the cluster and the color range, and uses the identified cluster and landmark data. Accordingly, the similarity of the captured image to be learned with respect to the captured image for position estimation is calculated, and processing related to position estimation is performed based on the similarity and the position information associated with the captured image to be learned.

  For example, in a general apparatus, it is necessary to arrange landmarks at predetermined positions in advance. On the other hand, the mobile information processing apparatus 1 according to the present invention can generate landmark data in which each imaging position is associated with a color distribution feature without arranging landmarks in advance.

Further, in a general apparatus, when classifying color information of a captured image, it is necessary to set a color range for classification in advance.
On the other hand, the mobile information processing apparatus 1 according to the present invention uses a self-organizing map of a neural network to perform clustering by unsupervised learning, so that a captured image obtained by performing color imaging on a subject in a moving environment. According to the color information of each pixel, a cluster for classifying the color information and a color range of the cluster can be automatically defined. Therefore, the color information of each pixel of the captured image can be easily classified.

In addition, landmark data is generated based on the color information of a plurality of captured images, and the position of the moving object is estimated based on the landmark data and the captured image for position estimation. Compared to the case of generating landmark data, position estimation can be performed with relatively high accuracy and relatively high speed.
Hereinafter, each component of the mobile information processing apparatus 1 will be described in detail.

[Configuration of Mobile Information Processing Apparatus 1]
As shown in FIG. 1, the mobile information processing apparatus 1 according to the present embodiment includes an imaging unit 2, a quantization unit 3, a storage unit 4, a drive unit 5, wheels (moving means) 6, a sensor 7, and a control unit. 8 has.
The imaging unit 2 corresponds to one embodiment of the imaging unit according to the present invention, the storage unit 4 corresponds to one embodiment of the storage unit according to the present invention, and the driving unit 5 and the wheel 6 are the driving unit according to the present invention. This corresponds to one embodiment. The control unit 8 corresponds to an embodiment of the control unit according to the present invention.

  For example, as shown in FIGS. 1 and 2, the mobile body (mobile robot 100) according to the present embodiment includes a casing 10 having a predetermined shape such as a substantially cylindrical shape or a substantially rectangular parallelepiped shape.

  For example, under the control of the control unit 8, the imaging unit 2 performs color imaging on a subject in a moving environment around the robot 100 (moving body) and outputs a color image. In addition, the imaging unit 2 outputs a color image in which each pixel is configured by color information defined by RGB values of R (Red), G (Green), and B (Blue), for example. As the imaging unit 2, for example, an imaging device including an imaging element such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) image sensor can be employed. For example, as shown in FIG. 2, the imaging unit 2 is provided at a predetermined position of the housing 10, for example, at an upper portion or a front surface portion of the housing 10. The imaging unit 2 has a prescribed viewing angle with reference to the traveling direction of the robot 100. For example, the viewing angle is about 30 to 180 degrees. The imaging unit 2 may be fixed to the housing 10 or may be provided in the housing 10 so that the imaging direction is variable.

The quantization unit 3 quantizes the color information of each pixel of the color image to a specified gradation, and outputs a color image including the quantized color information. The image output from the quantization unit 3 may be stored, for example, in the storage unit 4 or may be input to the control unit 8.
For example, when the RGB values of the pixels of the captured image are set to 256 gradations, the quantization unit 3 according to the present embodiment converts the RGB values into relatively small gradations, for example, 8 gradation RGB values. . By converting the color information into relatively small gradations as described above, it is possible to reduce the small difference in the color of the subject and output the color information having a relatively small amount of information while reflecting the color of the subject. . Further, the quantization unit 3 may perform a quantization process on the color image output from the imaging unit 2 or may perform the quantization process on an image stored in the control unit 8.

The storage unit 4 stores, for example, a plurality of captured images P to be learned and positional information regarding the imaging positions of the captured images P in association with each other. In addition, the storage unit 4 functions as a buffer for images output from the imaging unit of the imaging unit 2. The storage unit 4 stores a program (PRG) 41 having a function according to the present invention.
As the storage unit 4, for example, a storage device such as a semiconductor memory such as a random access memory (RAM) or a read only memory (ROM), a magnetic disk storage device such as a hard disk drive, or the like may be employed. The storage unit 4 also functions as a work area for the control unit 8.

  The drive unit 5 drives, for example, a moving unit provided in the moving body (robot 100) under the control of the control unit 8 to move the robot 100 in the moving environment. As the moving means, for example, a prescribed moving device such as a wheel 6 or a movable leg provided at the lower part or the side part of the robot 100 can be adopted.

  The sensor 7 detects, for example, the driving state of the driving unit 5, the moving distance or moving speed of the moving unit, acquires position information regarding the imaging position, and outputs a signal indicating the acquired information to the control unit 8.

  The control part 8 controls each component of this apparatus centrally, and implement | achieves the function based on this invention. Moreover, the control part 8 implement | achieves the function which concerns on this invention by running the program (PRG) 41 memorize | stored in the memory | storage part 4, for example. As the control unit 8, for example, an electronic circuit such as a CPU (Central Processing Unit) can be employed.

FIG. 3 is a functional block diagram for explaining functions of the control unit of the mobile information processing apparatus shown in FIG.
As illustrated in FIG. 3, the control unit 8 includes, for example, a first processing unit 81 and a second processing unit 82.

[First processing unit 81]
The first processing unit 81 learns a self-organizing map based on the color information of each pixel of the plurality of captured images P, and a plurality of clusters for classifying the color information based on the self-organizing map after learning. And defining a color range of the cluster, specifying a cluster in which the color information of the captured image is classified for each captured image, and generating landmark data in which the identified cluster, captured image, and position information are associated with each other .

  Specifically, as shown in FIG. 3, for example, the first processing unit 81 includes a drive control unit 811, a first clustering processing unit (first SOM) 812, a maximum / minimum value extracting unit 813, a second clustering processing unit ( 2nd SOM) 814, cluster list generation unit 815, and landmark data generation unit 816.

  The drive control unit 811 performs various controls such as drive control related to imaging of the imaging unit 2 and drive control of the drive unit 5.

  The first clustering processing unit (first SOM) 812 learns the self-organizing map based on the color information of each pixel of the captured image, and a plurality of classes for classifying the color information based on the self-organizing map after learning. Define the cluster and the color range of the cluster.

  The maximum / minimum value extraction unit 813 extracts the color range (maximum value and minimum value) of each cluster, and outputs information on the cluster and the color range.

  The second clustering processing unit (second SOM) 814 performs self-organization map learning based on the clusters and color ranges defined for each of the plurality of captured images P, and a plurality of self-organization maps after learning A plurality of clusters for classifying the color information of the captured images and the color range of the clusters are defined.

  The cluster list generation unit 815 specifies a cluster into which the color information of the captured image is classified for each captured image. In addition, the cluster list generation unit 815 preferentially excludes a plurality of clusters in descending order of the number of captured images associated with each cluster. As described above, the cluster list generation unit 815 removes clusters in which colors that are commonly included in a plurality of captured images are classified, so that the characteristics of the color distribution (color pattern) included in each image can be obtained. A cluster list (CL) can be generated.

  The landmark data generation unit 816 generates landmark data (RMD) that associates the identified cluster, the captured image, and the position information based on the cluster list and the captured image P described above.

[Second processing unit]
The second processing unit 82 identifies a cluster into which color information of each pixel of the captured image for position estimation is classified based on the cluster and the color range, and according to the identified cluster and landmark data, The degree of similarity of the captured image to be learned with respect to the captured image for position estimation is calculated, and processing related to position estimation is performed based on the similarity and the position information associated with the captured image to be learned.

  Based on the cluster and the color range, a cluster in which color information of each pixel of the captured image for position estimation is classified is identified, and similar to the captured image of the position estimation according to the cluster and landmark data A captured image with a high degree of learning is specified, and processing related to position estimation is performed based on position information associated with the specified captured image.

  Specifically, for example, as illustrated in FIG. 3, the second processing unit 82 includes a cluster acquisition unit 821, a matching unit 822, a position estimation unit 823, and a drive control unit 824.

  The cluster acquisition unit 821 specifies and acquires a cluster into which the color information of each pixel of the captured image for position estimation is classified based on the cluster and the color range. Specifically, the cluster acquisition unit 821 specifies a cluster into which color information of each pixel of the captured image is classified based on, for example, the cluster and color range based on the self-organizing map after learning.

  The matching unit 822 calculates the similarity of the captured image to be learned with respect to the captured image for position estimation according to the identified cluster and landmark data.

  The position estimation unit 823 performs processing related to position estimation based on the similarity and the position information associated with the captured image to be learned.

  The drive control unit 824 performs various controls such as drive control related to imaging of the imaging unit 2 and drive control of the drive unit 5 according to the result of the position estimation process by the position estimation unit 823.

[Operation during learning of mobile information processing apparatus 1]
Next, the operation of the mobile information processing apparatus 1 configured as described above will be described.
FIG. 4 is a flowchart for explaining the operation of the mobile information processing apparatus shown in FIG. The operation at the time of learning of the mobile information processing apparatus 1 will be described with reference to FIG. In detail, the process which produces | generates the landmark data of the mobile body information processing apparatus 1 is demonstrated.

  In step ST1, the mobile information processing apparatus 1 captures a subject in the moving environment 200 and acquires a plurality of color images P as shown in FIG. In the present embodiment, as shown in FIG. 2, for example, in a room of the mobile environment 200, brown wooden boxes and gray shelves are arranged at two corners of the four corners. A foliage plant with red-brown pots and green leaves is placed near the wooden boxes and shelves. A gray desk is arranged on the wall facing the wall on which the foliage plants are arranged. In addition, three chairs having blue cushions and gray legs are arranged in the vicinity of the wooden box, in the vicinity of the shelf, and in the corner opposite to the corner in which the shelf is disposed. The wall color is substantially white and the floor color is indigo.

  For example, as shown in FIG. 2, the mobile information processing apparatus 1 captures an image while making a round in the mobile environment 200. Specifically, the mobile information processing apparatus 1 moves 90 degrees to the right after moving straight from the vicinity of the desk toward the chair arranged at the corner, and then moves 90 degrees to the right after moving straight to the vicinity of the wooden box. Rotate, go straight to the shelf, turn right, go straight to the vicinity of the corner, turn 90 degrees to the right, go straight in that state and return to the vicinity of the desk. As shown in FIG. 2, the mobile information processing apparatus 1 captures images at imaging positions 1 to 22 at a predetermined timing.

Specifically, the control unit 8 outputs a drive control signal to the drive unit 5 to rotate the wheel 6 to move in the moving environment 200 as shown in FIG. 2, for example. Further, the control unit 8 captures a subject within a viewing angle with reference to the traveling direction in the moving environment 200 at a predetermined timing by the imaging unit 2 and acquires a plurality of color images P. The control unit 8 stores the captured image P in the storage unit 4 in association with the position information regarding the imaging position.
Specifically, as shown in FIG. 2, for example, the control unit 8 determines the imaging position in the moving environment based on the movement distance, movement trajectory, direction of travel relative to the reference direction, and the like. Position information relating to the imaging position is stored in the storage unit 4 in association with the captured image.

At this time, the control unit 8 calculates, for example, the moving distance and the moving direction based on the detection results of the sensor 7 that detects the rotation speed and the rotation speed of the left and right drive motors of the driving unit 5 and performs imaging based on the calculation results. You may acquire a position.
Further, the control unit 8 calculates the moving distance and the moving direction based on the detection result of the sensor 7 that detects the rotation speed and the rotation speed of the left and right wheels 6, for example, and acquires the imaging position based on the calculation result. Also good.
In addition, the imaging unit 2 provided in the robot 100 acquires a captured image P having a constant viewing angle along the traveling direction.

  In step ST2, the mobile information processing apparatus 1 performs clustering processing (SOM) for one image. Specifically, the control unit 8 learns the self-organizing map based on the color information of each pixel of the captured image, and includes a plurality of clusters for classifying the color information based on the self-organizing map after learning. Specifies the color range of the cluster.

  In step ST3, the mobile information processing apparatus 1 performs a process of extracting the color range of each cluster, specifically the maximum value and the minimum value of the cluster color. Specifically, the control unit 8 automatically generates a cluster for classifying the color information of each image in the competitive layer based on the self-organization map of the learning result, according to each color information. Specifically, the maximum value and the minimum value in the RGB space of each cluster are extracted.

  In step ST4, the control unit 8 learns the self-organizing map based on the clusters and color ranges defined for the plurality of picked-up images P, and the plurality of picked-up images by the self-organizing map after learning. A plurality of clusters for classifying the color information and the color range of the clusters are defined.

  In step ST5, the mobile information processing apparatus 1 performs a process for generating a cluster list. Specifically, the control unit 8 specifies a cluster in which the color information of the captured image is classified for each captured image. In addition, the cluster list generation unit 815 preferentially excludes a plurality of clusters in the descending order of the number of captured images associated with each cluster, and distributes a unique color (color color) included in each image. A cluster list (CL) indicating the pattern of

  In step ST6, the mobile information processing apparatus 1 performs processing for generating landmark data. Specifically, the control unit 8 generates landmark data (RMD) that associates the identified cluster, the captured image, and the position information based on the cluster list and the captured image P described above.

  Hereinafter, the operation of the mobile information processing apparatus 1 will be described in detail with reference to the drawings. In the present embodiment, the control unit 8 performs processing related to learning of the self-organizing map.

  FIG. 5 is a diagram for explaining clustering based on a captured image and a self-organizing map. Specifically, FIG. 5A is a diagram for explaining a color captured image P to be learned, and FIG. 5B shows color information of each pixel of the captured image P shown in FIG. It is the figure shown in RGB space. FIG. 5C is a diagram for explaining a plurality of clusters generated using the captured image P shown in FIG. 5B as input data and the color range of each cluster.

In step ST1, the control unit 8 acquires a color image P as shown in FIG. The color information of the pixels of the color image is converted into RGB values of a predetermined gradation by the quantization unit 3. The RGB value of each pixel of the captured image P can be represented in the RGB space as shown in FIG.
Further, as shown in FIG. 5C, the color pattern in the RGB space is classified into a plurality of defined clusters CL ₁ ^j by learning using a self-organizing map described later. The cluster CL _i ^j indicates the j-th cluster CL in the i-th captured image P.

Figure 6 is a diagram for explaining a color information included in the cluster CL ₁ ¹ shown in FIG. 5 (C).
For example, as shown in FIG. 6, the color information of the pixels included in the cluster CL ₁ ¹ is eight pieces of color information {(R11, G11, B11), (R12, G12, B12), (R13, G13, B13). , (R14, G14, B14), (R15, G15, B15), (R16, G16, B17), (R18, G18, B18)}.

FIG. 7 is a diagram for explaining each cluster illustrated in FIG. 5C and color information included in the cluster.
Each of the clusters CL ₁ ¹ , CL ₁ ² ,..., CL ₁ ⁿ shown in FIG. 5C includes color information of each pixel as shown in FIG.
Specifically, the cluster CL ₁ ¹ includes {(R11, G11, B11), (R12, G12, B12),..., (R18, G18, B18)}, and the cluster CL ₁ ² includes , {(R21, G21, B21), (R22, G22, B22),..., (R2b, G2b, B2b)} are included, and the cluster CL ₁ ⁿ includes {(Rn1, Gn1 , Bn1), (Rn2, Gn2, Bn2), ..., (Rnc, Gnc, Bnc)}.

The cluster CL and the color range are defined by learning a self-organizing map described later.
The self-organizing map learning (clustering process) will be described below.

[Self-organizing map]
FIG. 8 is a diagram for explaining a self-organizing map according to the present invention. Specifically, FIG. 8A is a diagram for explaining a self-organizing map in which a plurality of nodes are defined in advance in the competitive layer, and FIG. 8B is a diagram in which one node is present in the competitive layer at the initial stage. FIG. 8C is a diagram for explaining a prescribed self-organizing map, and FIG. 8C is a diagram for explaining an operation in which nodes in a competitive layer of the self-organizing map shown in FIG. 8B proliferate. It is.

First, a general self-organizing map algorithm will be described with reference to FIG.
For example, as shown in FIG. 8A, in a general self-organizing map algorithm, the configuration has a two-layer structure of an input layer 40 and a competitive layer 50. A common self-organizing map is an unsupervised learning neural network.
The input layer 40 corresponds to, for example, an n-dimensional input vector x (x ₁ , x ₂ , x ₃ ,..., X _n ). The input vector x is also called input data. The input vector x according to the present embodiment is, for example, data such as color information of each pixel of the captured image, a color range of a cluster defined by classifying color information of each pixel of the captured image, and the like.

The competitive layer 50 has a plurality of nodes (elements) arranged in a prescribed dimension, for example, one dimension or two dimensions, or two dimensions in this embodiment. Each node i has a load vector m _i (also referred to as a weight vector or load coupling) of the same dimension (n dimension) as the input vector x. The input data x is coupled to each node i of the competitive layer 50. In the competitive layer 50 according to the present embodiment, for example, a predetermined number of nodes, for example, h × h nodes i are arranged in a lattice pattern.

A general self-organizing map learning method shown in FIG. 8A will be described.
First, define the initial value of the weight vector m _i of each node i of competitive layer 50 at random.
Then, identify the measures with respect to the input vector x, for example, a node having a load vector m _i to the Euclidean distance between the minimum i (Win node), then the load vector m _i victory node i, the load vector The load vector of the node near m _i is updated so as to approach the input vector x (competitive learning).

Load vector m _i, for example by using learning rate α a (t), can be expressed as Equation (1). The learning rate coefficient α (t) is a function using the number of learnings and the distance from the winning node as parameters, and is defined such that the value decreases as the number of learnings and the distance increase, for example.

In the self-organizing map algorithm shown in FIG. 8A, the load vector mi of the winning node _i is updated in the competitive layer 50, and the load vectors of the nodes near the winning node i are also updated. As a result, a plurality of clusters are formed in the post-learning self-organizing map (competitive layer 50) by nodes having load vectors according to the input data x.
In the mobile information processing apparatus 1 according to the present invention, learning may be performed using the above-described self-organizing map algorithm. However, by performing clustering processing using a self-organizing map (self-propagating type) described later, Landmark data can be generated at high speed and with high accuracy.

[Self-organizing map according to this embodiment]
The self-organizing map algorithm according to the present embodiment will be described with reference to FIGS.
The self-organizing map according to the present embodiment has a two-layer structure including an input layer 40 and a competitive layer 50. The number of nodes in the input layer 40 is constant. The number of nodes in the competitive layer 50 is set to one at the initial time.

[Node proliferation]
Also, new to when the firing frequency of the node ia (selection count) exceeds the threshold value, excluding the node ia from the learning of the target by a specified period (refractory period), with the same load vector m _i and the node ia The node ib is propagated to learn the self-organizing map. That is, during the refractory period, the node ib is a learning target.
In this way, by multiplying the color nodes having a high appearance frequency, the resolution (resolution) in the vicinity of the color having the high appearance frequency can be increased. For this reason, more accurate clustering processing can be performed.

[Competitive learning]
Moreover, the control part 8 also performs the competitive learning mentioned above as learning of the self-organization map which concerns on this embodiment. At this time, the control unit 8 does not learn the neighboring nodes, but performs hierarchical learning instead.

[Layer learning]
The control unit 8 sets the propagated node ib as a child node and the propagated node ia as a parent node, and updates one of the parent node ia and the child node ib together with the other node. , Self-organizing map learning (hierarchical learning).

  In addition, when the control unit 8 selects a node in which the load vector having the minimum distance is associated with the input vector and the distance exceeds the threshold, the control unit 8 excludes the node from the learning target, A new node having the same load vector as the vector is generated to learn the self-organizing map.

Remove node
In addition, the control unit 8 removes nodes that have not been updated for a specified period and learns a self-organizing map.

  Next, learning of the self-organizing map of the mobile information processing apparatus 1 is expressed in RGB space and described in detail.

  FIG. 9 is a flowchart for explaining the clustering process by learning the self-organizing map according to the first embodiment of the present invention. FIG. 10 is a diagram showing an initial node i in the RGB space.

  In step ST101, the control unit 8 generates one initial node i as shown in FIG. The load vector (load coupling) is set to a random value within a specified range.

FIG. 11A is a captured image P according to a specific example, and FIG. 11B is a diagram showing color information of each pixel of the captured image in the RGB space.
In step ST102, as shown in FIG. 11B, the control unit 8 has diamond-shaped points as input vectors of color information, and inputs the input vectors as an input data set.

FIG. 12 is a diagram for explaining the input data x and the Euclidean distance of all nodes i.
In step ST103, the control unit 8 measures the input data x and the Euclidean distance d of all the nodes i as shown in Equation (2), for example, and selects the node i1 (victory node) having the smallest distance d. Do (determine). The i-th input vector x _{i of} the input layer and the i-th input vector of the input layer and the j-th load vector w _{i, j} of the competitive layer are represented.

FIG. 13 is a diagram illustrating the relationship between the distance d between the input vector x and the node i and the threshold value. FIG. 14 is a diagram for explaining addition of a new node.
The control unit 8 determines whether or not the distance between the input vector x and the node i is smaller than the threshold value th, and as a result, if the distance is determined to be larger than the threshold value, the control unit 8 proceeds to the processing of step ST105. If it is determined that the value is smaller, the process proceeds to step ST106.

FIG. 14 is a diagram for explaining addition of a new node.
In Step ST105, the control unit 8 selects a node in which the load vector i1 having the minimum distance is associated with the input vector x, and when the distance exceeds the threshold value ih, the control unit 8 learns the node i1. And a new node i5 having the same load vector as the input vector x is generated to learn the self-organizing map, and the process proceeds to step ST110.

FIG. 15A is a diagram for explaining node updating, and FIG. 15B is an enlarged view of the vicinity of the winning node shown in FIG. 15A.
In step ST106, the control unit 8 performs a process of updating the load coupling of the winning node. Specifically, the control unit 8 calculates the victory node by using the load vector w _{i, win} of the victory node and the learning rate coefficient α (t), for example, as shown in Equation (3), and updates the victory node.

[Equation 3]
w _{i, win} (t + 1) = w _{i, win} (t) + α (x _i (t) −w _{i, win} (t)) (3)

As shown in FIGS. 15A and 15B, the weight vector of the winning node i approaches the input data x and becomes the node i ′ by learning.
FIG. 16A is a diagram for explaining hierarchical learning of the same cluster as the winning node, and FIG. 16B is an enlarged view of the vicinity of the winning node shown in FIG.
In step ST107, the control unit 8 also uses the parent node load vector w _{i, mo} and the child node load vector w _{i, da} , for example, for the node i2 having the same hierarchical structure as the winner node i1. Update is performed as shown in (4) (hierarchical learning). The coefficient 0.1 of the learning rate coefficient α is appropriately set according to various conditions.

[Equation 4]
w _{i, mo} (t + 1) = w _{i, mo} (t) + 0.1α (w _{i, da} (t) −w _{i, mo} (t)) (4)

Next, when the number of selections of the node selected according to the input vector exceeds the threshold value, the control unit 8 excludes the node from the learning target for a specified period, and adds a new load vector having the same load vector as the node. A self-organizing map is learned by multiplying various nodes.
Specifically, in step ST108, the control unit 8 determines whether or not the number of reactions of the winning node is greater than a predetermined threshold value, and proceeds to the processing of step ST109 when determining that the number of reactions is greater than the threshold value. If the number of recoils is less than or equal to the threshold value, the process proceeds to step ST110.

FIG. 17A is a diagram for explaining the proliferation of victory nodes, and FIG. 17B is an enlarged view of the vicinity of the victory node shown in FIG.
In step ST109, the control unit 8 propagates the winning node (parent node) i2a to generate a child node i2b having the same load vector as the parent node i2a, and sets the winning node that is the parent node i2a as a refractory period for a certain period. And For example, in FIG. 17B, the side where the black circle is formed at the end of the line connecting the nodes is set as the child node, and the side where the black circle is not formed at the end of the line connecting the nodes is set as the parent node. The process proceeds to ST110.
That is, when updating one of the parent node and the child node, the control unit 8 updates the other node and learns the self-organizing map.

  In step ST110, when a plurality of pixels are set as input data, the control unit 8 determines whether or not the processing is completed for the data set, and as a result of the determination, for all input data in the data set. It is determined whether or not the above process is completed. If it is determined that the process is completed, the process proceeds to step ST111. Otherwise, the process returns to step ST103.

In step ST111, the control unit 8 performs node consolidation processing. Specifically, the control unit 8 removes nodes that have not been updated for a specified period and learns a self-organizing map.
FIG. 18A is a diagram for explaining node consolidation processing, FIG. 18B is an enlarged view of the vicinity of the node i3a, and FIG. 18C is an enlarged view of the vicinity of the node i3a.

As shown in FIGS. 18A and 18B, when the child node i3b and the child node i3c belong to the same cluster in the parent node i3a and the node i3a is stopped (removed), each of the nodes i3b and i3c is a new cluster. It becomes.
As shown in FIGS. 18A and 18C, when the node i1a has child nodes i1b, i1c, and i1d belonging to the same cluster, and the node i1b is stopped (removed), the node i1a and the node i1c are reconnected. Connected. If there is no parent node, the child node becomes a new cluster.

  In step ST112, the control unit 8 determines whether or not the number of learning is larger than the prescribed number, and if it is determined that the number of learning is larger, the process proceeds to step 113. Otherwise, the process proceeds to step 102. Return. Specifically, in this embodiment, the above steps are repeated about 1000 to 3000 times to learn the self-organizing map.

  In step ST113, the control unit 8 defines a plurality of clusters corresponding to the distribution of the color information and the color range (maximum value, minimum value) of the clusters based on the self-organizing map after learning.

FIG. 19A is a diagram illustrating a plurality of captured images P, FIG. 19B is a diagram in which clusters of the plurality of captured images P are superimposed, and FIG. It is a figure for demonstrating each cluster list.
Next, as shown in FIG. 19C, the control unit 8 specifies a cluster in which the color information of the captured image P is classified into the captured image P, and creates a cluster list in which the cluster is associated with each image. Generate.
For each image P, a list of clusters into which the color information of each pixel is classified is generated.
Specifically, as shown in FIG. 19C, the image P1 includes a cluster CL ₁ ¹ , a cluster CL ₁ ² ,..., A cluster CL ₁ ^a . The image P2 includes a cluster CL ₂ ¹ , a cluster CL ₂ ² ,..., A cluster CL ₂ ^b . The image Pn includes a cluster CL _n ¹ , a cluster CL _n ² ,..., A cluster CL _n ^c .
Each cluster CL defines a color range, specifically, a minimum value and a maximum value of the color.

20A is a diagram in which a plurality of clusters of a plurality of captured images P are overlapped, and FIG. 20B is a clustering process performed on the clusters illustrated in FIG. 20A using a self-organizing map. It is a figure which shows a back state.
For example, as described above, the control unit 8 learns the self-organizing map based on the cluster and the color range (maximum value and minimum value) of the cluster. As a result, as shown in FIG. 20B, the entire cluster is formed by clusters having similar RGB values.

Specifically, as shown in FIG. 20C, as a result of learning of the self-organizing map, the color information cluster of each pixel of the plurality of captured images P and the color range of the cluster can be defined.
As shown in FIG. 20C, the relationship between the cluster ID and the color range of the cluster (minimum RGB value, maximum RGB value) is, for example, CL _ALL ¹ = (R11, G11, B11, R12, G12, B12), CL _ALL ² = (R21, G21, B21, R22, G22, B22), ..., CL _ALL ^m = (Rm1, Gm1, Bm1, Rm2, Gm2, Bm2). Here, the j-th cluster after the second clustering process is represented as CL _ALL ^j .

[Classification of clusters based on appearance frequency (removal of frequent clusters)]
FIG. 21 is a diagram for explaining frequent cluster removal processing of clusters.
The cluster CL _ALL ^m includes a cluster into which color information that can be acquired anywhere in the mobile environment 200 such as a wall or a floor is classified. By removing clusters that are not directly useful for such position estimation, highly accurate landmark data can be generated. Specifically, a common cluster is removed from a plurality of images P taken in the mobile environment 200.

  Specifically, the control unit 8 preferentially excludes a plurality of clusters from the plurality of clusters in descending order of the number of captured images associated with each cluster. That is, a cluster list (CL) indicating the characteristics of the color distribution (color pattern) included in each image is generated by removing clusters in which colors included in a plurality of captured images are classified. be able to.

More specifically, as shown in FIG. 21, the image P1, P2 taken in the mobile environment 200, ..., for each PN, clusters _{^{CL ALL 1, CL ALL 2,}} ···, CL ALL j ,..., A list (cluster list) indicating whether CL _ALL ^m is included is generated. As shown in FIG. 21, for example, the captured image P1 includes clusters CL _ALL ¹ and CL _ALL ^j , and does not include other clusters.
As shown in FIG. 21, based on the cluster list, the cluster CL _ALL ^j that is commonly included in each image is included in all the images P. Therefore, the cluster CL _ALL ^j is removed and a new color is added. Cluster IDs CCL ¹ , CCL ² ,..., CCL ^x are assigned. No color cluster ID is assigned to this cluster CL _ALL ^j .

22A is a diagram for explaining a color cluster list, FIG. 22B is a diagram showing a captured image to be learned, and FIG. 22C is a color cluster included in the image to be learned. It is a figure for demonstrating the list (landmark data) of.
In the color cluster list shown in FIG. 21, the color range (minimum RGB value and maximum RGB value) of each color cluster CCL is defined as shown in FIG. 22A, for example. The cluster CCL and the color range are obtained as a result of the second clustering process.
In particular, the color range to the color cluster ^{CCL 1 (R11, G11, B11} , R12, G12, B12) corresponds, color range to the color cluster ^{CCL 2 (R21, G21, B21} , R22, G22, B22) corresponds to..., And the color range (RX1, GX1, BX1, RX2, GX2, BX2) corresponds to the color cluster CCL ^j .

Next, as shown in FIG. 22B, the reaction between each of the color images P captured in the mobile environment 200 and the color cluster list shown in FIG. 22A is examined. As a result, a list in which the reacted color cluster CLL is associated with each image P is generated as shown in FIG.
Specifically, as shown in FIG. 22C, for example, the image P1 includes color clusters CCL ¹ and CCL ² , and the image P2 includes color clusters CCL ² and CCL ^5. P3 does not include a color cluster (NULL), and the image PN includes a color cluster CCL ^x .

  Further, as a method for specifying the color cluster, the control unit 8 uses the learned self-organization map to input the color information of the pixels of each image as an input vector, and as a result, determines by the reacting cluster. May be.

As shown in FIG. 22C, a list in which color clusters are associated with each image P is referred to as landmark data (RMD). Further, as shown in FIG. 22C, each image of the landmark data is associated with position information regarding the imaging position.
That is, the control unit 8 generates landmark data (RMD) in which a captured image, a color cluster in which color information of the captured image is classified, and position information regarding the imaging position of the captured image are associated. The control unit 8 stores the landmark data (RMD) in the storage unit 4.
In the above-described embodiment, each captured image P, color cluster, and position information are associated as landmark data. However, the present invention is not limited to this form. For example, data (list) in which the captured image is associated with the color cluster may be generated and stored in the storage unit 4. As a result, the storage unit 4 stores the positional information and the captured image in association with each other, and further stores the captured image and the color cluster in association with each other. That is, each element of the landmark data is not limited to the above form as long as it is stored in association with each other.
In the landmark data according to the present embodiment, as described above, a captured image and a color cluster (a combination of color clusters) into which color information of the captured image is classified are clearly defined.

[Operation when using landmark data (position estimation processing)]
FIG. 23 is a diagram for explaining an operation of performing position estimation after the mobile information processing apparatus 1 generates landmark data. FIG. 24 is a flowchart for explaining the operation of the mobile information processing apparatus shown in FIG. 1 when using landmark data. FIG. 25A is a diagram illustrating a captured image P for position estimation, and FIG. 25B is a diagram illustrating color information of the captured image P for position estimation in RGB space.
FIG. 26A is a diagram showing a cluster CLL in which color information of a captured image is classified, and FIG. 26B is a diagram of RGB space indicating input data (captured image for position estimation) and clusters.

The mobile body information processing apparatus 1 generates landmark data by learning the self-organizing map, and then estimates the position of the mobile body (robot) using the landmark data and a captured image for position estimation. Specifically, when the position of the moving body 100 is estimated when the moving body (robot) 100 is arranged in the environment 200, the moving body information processing apparatus 1 captures a captured image for position estimation captured at that position. Then, the degree of similarity between the captured images of the learning target is calculated, and processing related to position estimation is performed based on the similarity and the position information associated with the captured image of the learning target. Specifically, the control unit 8 identifies a captured image for learning having a relatively high similarity to the captured image for position estimation, and estimates the position based on position information associated with the identified captured image. I do.
Hereinafter, operations related to position estimation of the mobile information processing apparatus 1 will be described in detail.

  In the present embodiment, the mobile information processing apparatus 1 is placed at an arbitrary position in the moving environment 200, for example, at a position between a wooden box and a houseplant provided near the wooden box, as shown in FIG. Placed. The traveling direction of the mobile information processing apparatus 1 is substantially the same as the direction in which the shelf is located, and the imaging direction of the imaging unit is the same.

  In step ST201, the mobile information processing apparatus 1 images the subject in the moving environment 200 by the imaging unit 2 at the initial position as shown in FIG. 24, for example, and estimates the position as shown in FIG. A color image P is obtained. Specifically, the control unit 8 drives the imaging unit 2 to acquire the color image P for position estimation, and acquires color information as shown in FIG. For example, in this captured image P for position estimation, red-brown pots, houseplants with green leaves, a substantially white wall, and an indigo floor are captured as subjects.

FIG. 27 is a diagram for explaining matching between the color cluster CLL to which the color information of each pixel of the captured image for position estimation reacts and landmark data.
In step ST202, as shown in FIGS. 26A and 26B, the control unit 8 classifies the color information of each pixel of the captured image for position estimation based on the cluster CLL and the color range. Specify CLL. In this embodiment, for example, as shown in FIGS. 26 and 27, color clusters CLL ¹ and CLL ² are specified.

In step ST204, the control unit 8 calculates the similarity of the captured image to be learned with respect to the captured image for position estimation in accordance with the identified cluster and landmark data. As the similarity, for example, a value obtained by quantifying the degree of coincidence between the combination of the clusters CLL identified from the color information of the captured image for position estimation and the combination of each cluster CLL of the landmark data (RMD) is adopted. . At this time, a weight may be set for each cluster CLL of the landmark data, and the similarity may be calculated in consideration of the weight. For example, the weight of each cluster is preferably set to be larger as the number of images including the cluster is smaller. By doing so, the similarity can be determined with high accuracy.
The similarity is not limited to this form. For example, the similarity may be calculated by an existing similarity determination method.

  Next, the control unit 8 determines whether or not the calculated similarity is greater than a threshold value. When the control unit 8 determines that the similarity is greater than the threshold value, the control unit 8 proceeds to the process of step ST105. Proceed to the process.

  In step ST205, the control unit 8 specifies a captured image P for learning whose similarity is greater than a threshold value, and performs processing related to position estimation based on position information associated with the captured image P. For example, the control unit 8 determines that the difference between the position information associated with the specified captured image P and the self position of the moving body (robot 100) is relatively small. That is, the control unit 8 estimates that the mobile object's own position is in the vicinity of the imaging position of the specified captured image.

When determining that the similarity is equal to or less than the threshold value, the control unit 8 drives the drive unit 5 to move the moving body (robot) 100, and returns to the process of step ST201.
At this time, for example, the control unit 8 specifies the position information associated with the captured image having the highest similarity, which is equal to or lower than the threshold, moves to that position, and returns to the process of step ST201. The position estimation process described above may be performed again. In this way, by performing position estimation while moving until the similarity is equal to or greater than the threshold, the self-position can be specified with high accuracy.

As another specific example of the movement control, for example, in step ST206, the control unit 8 determines whether or not the rotation angle with respect to the vertical axis of the robot 100 is larger than a threshold, and determines that the rotation angle is large. After the robot 100 is rotated by controlling the drive unit 5 so as to rotate right by a predetermined angle, for example, 90 degrees (ST207), the process returns to step ST201. On the other hand, when determining that the rotation angle is equal to or less than the threshold value, the control unit 8 moves by a predetermined distance and returns to the process of step ST201.
In this way, by repeating the above-described processing until the similarity is equal to or higher than the threshold value, the self-position can be specified with high accuracy.

  Further, for example, when the similarity of the captured image for position estimation is equal to or lower than the threshold value and it cannot be classified into the predetermined color cluster CLL, the control unit 8 selects the captured image for position estimation. On the other hand, the above-described processing for generating landmark data may be performed to generate a new cluster for the landmark data.

[Second Embodiment]
FIG. 28 is a diagram for explaining a mobile information processing apparatus according to the second embodiment of the present invention.
As shown in FIG. 28, the mobile information processing apparatus 1A according to the present embodiment includes a mobile (robot) 100A and an external computer 100B. The description of the same components, functions, operations, etc. as in the first embodiment and the second embodiment is omitted.

The robot 100A according to the present embodiment includes a communication unit 9 that can communicate with the external computer 100B. The communication unit 9 performs data communication with the external computer 100B under the control of the control unit 8.
The external computer 100B includes a storage unit, a communication unit, a control unit, and the like, and can control the robot 100A by performing data communication with the robot 100A. The external computer 100B performs, for example, processing that requires relatively high arithmetic processing capability, such as landmark generation processing and position estimation processing by clustering processing according to the present invention.

  When the robot 100A receives a control signal indicating an imaging command from the external computer 100B, the robot 100A performs imaging by the imaging unit 2 and transmits the captured image P to the external computer 100B via the communication unit 9. When receiving a control signal indicating a wheel driving command from the external computer 100B, the robot 100A drives and moves the wheel according to the control signal.

In the mobile information processing apparatus 1A according to the present embodiment, since the robot 100A and the external computer 100B perform distributed processing as described above, the robot 100A can be configured to be relatively lightweight.
Also, landmark data can be generated with higher accuracy by providing a plurality of robots 100A and the external computer 100B comprehensively controlling the plurality of robots 100A. In addition, since the amount of information to be acquired increases, position estimation can be performed with higher accuracy.

  The present invention is not limited to the embodiment described above. If the functions according to the present invention can be realized, various forms can be taken.

  For example, in the above-described embodiment, each pixel of the color image has an RGB value as color information, but the present invention is not limited to this form. For example, the color information of each pixel of the color image may be data expressed by a prescribed color space such as YIQ (luminance signal Y, color difference signals RY, BY), YCbCr, or the like.

  The functions according to the present invention may be realized by a computer executing a program, or may be realized by a dedicated electronic circuit performing information processing.

  Moreover, the operation | movement at the time of the position estimation of the mobile information processing apparatus 1 is not restricted to embodiment mentioned above. Arbitrary operations may be performed so that the functions according to the present invention can be realized.

  Further, in the above-described embodiment, a plurality of color images are captured at a predetermined timing by the imaging unit. However, for example, about several tens of frames of moving images may be captured per second. The mobile information processing apparatus 1 can generate more accurate landmark data by performing clustering processing according to the present invention on a moving image, and can estimate a position with high accuracy using the landmark data. It can be performed.

  Moreover, the mobile body information processing apparatus 1 according to the present invention can be applied to a robot that can move in a moving environment such as a cleaning robot, a transport robot in a factory, a road guide robot, and the like.

  As described above, the mobile information processing apparatus 1 according to the present invention includes an imaging unit 2 that captures a color image by capturing a subject in a moving environment, a plurality of captured images to be learned, and the captured image. The storage unit 4 that stores the positional information related to the imaging positions in association with each other, and learns the self-organizing map based on the color information of each pixel of the plurality of captured images, and the color information is obtained by the self-organizing map after learning. Defines a plurality of clusters to be classified and the color range of the clusters, identifies a cluster in which the color information of the captured image is classified for each captured image, and associates the identified cluster, captured image, and position information Control unit 8 for generating landmark data, for example, from a plurality of color images photographed within the moving range without arranging landmarks in advance in the traveling environment of the moving body. It can generate landmark data for specifying the position of the moving object.

  Further, the control unit 8 identifies a cluster in which color information of each pixel of the captured image for position estimation is classified based on the cluster and the color range, and according to the identified cluster and landmark data, The degree of similarity of the captured image to be learned with respect to the captured image for position estimation is calculated, and processing related to position estimation is performed based on the similarity and the position information associated with the captured image to be learned. In addition, the position of the moving object can be estimated from the captured image for position estimation and the landmark data.

  Further, the control unit 8 specifies a cluster into which the color information of each pixel of the captured image is classified based on the cluster and the color range based on the self-organizing map after learning, so that the color information of the captured image can be easily obtained. Clusters to be classified can be specified.

  In addition, the control unit 8 performs processing for generating landmark data by preferentially excluding the clusters in the descending order of the number of captured images associated with each cluster from among a plurality of clusters. By removing clusters in which color information that can be acquired anywhere in the mobile environment 200 such as a floor is classified, highly accurate landmark data can be generated.

  In addition, the mobile information processing apparatus 1 according to the present invention is a competitor that includes an input layer 40 in which data relating to color information of a captured image is input as an input vector, and a node in which a load vector having the same dimension as the input vector is associated. The control unit 8 selects a node in which a load vector having a minimum distance is associated with an input vector, updates the load vector, and updates the self-organizing map. Since unsupervised learning is performed, clustering can be performed by simple processing.

  In addition, when the number of selections of the node selected according to the input vector exceeds the threshold, the control unit 8 excludes the node from the learning target for a specified period, and creates a new load vector having the same load vector as the node. Since nodes are proliferated and self-organizing maps are learned, more accurate clustering processing can be performed by increasing the resolution (resolution) in the vicinity of frequently occurring colors through node proliferation. .

  In addition, when the control unit 8 sets the propagated node as a child node and the propagated node as a parent node, and updates one of the parent node and the child node, the control unit 8 updates the other node together with the self-organization. Since the map is learned, more accurate clustering processing can be performed.

  In addition, when the control unit 8 selects a node in which the load vector having the minimum distance is associated with the input vector and the minimum distance exceeds the threshold, the control unit 8 excludes the node from the learning target, Since a new node having the same load vector as the input vector is generated and the self-organizing map is learned, more accurate clustering processing can be performed.

  In addition, the control unit 8 removes the nodes that have not been updated for the specified period and learns the self-organizing map, so that more accurate clustering processing can be performed.

  Further, the control unit 8 learns a self-organizing map based on the color information of each pixel of the captured image, and a plurality of clusters for classifying the color information by the self-organizing map after learning, and the clusters Based on the first clustering processing unit 812 that defines the color range and the clusters and color ranges defined for each of the plurality of captured images, learning of the self-organizing map is performed, and the self-organizing map after learning is used. A plurality of clusters for classifying the color information of the plurality of captured images and a second clustering processing unit 814 that defines a color range of the clusters, so that each pixel of the plurality of captured images can be obtained by simple processing. Color information can be clustered with relatively high accuracy.

  The mobile information processing apparatus 1 includes an imaging unit 2 that captures a subject around the mobile unit 100 to generate a color image, and a drive unit 5 that moves the mobile unit 100 in a mobile environment. The unit 8 drives and controls the drive unit 5 to move the moving body 100 and perform processing according to the image captured by the imaging unit 2 of the moving body 100 to generate landmark data with high accuracy, for example. Processing and position estimation processing can be performed.

  In addition, the mobile information processing apparatus 1 can generate landmark data using an image corresponding to the imaging direction even when the imaging unit 2 is fixed so that the imaging direction is substantially the same as the traveling direction. And processing related to position estimation can be performed.

1 is an overall functional block diagram of a mobile information processing apparatus according to a first embodiment of the present invention. It is a figure for demonstrating the movement environment of the mobile information processing apparatus shown in FIG. It is a functional block diagram for demonstrating the function of the control part of the mobile information processing apparatus shown in FIG. 3 is a flowchart for explaining the operation of the mobile information processing apparatus shown in FIG. It is a figure for demonstrating the clustering by a captured image and a self-organizing map, (A) is a figure for demonstrating the color captured image P of learning object, (B) is the captured image shown to (A). FIG. 4C is a diagram showing color information of each pixel of P in RGB space, and FIG. 8C illustrates a plurality of clusters generated using the captured image P shown in FIG. 5B as input data and the color range of each cluster. FIG. It is a diagram for explaining a color information included in the cluster CL ₁ ¹ shown in FIG. 5 (C). It is a figure for demonstrating the color information contained in each cluster shown in FIG.5 (C), and a cluster. It is a figure for demonstrating the self-organization map which concerns on this invention, (A) is a figure for demonstrating the self-organization map by which the some node is previously prescribed | regulated by the competitive layer, (B) It is a figure for demonstrating the self-organization map by which one node is prescribed | regulated to the competitive layer at the time of initial stage, (C) is the operation | movement which the node of the competitive layer of the self-organization map shown to (B) proliferates. It is a figure for demonstrating. It is a flowchart for demonstrating the clustering process by the learning of the self-organization map which concerns on 1st Embodiment of this invention. It is a figure which shows the initial node i in RGB space. (A) is the captured image P which concerns on one specific example, (B) is the figure which showed the color information of each pixel of a captured image in RGB space. It is a figure for demonstrating the input data x and the Euclidean distance of all the nodes i. It is a figure which shows the relationship between the distance d between the input vector x and the node i, and a threshold value. It is a figure for demonstrating addition of a new node. (A) is a figure for demonstrating the update of a node, (B) is the figure which expanded the victory node vicinity shown to (A). (A) is a figure for demonstrating the hierarchical learning of the same cluster as a victory node, (B) is the figure which expanded the victory node vicinity shown to (A). (A) is a figure for demonstrating the proliferation of a victory node, (B) is an enlarged view of the vicinity of the victory node shown to (A). (A) is a figure for demonstrating a node consolidation process, (B) is an enlarged view near node i3a, (C) is an enlarged view near node i3a. (A) is a diagram showing a plurality of captured images P, (B) is a diagram in which clusters of the plurality of captured images P are overlaid, and (C) describes a cluster list for all captured images P. It is a figure for doing. (A) is a diagram showing a plurality of clusters of a plurality of captured images P, and (B) is a diagram showing a state after the cluster shown in (A) is subjected to clustering processing by a self-organizing map. It is. It is a figure for demonstrating the removal process of the frequent cluster of a cluster. (A) is a figure for demonstrating a color cluster list, (B) is a figure which shows the captured image of learning object, (C) is a list (landmark data) of the color cluster contained in the image of learning object. FIG. It is a figure for demonstrating the operation | movement which performs position estimation, after the mobile body information processing apparatus 1 produces | generates landmark data. 3 is a flowchart illustrating an operation when landmark data is used in the mobile information processing apparatus shown in FIG. (A) is the figure which shows the captured image P for position estimation, (B) is the figure which showed the color information of the captured image P for position estimation in RGB space. (A) is a diagram showing a cluster CLL in which color information of a captured image is classified, and (B) is a diagram of RGB space indicating input data (captured image for position estimation) and clusters. It is a figure for demonstrating matching with the color cluster CLL with which the color information of each pixel of the captured image for position estimation reacts, and landmark data. It is a figure for demonstrating the mobile information processing apparatus which concerns on 2nd Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Mobile body information processing apparatus 2 Imaging part 3 Quantization part 4 Storage part 5 Drive part 6 Wheel 7 Sensor 8 Control part 10 Case 81 1st process part 82 2nd process part 100 Robot 811 Drive control part 812 1st Clustering processing unit (first SOM)
813 Maximum value / minimum value extraction unit 814 Second clustering processing unit (second SOM)
815 Cluster list generation unit 816 Landmark data generation unit 821 Cluster acquisition unit 822 Matching unit 823 Position estimation unit 824 Drive control unit

Claims (15)

A mobile information processing apparatus that generates landmark data for performing position estimation from a color image captured in a mobile environment,
A storage unit that associates and stores a plurality of captured images to be learned and position information related to an imaging position of the captured image;
Learning a self-organizing map based on color information of each pixel of the plurality of captured images, and a plurality of clusters for classifying the color information by the self-organizing map after learning, and a color range of the cluster Control for specifying the cluster in which color information of the captured image is classified for each captured image, and generating the landmark data in which the identified cluster, the captured image, and the position information are associated with each other And a mobile information processing apparatus.   The control unit identifies the cluster into which color information of each pixel of the captured image for position estimation is classified based on the cluster and the color range, and according to the identified cluster and the landmark data Then, the similarity of the captured image to be learned with respect to the captured image for position estimation is calculated, and processing related to the position estimation is performed based on the similarity and the position information associated with the captured image to be learned. The mobile information processing apparatus according to claim 1.   The control unit identifies the cluster into which color information of each pixel of the captured image is classified based on a cluster based on the self-organizing map after learning and the color range. The mobile information processing apparatus according to claim 1.   The control unit performs processing for generating the landmark data by preferentially excluding the clusters from the plurality of clusters in descending order of the number of captured images associated with each cluster. The mobile information processing apparatus according to claim 1. A neural network including an input layer in which data regarding color information of the captured image is input as an input vector, and a competitive layer including a node associated with a load vector of the same dimension as the input vector;
The control unit selects a node associated with the load vector having a minimum distance with respect to the input vector, updates the load vector, and learns the self-organizing map. The mobile information processing apparatus according to claim 1.   When the number of times of selection of the node selected according to the input vector exceeds a threshold, the control unit excludes the node from the learning target for a specified period, and adds a new load vector having the same load vector as the node. The mobile information processing apparatus according to claim 5, wherein the self-organizing map is learned by multiplying various nodes.   The control unit sets the propagated node as a child node and the propagated node as a parent node, and when updating one of the parent node and the child node, updates the other node together with the self node The mobile information processing apparatus according to claim 6, wherein learning of an organized map is performed.   When the control unit selects a node associated with the load vector having the minimum distance with respect to the input vector and the minimum distance exceeds a threshold, the control unit excludes the node from learning targets; 6. The mobile information processing apparatus according to claim 5, wherein a new node having the same load vector as the input vector is generated to learn the self-organizing map.   The mobile information processing apparatus according to claim 5, wherein the control unit learns the self-organizing map by removing a node that has not been updated for a specified period. The control unit learns a self-organizing map based on color information of each pixel of the captured image, and a plurality of clusters for classifying the color information by the self-organizing map after learning, and the cluster A first clustering processing unit for defining a color range;
Based on the cluster and the color range defined for each of the plurality of captured images, learning of a self-organizing map is performed, and the color information of the plurality of captured images is classified based on the self-organizing map after learning. The mobile information processing apparatus according to claim 1, further comprising: a plurality of clusters and a second clustering processing unit that defines a color range of the clusters. An imaging unit that images a subject around the moving object to generate a color image;
A drive unit for moving the moving body in a moving environment;
2. The information according to claim 1, wherein the control unit drives and controls the driving unit to move the moving body and performs processing according to an image captured by the imaging unit of the moving body. Processing equipment. A landmark for storing a plurality of captured images that have been color-captured in a moving environment and positional information related to the imaging position of the captured image, and performing position estimation based on the color image captured in the moving environment An information processing method for a mobile information processing apparatus for generating data,
Learning a self-organizing map based on color information of each pixel of the plurality of captured images, and a plurality of clusters for classifying the color information by the self-organizing map after learning, and a color range of the cluster A first step defining:
A second step of identifying the cluster into which color information of the captured image is classified for each captured image, and generating the landmark data that associates the identified cluster, the captured image, and the position information; An information processing method for mobile information processing, characterized by comprising:   Based on the cluster and the color range, the cluster in which color information of each pixel of the captured image for position estimation is classified is identified, and the position estimation is performed according to the identified cluster and the landmark data Calculating a degree of similarity of the captured image to be learned with respect to the captured image, and performing a process relating to the position estimation based on the similarity and position information associated with the captured image to be learned 13. The information processing method for mobile object information processing according to claim 12, further comprising: A landmark for storing a plurality of captured images that have been color-captured in a moving environment and positional information related to the imaging position of the captured image, and performing position estimation based on the color image captured in the moving environment A program for causing a mobile information processing apparatus to generate data to execute,
Learning a self-organizing map based on color information of each pixel of the plurality of captured images, and a plurality of clusters for classifying the color information by the self-organizing map after learning, and a color range of the cluster A first step defining:
A second step of identifying the cluster into which color information of the captured image is classified for each captured image, and generating the landmark data that associates the identified cluster, the captured image, and the position information; For causing the mobile information processing apparatus to execute the program.   Based on the cluster and the color range, the cluster in which color information of each pixel of the captured image for position estimation is classified is identified, and the position estimation is performed according to the identified cluster and the landmark data Calculating a degree of similarity of the captured image to be learned with respect to the captured image, and performing a process relating to the position estimation based on the similarity and position information associated with the captured image to be learned The program according to claim 14, comprising:

Images