JP2011085527A - Time series learning system, method thereof, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a time series learning system capable of achieving a local navigation system that discriminates one official route from among a plurality of routes by storing a plurality of routes while suppressing a required memory amount to be minimum. <P>SOLUTION: The time series learning system learns the time series information of imaging information that is taken from a moving body, in which the imaging information is input and then a landmark is determined based on the input imaging information. The landmark time series, which is specified in the order of appearance of landmarks that are determined, is stored as the route information of a moving body. If part of or all of a plurality of landmark time series overlap each other, the overlapped landmarks are collectively taken as a single route information. If part of or all the plurality of landmark time series overlap each other, the information of past series is stored as well. At recalling, a single route information is determined and output from the plurality of landmark time series, based on the information about past series. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a time series learning device for learning a time series, and more particularly to a time series learning device using a hippocampal-olfactory cortex loop circuit.

  Humans can reach their destinations through experience even if the environment changes slightly. In recent years, place cells have been found in the hippocampus and entorhinal cortex, and the mechanism of theta phase coding of place series has been elucidated (see Non-Patent Documents 1 and 2). As one of the time series learning models in the brain, a network model of the second layer of the olfactory cortex having a hippocampus-olfactory cortex loop circuit has been proposed (see Non-Patent Document 3). Loop junctions between astrocytes in ECII are selectively enhanced by afferent signals applied to ECII, and astrocytes connected by successively enhanced loop junctions fire well in each theta cycle .

  The inventors have proposed a local navigation system inspired by the time series learning mechanism in the hippocampus-olfactory cortex loop circuit, implemented it in an autonomous mobile robot, and confirmed the validity of the proposed method (Non-Patent Documents 4 to 4). 6).

  In addition, in the case of using the past time series, contents suggesting that past individual series codes play an important role when deciding which animal should go are disclosed ( (Refer nonpatent literature 7). Further, a computer model of past time series learning in the hippocampus has been proposed, and it is disclosed that a neuron having learned the order of signals is generated in the hippocampus CA1 using propagation of neural activity in CA3. (See Non-Patent Document 8).

J.O'Keefe and M.L.Recce, "Phase relationship between hippocampal place units and the EEG theta rhythm" Hippocampus, 3, pp.317-330, 1993 T.Hafting, M.Fyhn, T.Bonnevie, M-B.Moser and E.Moser, "Hippocampus-independent phease precession in entorhinal grid cells" Nature, 453, pp.1248-1252, 2008 J. Igarashi, H. Hayashi and K. Tateno, "Theta phase coding in a network model of the entorhinal cortex layer II with entrohinal-hippocampal loop connections" Cognitive Neurodynamics, 1, pp.169-184, 2007 T.Miki, et al., "Practical Local Navigation System Based on Entorhino-hippocampal Functions" Proc.of SCIS & ISIS 2008, pp.1783-1787, 2008 T.Miki, T.Morie, H.Liang, Y.Suzuki, K.Nakada, H.Hayashi, "Dedicated systems inspired by hippocampal memory mechanisms: Moving object detection and human-like local navigation" The 4th International Conference on Brain- inspired Information Technology (BrainIT2007), p.30, Kitakyushu (2007.11) Tsutomu Miki (KIT, Japan) et al., "[BRD08] Human-like local navigation inspired by hippocampal memory mechanism and its demonstration using an autonomous mobile robot" The 4th International Conference on Brain-inspired Information Technology (BrainIT2007), Kitakyushu ( (2007.11) L.M.Frank, E.N.Brown and M.Wilson, "Trajectory Encoring in the Hipppcampus and Entorhinal Cortex" Neuron, Vol.27, pp.169-178, 2000 M. Yoshida and H. Hayashi, "Emergence of sequence sensitivity in a hippocampal CA3-CA1 model" Neural Networks, 20, pp.653-667, 2007

However, the local navigation system proposed by the inventors is based on the time series learning mechanism in the hippocampus-olfactory cortex and uses the GPS and map information as a hint, and the route according to the order of appearance of landmarks only in the local environment. Although it offers an unprecedented navigation that can be memorized, it can handle only one route, and even if it is the same route as the already learned route, it is time-series by a new neuron Therefore, there is a problem that a necessary memory capacity increases.
In addition, it is necessary to determine information that can be a landmark in advance, which causes a problem that work is troublesome.

  Therefore, the present invention is a hippocampus-olfactory cortex loop circuit capable of realizing a local navigation system that stores a plurality of routes while minimizing a necessary memory capacity and identifies one formal route from the plurality of routes. An object is to provide a time-series learning device that is conceived from the above.

  (1) A time-series learning device disclosed in the present application is a time-series learning device that learns time-series information of imaging information captured from a moving body, and includes imaging information input means that inputs the imaging information, and the imaging Landmark determination means for determining a landmark from imaging information input by the information input means, and a landmark time series specified by the appearance order of the landmarks determined by the landmark determination means are used as the route information of the mobile object. When a part or all of the plurality of landmark time series overlaps, the overlapping landmarks are collectively stored as one storage information and stored as multiple path information including the one storage information. And a time-series storage unit.

  As described above, in the time series learning device disclosed in the present application, landmarks are determined from information captured from a moving body, and landmark time series specified by the determined order of appearance of landmarks are stored as route information. In this case, a plurality of landmark time series overlapping a part or all of them are combined into one storage information, and multiple path information is stored by the storage information. There is an effect that it is possible to store a plurality of multiplexed path information while suppressing the storage capacity for storing.

  (2) The time-series learning device disclosed in the present application stores past information storage that stores information about a past route at the time when the time-series storage unit stores a landmark immediately after duplication in a landmark time series that partially overlaps. The time-series storage means stores information on the basis of the information on the means, the landmark determined by the landmark determination means, the path information stored in the time-series storage means, and information on the past path stored in the past information storage means. Route information determining means for determining one piece of route information from multiple route information.

  As described above, in the time-series learning device disclosed in the present application, a plurality of multiplexed information is obtained based on the information about the past path at the time of storing the landmark immediately after overlapping in the partially overlapping landmark time series. Since one route information is selected from the route information, there is an effect that even if there are a plurality of choices in the route information, it is possible to accurately determine the one route information and prompt the user's action. .

  (3) In the time-series learning device disclosed in the present application, the candidate information detection unit that detects candidate information that is a landmark candidate from the imaging information input by the imaging information input unit, and the candidate information detection unit includes the candidate information. History information calculation means for calculating the history of the candidate information at the time of detecting the candidate information, and candidate information determination means for determining whether the candidate information is a fixed object based on the result calculated by the history calculation means The landmark determining means determines the candidate information as a landmark when the candidate information determining means determines that the candidate information is a fixed object.

  As described above, in the time-series learning device disclosed in the present application, it is determined whether or not candidate information that is a landmark candidate is a fixed object. If the candidate information is a fixed object, the landmark information is determined in advance. Landmarks can be acquired without performing an operation for registering possible information, and the operation can be made more efficient.

  (4) In the time-series learning device disclosed in the present application, the route information storage unit stores the route information based on connection information between neurons corresponding to each landmark determined by the landmark determination unit. It is characterized by doing.

  As described above, in the time-series learning device disclosed in the present application, since the path information is stored based on the connection information between neurons corresponding to each landmark, the landmarks linked based on one landmark are time-series. And the route information can be stored accurately.

  (5) In the time-series learning device disclosed in the present application, each neuron can output two output signals in the row direction and the column direction, and the output signals intersect with each other in a matrix shape. When a landmark is input, the output signal in the column direction of the neuron corresponding to the arbitrary one landmark is turned on, and when any other landmark is input, the arbitrary other land The output signal in the row direction of the neuron corresponding to the mark is turned on, and the intersection of the output signal in the row direction and the output signal in the column direction is the connection information between the one arbitrary neuron and any other neuron. The landmark time series is stored according to the combination information.

  In this way, in the time series learning device disclosed in the present application, the landmark time series can be accurately obtained with a simple configuration by using the connection information between neurons as the intersection of two matrix output signals output from each neuron. In addition, it is possible to read out easily and accurately by tracing the neuron and the output signal. Note that the rows and columns may be reversed.

  (6) In the time series learning device disclosed in the present application, when some of the landmark time series overlap, one output signal in the column direction forms combined information with a plurality of output signals in the row direction. The overlapping landmarks are collectively stored as one piece of storage information and stored as multiple path information including the one piece of storage information.

  As described above, in the time series learning device disclosed in the present application, when some landmark time series overlap, one output signal in the column direction forms combined information with a plurality of output signals in the row direction. Thus, since the multiplexed route information is stored, a plurality of route information can be stored while suppressing a storage capacity for storing overlapping landmarks.

  (7) The time series learning device disclosed in the present application uses, as past information, the connection information between the neurons at the time when the time series storage unit stores a landmark immediately after duplication in a landmark time series that partially overlaps. Past information storage means for storing, wherein the path information determination means forms one output signal in the column direction and a plurality of output signals in the row direction based on the past information stored in the past information storage means. One piece of combined information is determined from a plurality of pieces of combined information.

  As described above, in the time series learning device disclosed in the present application, based on the connection information between the neurons at the time of storing the landmark immediately after the overlap in the partially overlapped landmark time series, a plurality of pieces of connection information are obtained. Since one combination information is determined, even when there are a plurality of choices, it is possible to accurately select and determine one formal combination information, and it is possible to promote the user's action. .

  (8) The time-series learning device disclosed in the present application includes a landmark certainty factor indicating the accuracy of the connection in the connection information between the neurons, and the landmark included in the route information stored in the route information storage unit, When a different landmark is input, the landmark certainty factor is weakened, and when the same landmark as the landmark included in the route information stored in the route information storage means is input, the landmark The certainty factor is strengthened within a range equal to or less than a preset maximum value, and the route information determination unit selects route information that maximizes the landmark certainty factor as formal route information. .

  As described above, in the time-series learning device disclosed in the present application, the landmark certainty factor indicating the accuracy of the connection information between neurons is calculated, and the route information is selected based on the calculated landmark certainty factor. In addition, it is possible to engineeringly realize a sophisticated process when a human determines a route and to determine an accurate route.

  (9) The time series learning device disclosed in the present application compares the landmark time series included in the route information with the input landmark time series, and is included in the route information based on the number of matching landmarks. The route certainty indicating the accuracy of the landmark time series to be calculated is calculated, and based on the calculated route certainty, it is determined whether or not a landmark on the route indicated by the route information is missing and / or added. It is characterized by this.

  As described above, in the time series learning device disclosed in the present application, the route certainty factor is calculated from the number of landmarks input in the landmark time series, and the missing and / or added landmarks on the route are calculated. In order to determine whether or not there is a problem, it is possible to accurately recognize changes in landmarks based on differences from landmark time series stored previously, and to perform learning that flexibly responds to changes in the environment. Play.

  (10) In the time-series learning device disclosed in the present application, the connection information between the neurons at the time when the time-series storage unit stores a landmark immediately after duplication in a partially overlapping landmark time-series is used as past information. Based on past information storage means to be stored, missing landmarks determined based on the route certainty factor, and / or additional information, the difference between the past information storage means and the column direction is corrected. The present invention is characterized by comprising path information determining means for determining one combination information from a plurality of combination information formed by one output signal and a plurality of output signals in the row direction.

    As described above, in the time-series learning device disclosed in the present application, the difference from the past information storage unit is corrected based on the lack of the landmark determined based on the path certainty factor and / or the additional information. In order to determine one combination information from a plurality of pieces of combination information, even when one piece of route information is selected from a plurality of pieces of route information based on past information, the landmark time series is corrected to correct the information. There is an effect that a route can be selected.

(11) The time-series learning device disclosed in the present application includes an intersection determination unit that determines whether or not the mobile body enters an intersection based on a change mode of the route certainty factor. is there.
As described above, the time series learning device disclosed in the present application accurately detects that the moving object has entered the intersection by capturing the landmark time series characteristics when approaching the intersection from the change in route certainty. There is an effect that can be done. For example, when the route certainty factor gradually decreases, it can be said that the direction is changed earlier and the approach to the intersection is suggested.

(12) In the time-series learning device disclosed in the present application, when the intersection determination unit determines that the moving body has entered the intersection, the direction of the route is determined based on the magnitude of the route certainty. It is characterized by.
Thus, in the time-series learning device disclosed in the present application, the direction of the route at the intersection is determined based on the magnitude of the certainty of the route, so the route is surely presented without losing sight of the route at the intersection. There is an effect that can be done. For example, the certainty factor can be calculated for directions other than the direction that came within the intersection, and the direction with the highest certainty factor can be determined as the direction of the route to be followed.

(13) The time-series learning device disclosed in the present application captures the imaging information with an omnidirectional camera, and the path information determination unit returns the moving object based on the imaging information captured with the omnidirectional camera. The route information is stored.
As described above, in the time-series learning device disclosed in the present application, in order to store the route information of the return path of the mobile body based on the imaging information captured by the omnidirectional camera, the route information of the return path is used to The route information can be stored, and the return route information can be stored without increasing the storage capacity.

(14) The time series learning device disclosed in the present application displays the route information determined by the route information determination unit with the landmark according to the appearance order of the landmark indicated by the landmark time series, and / or audio. It is characterized by comprising output control means for outputting at.
As described above, in the time-series learning device disclosed in the present application, the route information is displayed in landmarks and / or output in voice according to the appearance order of landmarks indicated by the landmark time series. There is no need for cooperation with map information or GPS that is generally used, and the route information can be shown only in the local environment. In addition, it is not only necessary to rewrite the map information, but conversely, by cooperating with the map information, it is possible to recognize the possibility of an error in the map information. The learning device has a great effect.

  (15) A time-series learning device disclosed in the present application is a time-series learning device that learns time-series information of imaging information captured from a moving body, the landmark input unit that inputs landmark information, Time series storage means for storing landmark time series specified by the appearance order of landmarks as route information, landmark time series stored by the time series storage means, and landmarks input by the landmark input means Route information determination means for determining route information based on the information of the information, wherein the landmark time series includes external memory information, code reading information, voice input information, text information, image information, video information, and / or Or it is memorize | stored as route information by the input of map information.

  Thus, in the time series learning device disclosed in the present application, the landmark time series based on the external memory information, the code reading information, the voice input information, the text information, the image information, the moving picture information, and / or the map information is used. Since the route information is stored, for example, the route information learned by another time-series learning device can be input as the external memory information to easily increase the memory. In addition, route information from a reading point to a destination can be stored by reading information of a code such as a bar code or a QR code, for example, and it is possible to guide to the destination without route guidance.

  Although the present invention has been shown as an apparatus so far, as will be apparent to those skilled in the art, the present invention can also be understood as a method and a program. These outlines of the invention do not enumerate the features essential to the present invention, and a sub-combination of these features can also be an invention.

It is a figure which shows the mechanism of a hippocampus-olfactory cortex loop circuit. It is a hardware block diagram of the navigation apparatus containing the time series learning apparatus which concerns on 1st Embodiment. It is a functional block diagram of the time series learning device concerning a 1st embodiment. It is a figure which shows the function of the data register part in the time series learning apparatus which concerns on 1st Embodiment. It is a figure which shows the function of the comparison part in the time series learning apparatus which concerns on 1st Embodiment. It is a 1st figure which shows the function of the route memory part in the time series learning apparatus which concerns on 1st Embodiment. It is a 2nd figure which shows the function of the route memory part in the time series learning apparatus which concerns on 1st Embodiment. It is a figure which shows the switch control function which performs the selection of a path | route in the time series learning apparatus which concerns on 1st Embodiment. It is a flowchart which shows the operation | movement of the memorization process in the time series learning apparatus which concerns on 1st Embodiment. It is a flowchart which shows the operation | movement of the recall process in the time series learning apparatus which concerns on 1st Embodiment. It is a 1st figure which shows the process at the time of using landmark reliability in the time series learning apparatus which concerns on 2nd Embodiment. It is a 2nd figure which shows the process at the time of using landmark reliability in the time series learning apparatus which concerns on 2nd Embodiment. It is a figure which shows the process of the intersection at the time of using route reliability in the time series learning apparatus which concerns on 3rd Embodiment. It is a figure which shows the addition process of the landmark which used the route reliability in the time series learning apparatus which concerns on 3rd Embodiment. It is a figure which shows the deletion process of the landmark which used the route reliability in the time series learning apparatus which concerns on 3rd Embodiment.

  Embodiments of the present invention will be described below. The present invention can be implemented in many different forms. Therefore, the present invention should not be construed based only on the description of this embodiment. Also, the same reference numerals are given to the same elements throughout the present embodiment.

  In the following embodiments, the apparatus will be mainly described. However, as is apparent to those skilled in the art, the present invention can also be implemented as a method and a program for operating a computer. In addition, the present invention can be implemented in hardware, software, or hardware and software embodiments. The program can be recorded on any computer-readable medium such as a hard disk, CD-ROM, DVD-ROM, optical storage device, or magnetic storage device. Furthermore, the program can be recorded on another computer via a network.

(First embodiment of the present invention)
A time-series learning apparatus according to the present embodiment will be described with reference to FIGS. The time series learning device according to the present embodiment uses prediction and past landmark sequences in order to realize sophisticated navigation like a human being. Past landmark sequences allow memorization and recall of multiple overlapping routes. The time series learning device is designed based on standard digital technology, and the phase coding register handles both prediction (future) and past landmark sequences.

  First, the hippocampus-olfactory cortex loop circuit that has been a hint of the time-series learning device according to the present embodiment will be described. FIG. 1 is a diagram showing the mechanism of the hippocampal-olfactory cortex loop circuit. In an ECII network model with hippocampal-olfactory cortex loop coupling, pairs of afferent pulse trains consisting of strictly different frequencies are selected by loop coupling, and the loop coupling is selectively enhanced by afferent signal pairs. Phase coding with loop-coupled signal propagation delays also creates other places.

  Here, an arrangement of places A, B, C, D, and E is assumed. It is assumed that the signal for each place is represented by a frequency depending on the distance between the observer and the place. When the distance is short, the frequency becomes high. A higher frequency signal causes the cell membrane potential of the place cell to exceed the threshold in a shorter time. If A, B, and C are in an observable range, those signals are provided to ECII. In FIG. 1 (A), place cell A is first ignited by a signal at place A, and that signal is transmitted from the place cell A of ECII to other place cells via the hippocampus-olfactory cortex loop circuit.

  At this time, if the firing of the place cell B occurs simultaneously by the arrival of the propagated signal and the centripetal signal of the place B, a loop connection between the place cells A and B is established. Thereafter, the signal of the place cell B is transmitted to another cell, and the loop connection between the place cells B and C is enhanced by the synchronization of the spike transmitted and the firing of the place cell C. As a result, the order of places is embedded in the loop coupling load between place cells, and a spike train of place cells is formed in the theta cycle. The phase precession of the phase-coded spike train occurs as the animal moves.

  FIG. 1B is a view when viewed with an oscilloscope, and it appears that a plurality of pulses appear on the theta wave. The position of the pulse changes according to the strength of the landmark signal (corresponding to the phase coding). Since the landmark signal becomes stronger as the moving body moves, each pulse signal moves in the forward direction while maintaining the positional relationship (phase information) (referred to as phase precession).

  In the case of animals such as rats and mice, it is about 30 ms for the signal at location A to propagate from ECII to DG, CA3, CA1, and ECV, where it is combined with the signal at location B observed there. Conveniently no landmarks appear every 30ms. For example, it seems that some of the landmarks that human beings are aware of are memorized, and the detailed mechanism has not been elucidated. Therefore, in the present application, it is simplified as Hebb learning (Hebb learning) in which the coupling is strengthened if a signal is received at the same time.

  Next, the configuration of the time series learning device according to the present embodiment will be described. FIG. 2 is a hardware configuration diagram of the navigation device including the time-series learning device according to the present embodiment. In FIG. 2, the navigation apparatus 10 includes a CPU 21, a RAM 22, a ROM 23, an HD 24, a communication I / F 25, and an input / output I / F 26, which are connected by a bus.

  The CPU 21 operates based on a program stored in the ROM 23 storing a boot program executed when the computer is started up, a program depending on hardware, or the like stored in the HD 24 storing other programs via the RAM 22. Control each part. The communication I / F 25 receives information from another device via a communication line, sends it to the CPU 21, and transmits information generated by the CPU 21 to another device via the communication line. The CPU 21 controls the input of information from the input device and the output of information to the output device via the input / output I / F.

  FIG. 3 is a functional block diagram of the time-series learning device according to the present embodiment. The time series learning apparatus 20 includes a landmark candidate detection unit 31, a landmark code generation unit 32, a data register unit 33, a comparison unit 34, a route memory unit 35, a past information flag generator 36, and an output control unit 37.

  The landmark candidate detection unit 31 detects a candidate that can be a landmark target in imaging information (here, image information) captured by a camera from a moving body. More specifically, landmark candidates are detected based on differences in size, color, and background of the imaging body captured in the image.

  The landmark code generation unit 32 generates and assigns a unique code to each candidate information of the detected landmark. The data register unit 33 has a function based on the hippocampal-olfactory cortex phase coding mechanism, and handles both the prediction series and the past series. Also, a landmark candidate that is officially recognized as a landmark is determined.

  Here, the function of the data register unit 33 will be described in detail. FIG. 4 is a diagram illustrating the function of the data register unit in the time-series learning device according to the present embodiment. In FIG. 4, the data register unit 33 mainly includes a main register and a sub register, and each register includes, for example, five registers. The number of registers represents the number of landmarks and can be freely customized according to the use environment.

  In the main register, all landmark candidates within the observation range are recorded from the left end in order of their strength. As described above, the intensity is expressed in frequency, and landmark candidates having a high frequency are recorded in the leftmost register, and landmarks having a low frequency are recorded in the rightmost register. In other words, since there is a proportional relationship between the frequency and the distance, the landmark candidates located at a distance close to the moving body are sequentially recorded at the left end. Here, it is assumed that there are continuous landmarks, but in reality, there may be no landmark candidates of corresponding strength, in which case it is blank. Also, if landmark candidates of exactly the same strength are recorded at the same time, they are excluded.

  In FIG. 4, the upper register is the newer in time and the current information is recorded in the upper register. As the moving body moves, an MV signal (a signal indicating that it has moved) is input, whereby each piece of information recorded in the main register is first transferred to the sub-register and then shifted to the left. Between sub-registers, each piece of information is transferred as it is to the next sub-register at the same timing. Landmark candidates are detected at a certain interval, and when the landmark candidates are fixed (in a stationary state), the landmark candidates are continuously shifted to the left along with the movement of the moving object and observed. Is done. For example, a conspicuous shape such as an automobile is detected as a landmark candidate, but since it is a mobile object, it is excluded from the register with time and does not continuously appear in the sub-register. That is, landmark candidates that appear continuously on the diagonal lines in the register (here, E landmark candidates) are determined as official landmarks.

  As can be seen from FIG. 4, the past series data appears in the leftmost column, and the predicted series data appears in the main register. The past series of data is used for selection when route information is multiplexed and there is route selectivity. In other words, a plurality of routes are separated based on information on where they come from. Details of specific processing will be described later for this past series of data. Further, the data determined as the landmark is input to the route memory unit 35.

Returning to FIG. 3, the comparison unit 34 compares the route information stored in the route memory unit 35 with the landmark information input from the data register unit 33, and stores the same in the route memory unit 35 if they match. Then, information (r _ij = 0) indicating that the route does not have selectivity (a state where a common route among a plurality of routes is multiplexed and a branch is generated) is set. On the other hand, if they do not match, the past information flag generator 36 is prompted to generate a past information flag, and information (r _ij = 1) indicating that the route has selectivity is set.

Here, the function of the comparison unit 34 will be described in detail. FIG. 5 is a diagram illustrating the function of the comparison unit in the time-series learning device according to the present embodiment. In FIG. 5, the lower row is older in time, and the upper row is new information in time. Here, a case where the moving body follows a route of A → B → C → D → E → a → B → C → D → e is shown. Since the landmark a is a new landmark and no branch has occurred, it is stored as it is as new route information in the route memory unit 35 (because the comparison target is null, it is treated as a wild card and judged to match. ) The landmark B → C → D that appears for the second time coincides with the already stored route information, and is therefore stored as is in the route memory unit 35 as route information. At this time, the memory may be overwritten or the memorizing process may not be performed. In the processing so far, r _ij is set to 0.

Next, when the landmark of e appears, it can be determined that the moving object is moving along a new route by branching because it does not match the stored information, and r _ij is set to 1. Generation of a past information flag at that time is prompted. The past information flag is generated by the past information flag generator 36 with the past series data of a → B → C → D appearing at the left end as an address.

  Returning to FIG. 3, when the past information flag generator 36 determines that the moving body is moving on a new route by branching according to the comparison result in the comparison unit 34, the past information flag generator 36 is based on the past information sequence. A flag (ret (i)) is generated. The past information flag is assigned by the following formula.

ｒｅｔ（ｉ）はフラグｉ番目のビットを表し、経路情報を想起する際に用いられる。

ret (i) represents a flag i-th bit and is used when recalling route information.

  The route memory unit 35 is a memory that stores the route information based on the landmark information input from the data register unit 33 and the result of the comparison process performed by the comparison unit 34. Here, the function of the route memory unit 35 will be described in detail. FIG. 6 is a first diagram illustrating the function of the route memory unit in the time-series learning device according to the present embodiment.

In the inscription process, when the landmark information “A” is input in FIG. 6A, the neuron 61 corresponding to the landmark “A” is fired by the selector (not shown), and the output line u _i. Is activated. This continues until the next landmark information is entered. Next, when the landmark information “B” is input to the corresponding neuron 62, the program signal prg _j is generated, and the connection information _wij at the intersection of these two lines is set to generate connection information. When the connection weight w _ij satisfies the condition, an activation signal is input from there toward the neuron 62, and the output line of u _j is activated. Thereafter, the same processing is repeated, and the landmark time series is stored as grid points indicated by black dots in FIG. FIG. 6B shows a state in which the landmarks A, B, C, D, and E are input in this order and stored in chronological order in that order.

For the inscription process, an invocation process for recalling route information based on the inscribed information is performed. In this case, in FIG. 6B, when the landmark “A” is input, the output line u _i extending from the neuron 61 is activated. Since lattice points are formed in the inscription process, landmarks are recalled in the order in which they are recorded (indicated by bold lines in FIG. 6B).

  Here, the connection load is defined. The combined load shall be assigned by the following formula.

ｕ_i、ｐｒｇ_jとｗ_ijは、それぞれｉ−ｔｈ細胞、ｊ−ｔｈ細胞のプログラム信号と（ｉ，ｊ）の結合荷重である。

u _i , prg _j and w _ij are the combined loads of the program signals of (i, j) and (i, j), respectively.

  Next, the function of the route memory unit 35 in the case of memorizing and recalling multiplexed route information when part of a plurality of routes overlaps will be described in detail. FIG. 7 is a second diagram illustrating the function of the route memory unit in the time-series learning device according to the present embodiment. Here, route 1 (A → B → C → D → E) and route 2 (a → B → C → D → e) are assumed. These paths have partially overlapping sequences B → C → D.

First, as shown in FIG. 6, the route information is recorded by giving landmark information from “A” to “E” to the route memory unit 35, and then the route 2 is shown as shown in FIG. Memorize. When “e” is to be recorded after “D” in the path 2, lattice points 71 are already formed on the activation line. This means the above-described multiplexed route information having selectivity. Therefore, the past information flag is generated by the method described above. In other words, the past history at this time (here, a → B → C → D) is given to the past information flag generator 36, and the past information flag (ret ( i)) is assigned and stored. Further, for the lattice point 72, the coupling load w ₃₆ = 1 and the route selectivity r ₃₆ = 1 (with selectivity) are set.

When recall processing is performed on the route information recorded in FIG. 7A, as shown in FIG. 7B, on the activation line u _i extending from the neuron 73 corresponding to “a”, Two lattice points 71 and 72 are generated. In such a case, it is necessary to select one in the recall process, which causes a pause. In order to avoid this temporary stop, processing for selecting correct route information is performed using the past information flag generated by the past information flag generator 36. This selection process is controlled by the switch process shown in FIG.

FIG. 8 is a diagram illustrating a switch control function for selecting a route in the time-series learning device according to the present embodiment. As shown in FIG. 8, switch control is performed according to the table of FIG. 8B at the intersection of the line of the activation line u _i and the program signal prg _j from the neuron. That is, the value of sw _ij is determined by the combination of ret (i) and r _ij, and the lattice points are switched. In FIG. 7B, since sw _ij = 0 (switch OFF) at the lattice point 71 and sw _ij = 1 (switch ON) at the lattice point 72, the path information passing through the lattice point 72 here. That is, route information that passes through “e” instead of “E” is selected.

  In this example, one flag information can be selected from two route information by setting the flag code to 1 bit. However, by increasing the number of bits of the flag code, a plurality of route information (exponentially) ( For example, it is possible to select one route information from 4 multiplexed routes information for 4 bits and 16 multiplexed route information for 4 bits.

  Returning to FIG. 3, the output control unit 37 controls the output of route information to the user by video or audio. At this time, unlike a navigation system linked with map information that is generally used at present, only the order and direction of landmarks are output as video or audio. The above is the description of the configuration.

  Next, the operation of the time series learning apparatus according to the present embodiment will be described. FIG. 9 is a flowchart showing the operation of the memorizing process in the time series learning apparatus according to the present embodiment, and FIG. 10 is a flowchart showing the operation of the recall process in the time series learning apparatus according to the present embodiment.

  In the case of the memorizing process, in FIG. 9, first, a front image is picked up by the camera from the moving body (S91), and landmark candidates are detected from the picked-up image (S92). A unique code is generated and assigned to each landmark candidate information (S93). The landmark candidate history is calculated, and the landmark candidate determined as a fixed object is determined as an official landmark (S94). It is compared whether or not the determined landmark information matches the recalled route information stored in the memory (S95, S96). If they match, the time series of the determined landmark is selected as the selectivity. Is recorded in the memory as path information (S97), and the recording process at that time is terminated. If they do not match, a past information flag is generated from the past series data at that time and stored (S98), and the determined landmark time series is selective (a branch when some routes overlap). Is recorded in the memory as route information (S99), and the recording process at that time is terminated. Thereafter, the process is repeated as the moving body moves.

  In the case of the recall process, in FIG. 10, first, a front image is picked up by a camera from a moving body (S101), and landmark candidates are detected from the picked-up image (S102). A unique code is generated and assigned to each landmark candidate information (S103). The landmark candidate history is calculated, and the landmark candidate that is determined to be a fixed object is determined as an official landmark (S104). Up to this point, the processing is the same as in the case of FIG. Next, the route information recorded in the memory is detected based on the corresponding landmark information (S105). It is determined whether there is a branch in the detected route information (S106). If there is no branch, the route information is output as video or voice as the predicted route information as it is (S109), and the recall process is terminated. If there is a branch, the past information flag is referred to based on the past series data, and the past information flag at the time of recording is extracted (S107). Based on the extracted past information flag, switching control is performed to switch the route information (S108), and the route information after the switching is output as predicted route information in video or audio (S109), and the recall process is performed. finish.

  In addition, when imaging with a camera, it may be possible to record and recall the round-trip route information with one route information by using a camera capable of imaging all directions as well as the front. Also, by using an omnidirectional camera, the current position of the moving object is estimated based on the recorded route information when it is suddenly placed at a certain point (for example, when it goes out of the ground to the ground). It is also possible to do.

(Second embodiment of the present invention)
A time-series learning apparatus according to this embodiment will be described with reference to FIGS. 11 and 12. FIG. 11 is a first diagram illustrating processing when the landmark certainty factor is used in the time-series learning device according to the present embodiment, and FIG. 12 illustrates the landmark certainty factor in the time-series learning device according to the present embodiment. It is a 2nd figure which shows the process at the time of using. In the present embodiment, descriptions overlapping with those in the first embodiment are omitted.

  FIG. 11 shows the processing of the route memory unit 35 when the landmark “B” is missing when the route A → B → C → D → E is recorded. FIG. 11A shows a state in which routes A → B → C → D → E are recorded as route information in accordance with the processing described in the first embodiment. In this state, when the same route is followed, after the landmark “A” appears, the landmark “B” does not appear but the landmark “C” appears, and thereafter the landmarks “D” and “E” appear. Suppose it appears. Then, as shown in FIG. 11B, the path information forms a new path via the new grid point 111 between “A” and “C”. Instead of erasing, leave the bond weight reduced. If the landmark “B” appears in the remaining lattice point 112 when the same path is followed again, the coupling load is restored and the coupling load of the newly formed lattice point 111 is reduced. The magnitude of this combined load can be used for selecting route information as landmark confidence.

  At the time of the recall process, the lattice point having the maximum coupling load at that time is selected and recalled. That is, in the case of FIG. 11B, the lattice point 111 has a larger coupling load and higher landmark reliability than the lattice point 112, so the path A → C → D → E is selected.

  FIG. 12 shows a route memory when a new landmark “b” appears between the landmarks “A” and “B” when the route A → B → C → D → E is recorded. The process of the unit 35 is shown. FIG. 12A shows a state in which routes A → B → C → D → E are recorded as route information in accordance with the processing described in the first embodiment. When the same route is followed in this state, after the landmark “A” appears, the landmark “b” appears, and thereafter the landmarks “B”, “C”, “D”, and “E” appear. To do.

  Then, the route information includes a new route through the new lattice point 123 between “A” and “b” and a new lattice between “b” and “B” as shown in FIG. A new path through the point 122 is formed, but the original lattice point 121 at this time is not erased and is left in a state where the coupling load is reduced. If the landmark “B” appears next to the landmark “A” when the same lattice path 121 is followed again, the remaining lattice point 121 returns the bond load to the original lattice point. The joint load of 122 and 123 is reduced. Similar to the case of FIG. 11, the magnitude of this combined load can be used as a landmark certainty for selecting route information.

  In the recall process, as in the case of FIG. 11, the lattice point having the maximum combined load at that time is selected and recalled. In other words, in the case of FIG. 12B, the lattice point 123 has a larger coupling load and higher landmark confidence for the activation line “A” than the lattice point 121, so that the path A → b → B → C → D → E is selected.

  Thus, by giving weight to the combined load, it becomes possible to flexibly cope with an environmental change based on the landmark certainty factor. In FIG. 11 and FIG. 12, the increase / decrease in the coupling load is doubled and halved, but it can be freely set according to the usage environment of the user.

  At this time, as is clear from the above description, it is possible to recognize and store environmental changes such as missing or added landmarks based on changes in landmark certainty. That is, as described with reference to FIG. 7 in the first embodiment, when a plurality of pieces of route information are multiplexed, when switching control is performed based on the past information flag, the environmental change is taken into consideration. By performing the correction, it is possible to perform accurate matching with the past information flag.

  Specifically, when a landmark is missing, a wild card (null indicated by “*” in FIG. 5) is added to the corresponding location in the route information, and when a landmark is added, Correction can be performed by excluding the added landmark and performing matching with the past information flag when the next landmark is input. When such correction is performed, the past information flag may be updated when it can be determined that the addition and deletion of landmarks have been confirmed.

(Third embodiment of the present invention)
A time-series learning apparatus according to this embodiment will be described with reference to FIGS. FIG. 13 is a diagram illustrating intersection processing using the route certainty factor in the time-series learning device according to the present embodiment, and FIG. 14 illustrates landmarks using the route certainty factor in the time-series learning device according to the present embodiment. FIG. 15 is a diagram illustrating a landmark deletion process using a route certainty factor in the time-series learning device according to the present embodiment. In addition, in this embodiment, the description which overlaps with the said 1st Embodiment and the said 2nd Embodiment is abbreviate | omitted.

  In this embodiment, in order to determine a formal landmark from among landmark candidates at an earlier time, the cycle for receiving the MV signal is set short and the number of sub-registers is increased. When determining a landmark, it is determined whether it is a landmark that appears continuously in the increased sub-register or a landmark that appears only in a part, whether it is a fixed object or a moving object, and only a fixed object. As the official landmark. By doing so, a formal landmark can be determined in a short time.

  First, the process at the intersection using the route certainty will be described. In FIG. 13A, an arrow indicated by a bold line has already been routed as route information based on landmarks (black circles, black four strokes, black triangles, white circles, and five types indicated by stars) arranged along the arrows. It is recorded in the memory 35, and the moving body follows the path from the start to the goal. It is assumed that the order of landmarks on the route recalled at this time is the order ((1) to (13)) indicated by the numbers in FIG.

  FIG. 13B shows data in the registers (here, four registers are assumed) at the timing when each landmark is detected, and FIG. 13C is a graph showing the path certainty at each timing. It is shown by. The route certainty is indicated by the number of recorded data that matches the detected landmark. That is, at the time when the landmark (1) is detected, the memorized route ((1) → (2) → (3) → (4)) and the landmark detected at that time (before) To (1), (2), (3), (4)) all match, so the route certainty is 4. When the moving object detects the landmark (3), the memorized route and the detected landmark coincide with each other, so the route certainty is 3.

  When the mobile body enters the intersection 130, the route certainty gradually decreases, and finally decreases to 1, so that the intersection is recognized and the direction of the next route is determined. Here, the route certainty is calculated in the intersection 130 for each of the left and right directions, and the direction in which the route certainty is the maximum value 4 is determined as the formal route (left turn direction). Further, for a route in a direction (right turn direction) that is not an official route, the route certainty is set to 0. As can be seen from the graph of FIG. 13 (C), when there is a change of direction like an intersection, the route certainty gradually decreases and becomes 1 within the intersection. Detect and determine the next direction. Similarly, at the intersection 131, the right and left directions are determined when the route certainty is lowered to 1 (here, the right turn direction is determined to be the official route), and the route to the goal is recalled.

  Next, processing for adding and deleting landmarks using route certainty will be described. In FIG. 14A, the upper row shows the memorized route information, and the lower row shows the actually detected landmark. At this time, as shown in FIG. 14B, the landmark (4a) is added and detected, so that a deviation from the recorded route information occurs, and the route certainty decreases from 4 to 3. . In such a case, the next landmark (5) is compared with the recorded route information. When the landmark (5) matches, it is determined that the landmark (4a) is the added landmark, and the route information including the landmark (4a) is updated. After the update, since the route certainty is calculated based on the route information when the landmark (4a) is added, the value of the route certainty returns to 4.

  At this time, the update source route information may be deleted, or the landmark certainty factor may be updated as shown in the second embodiment. Further, the past information flag may be updated as necessary.

  FIG. 15 is a diagram in the case where the landmark is deleted. The upper part of FIG. 15A shows the route information recorded, and the lower part shows the actually detected landmark. At this time, since the landmark (9) is deleted as shown in FIG. 15B, a deviation from the recorded route information occurs, and the route certainty decreases from 4 to 3. In such a case, first, the same processing as in FIG. 14 is performed to determine whether or not a landmark is added. Here, since it is determined that the landmark is not added, it is next determined whether or not the landmark is to be deleted. In the determination of deletion, the next landmark (10) in the memorized route information is compared with the detected landmark. If the landmark (10) matches, it is determined that the landmark (9) is a deleted landmark, and the route information that does not include the landmark (10) is updated. After the update, since the route certainty is calculated based on the route information when the landmark (10) is deleted, the route certainty value returns to 4.

  At this time, the update source route information may be deleted, or the landmark certainty factor may be updated as shown in the second embodiment. Further, the past information flag may be updated as necessary.

  In FIGS. 14 and 15, the landmark deletion is determined after the landmark addition is determined. However, the landmark addition may be determined after the landmark deletion is determined. . However, in this case, since the determination process is performed with a plurality of landmarks, it is desirable to determine the deletion of the landmarks after determining the addition of the landmarks.

  In FIGS. 14 and 15, processing has been described in the case where there is only one change such as addition or deletion of a landmark. However, it is possible to cope with a case where there are continuous changes in a plurality of landmarks. is there. That is, if the number of landmark references is N and the number of landmark changes is r, the change in path certainty is Nr, N- (r-1), N- (r-2). , ... Even if the number of r is 2 or more by detecting this change, it is possible to cope by calculating the route certainty by referring to the detected landmark information or the landmark information recorded in advance. Can do.

  However, in this case, when r = N−1, there is a possibility that it cannot be distinguished from the intersection shown in FIG. 13. For example, in the case of an intersection approach, the route certainty is maximized by changing the direction. On the other hand, in the case of a landmark change, the route certainty factor does not reach the maximum value even if the direction is changed, so that the intersection approach and the landmark change may be distinguished. In the case of an intersection, the direction is changed so that the data in the main register is totally exchanged in the inscription process. You may enable it to recognize that it is an intersection at the time of a process.

  In each of the above-described embodiments, the route memory unit 35 obtains route information from an external memory (for example, an external storage medium such as a memory card or a CD-ROM), code reading information such as a barcode or QR code, and voice input information. , Text information, image information, video information, and / or map information can be stored based on the input. By doing so, it is possible to easily copy and add the memory, and it is possible to achieve reaching the destination in a state where the route is previously taught by another person, as in the case of a human being.

10 navigation device 20 time-series learning device 21 CPU
22 RAM
23 ROM
24 HD
25 Communication I / F
26 I / O I / F
31 Landmark Candidate Detection Unit 32 Landmark Code Generation Unit 33 Data Register Unit 34 Comparison Unit 35 Route Memory Unit 36 Past Information Flag Generator 37 Output Control Unit

Claims (17)

A time-series learning device for learning time-series information of imaging information captured from a moving body,
Imaging information input means for inputting the imaging information;
Landmark determination means for determining a landmark from imaging information input by the imaging information input means;
When a landmark time series specified by the appearance order of landmarks determined by the landmark determination means is stored as route information of the mobile body, and some or all of the plurality of landmark time series overlap And a time-series storage unit that stores the overlapping landmarks as one piece of storage information and stores them as multiple path information including the one storage information. The time series learning apparatus according to claim 1,
Past information storage means for storing information relating to past paths at the time when the time series storage means stores a landmark immediately after duplication in a landmark time series partially overlapping;
Based on the landmarks determined by the landmark determination means, the path information stored by the time series storage means, and information on the past paths stored by the past information storage means, the multiples stored by the time series storage means are stored. A time-series learning device comprising: route information determining means for determining one route information from route information. In the time series learning apparatus according to claim 1 or 2,
Candidate information detection means for detecting candidate information that is a landmark candidate from the imaging information input by the imaging information input means;
History information calculating means for calculating the history of the candidate information at the time when the candidate information detecting means detects candidate information;
Candidate information determination means for determining whether the candidate information is a fixed object based on the result calculated by the history calculation means,
A time-series learning device, wherein the landmark determining unit determines the candidate information as a landmark when the candidate information determining unit determines that the candidate information is a fixed object. In the time series learning apparatus in any one of Claims 1 thru | or 3,
The time-series learning device, wherein the route information storage unit stores the route information based on connection information between neurons corresponding to the landmarks determined by the landmark determination unit. The time-series learning device according to claim 4,
Each neuron can output two output signals in the row direction and the column direction, and the output signals intersect each other in a matrix,
When any one landmark is input, the output signal in the column direction of the neuron corresponding to the one arbitrary landmark is turned on,
When any other landmark is input, the output signal in the row direction of the neuron corresponding to the other other landmark is turned on,
The intersection of the row direction output signal and the column direction output signal is used as connection information between the arbitrary one neuron and any other neuron, and the landmark time series is stored by the connection information. Characteristic time-series learning device. The time-series learning device according to claim 5,
When some of the landmark time series overlap, one output signal in the column direction forms combined information with a plurality of output signals in the row direction, so that the overlapping landmarks are combined into one storage information. A time-series learning device characterized by storing as multiple path information including the one storage information. The time-series learning device according to claim 6,
Past information storage means for storing connection information between the neurons as past information at a time point when the time series storage means stores a landmark immediately after duplication in a landmark time series partially overlapping;
Based on past information stored by the past information storage means, path information determination for determining one combination information from a plurality of combination information formed by one output signal in the column direction and a plurality of output signals in the row direction. A time-series learning device. In the time series learning apparatus in any one of Claims 4 thru | or 7,
The landmark information indicating the accuracy of the connection is included in the connection information between the neurons,
When a landmark different from the landmark included in the route information stored by the route information storage unit is input, the landmark certainty factor is weakened,
When the same landmark as the landmark included in the route information stored in the route information storage means is input, the landmark certainty factor is strengthened within a preset maximum value range,
The time-series learning device, wherein the route information determination unit selects route information that maximizes the landmark certainty factor as formal route information. The time-series learning device according to any one of claims 4 to 8,
A route certainty factor that compares the landmark time series included in the route information with the input landmark time series and indicates the accuracy of the landmark time series included in the route information based on the number of matching landmarks. And determining whether or not a landmark is missing on the route indicated by the route information and / or whether or not it is added based on the calculated route certainty factor. The time-series learning device according to claim 9,
Past information storage means for storing connection information between the neurons as past information at a time point when the time series storage means stores a landmark immediately after duplication in a landmark time series partially overlapping;
Based on missing landmarks determined based on the path certainty factor and / or additional information, the difference from the past information storage means is corrected to correct the output signal in the column direction and the row direction. A time-series learning device comprising: path information determination means for determining one combination information from a plurality of combination information formed by a plurality of output signals. The time-series learning device according to claim 9 or 10,
A time-series learning device comprising: an intersection determination unit that determines whether or not the mobile body enters an intersection based on a change mode of the route certainty factor. The time-series learning device according to claim 11,
A time-series learning device, wherein a direction of a route is determined based on the magnitude of the route certainty when the intersection determination unit determines that the moving body has entered an intersection. The time-series learning device according to any one of claims 1 to 12,
Imaging the imaging information with an omnidirectional camera,
The time-series learning device characterized in that the route information determination unit stores route information of a return path of the moving body based on imaging information captured by the omnidirectional camera. The time-series learning device according to any one of claims 2 to 13,
Characterized in that it comprises output control means for displaying the route information determined by the route information determination means in accordance with the appearance order of the landmarks indicated by a landmark time series and / or outputting the information in audio. A time series learning device. A time-series learning device for learning time-series information of imaging information captured from a moving body,
Landmark input means for inputting landmark information;
Time-series storage means for storing landmark time series specified by the appearance order of the landmarks as route information;
A landmark time series stored by the time series storage means, and route information determination means for determining route information based on landmark information input by the landmark input means,
The landmark time series is stored as route information by inputting external memory information, code reading information, voice input information, text information, image information, moving picture information, and / or map information. Sequence learning device. A time series learning method in which a computer learns time series information of imaging information captured from a moving body,
An imaging information input step for inputting the imaging information;
A landmark determination step of determining a landmark from the imaging information input by the imaging information input step;
The landmark time series specified by the appearance order of the landmarks determined in the landmark determination step is stored as route information of the mobile body, and when a part or all of the landmark time series overlaps, A time-series learning method comprising: a time-series storage step of storing the duplicate landmarks together as one piece of storage information and storing it as route information including the one piece of storage information. A time-series learning program for causing a computer to function to learn time-series information of imaging information captured from a moving body,
Imaging information input means for inputting the imaging information;
Landmark determination means for determining a landmark from imaging information input by the imaging information input means;
The landmark time series specified by the appearance order of landmarks determined by the landmark determination means is stored as route information of the mobile body, and when a part or all of the landmark time series overlaps, A time-series learning program that causes a computer to function as time-series storage means for storing the overlapping landmarks together as one piece of storage information and storing it as route information including the one piece of storage information.

Images