JP2007287071A - System for controlling operation of group comprising multiple autonomous robots, supervising robot, searching robot, and display device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a system for controlling the operation of a group comprising a large number of autonomous robots, particularly a system for locating objects with different characters from the surroundings such as living bodies in environment of a disaster site or the like hard to examine carefully in a normal method. <P>SOLUTION: A large number of searching robots are arranged in the environment, and each of the robots autonomously performs appropriate control of its operation while exchanging information to locate objects with different characters from the surroundings, existing in the environment. Each searching robot takes a surrounding image with a camera 18 and extracts the feature of the image. A supervising robot collects the features to detect saliency and notifies each searching robot. The searching robot recognizes the position of the searching robot taking a salient image by a rotating antenna 20 and moves by a moving means 14. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

The present invention relates to a system for controlling the operation of a group of a large number of autonomous robots, and in particular, a group robot capable of searching for things having properties different from the surroundings such as living bodies in an environment that is difficult to scrutinize by ordinary methods such as disaster sites. Control and monitoring system.

  The idea of causing a large number of robots to perform rescue activities at a disaster site is described in, for example, Non-Patent Document 1, which proposes a communication method between robots.

  Moreover, the idea of patent document 2 etc. has already existed with respect to the subject of what kind of search activity is performed with many robots. Patent Document 2 discloses a technique of a group robot system that searches a wide area using robots having different roles such as a sensing robot, a pheromone robot, and a base station. Here, when the sensing robot finds an object and ends the search, the base station or the pheromone robot moves the sensing robot to the next search area.

  Further, a method for causing a large number of robots to work in cooperation is proposed in Non-Patent Document 2, in which a method of carrying an object in a relay manner is shown. What is characteristic in the method of Non-Patent Document 2 is that each robot has achieved a meaningful work of carrying an object as a whole even though each robot performs only a simple movement. Moreover, there exists a technique currently disclosed by patent document 1 as another prior art regarding this invention. This describes a method for determining where to focus attention on an image obtained by observation with a sensor.

JP 2001-236508 A JP 2004-338040 A Sugiyama Hisayoshi, Sasaoka Tetsuo, Murata Tadashi, Mobile Robot Action Algorithm for Network Formation in Disaster Sites, Proc. Edited by Koji Ito, Emergence of Knowledge, Chapter 2 Carrying Many Things

  However, for example, there are still problems to be solved in order to make a large number of robots search for victims in a disaster site. If you are looking for victims at the disaster site, you need to be careful not to destroy the rubble piles, and also check the depths of the small holes. Therefore, it is considered that one solution is to prepare many small robots and let them search for victims.

  From such a point of view, Non-Patent Document 1 shows a method for reliably performing communication between robots in such an environment, and this makes it possible to search for a victim while being controlled by a human by remote control, for example. It is expected. However, as many robots as possible can search for victims more quickly and reliably, but if the number of robots increases, there is a problem that it becomes very difficult to control all robots remotely.

  Therefore, it is desirable that a large number of robots can autonomously control their operations. In that sense, Non-Patent Document 2 shows one principle that enables autonomous operation of a large number of robots, but in practice it is still necessary to develop a specific method for each purpose. However, no method has been developed that allows many robots to find victims.

  On the other hand, in Patent Document 1, as a method for determining where to concentrate attention in an image, there is shown a method for finding an unfamiliar part or a part of an image that has a different property from the surroundings. It is impossible to make the robots search for victims.

  The present invention relates to a method for controlling the operation of a group of a large number of autonomous robots in order to solve these problems, and in particular, has a property different from that of surroundings such as a living body in an environment that is difficult to scrutinize by a normal method such as a disaster site. The purpose is to provide a method to find things with

  For the above purpose, the present invention has at least one supervisory robot, three or more search robots, and a display device for displaying position information of each robot, and each search robot has its own surroundings. When the saliency value is received from the supervisory robot, a saliency signal is transmitted, and a saliency signal transmitted by a search robot other than itself is received. And the supervisory robot receives the image information and the position information from each search robot and obtains a saliency value from the set of received image information. A group motion control system comprising a plurality of robots that transmit saliency values to the search robot that has transmitted image information corresponding to the obtained saliency values is provided.

  The search robot of the present invention includes a camera that captures a surrounding image, a moving device that moves its own casing, a signal transmitter that transmits a signal, a signal receiver that receives a signal, and the camera. An image processing unit for processing captured image information, an image transmission processing unit for transmitting data processed by the image processing unit by the transmitter, and a saliency signal when receiving a saliency value addressed to itself And a movement determination unit that receives a saliency signal transmitted from another robot and determines a movement direction based on the saliency signal.

  The display device of the present invention includes a display unit, a receiver that receives a signal, a transmitter that transmits a signal, a position display processing unit that displays position information of a plurality of robots on the display unit, and at least one An instruction transmission unit that issues an instruction to a robot of more than one unit and an image display processing unit that receives image information from a specific unit and displays the image information on the display unit.

  The system for controlling the operation of a group of a large number of autonomous robots according to the present invention makes it possible to search for things having different properties from the surroundings, such as a living body, in an environment that is difficult to scrutinize by ordinary methods such as disaster sites.

  First, an overview of an embodiment of the present invention will be described with reference to the drawings. The robot group of the present invention includes a large number of search robots and at least one supervisory robot. In a disaster site or the like, a large number of robots are randomly arranged in an area to be searched. FIG. 1A shows a situation where search robots are randomly arranged. A black circle (9) represents the search robot. The boundary (8) indicates a search area, but the search behavior of the search robot is not particularly limited to this area.

  Each search robot has a camera and captures the surrounding situation as an electronic image. The captured electronic image is subjected to compression processing and the like in each search robot, and after being made communicable, is transmitted to the supervisory robot.

  There is at least one supervisory robot among many robots. The supervisory robot accumulates the image data transmitted by the search robot and extracts the saliency from the set of image data. The saliency is an index that increases when there is an object different from the surroundings in the image data viewed by many robots.

  For example, assume that ten search robots have acquired images in the forest. Assume that only one of them has obtained a portrait placed in the forest as image data. Since nine robots acquire the scenery of the forest as an image, the image data of this portrait is clearly different in the surroundings of the forest. Therefore, such an image shows a high saliency. As will be described in detail later, the supervisory robot extracts an object with high saliency from the image data sent from the search robot.

  The supervisory robot assigns a value to the extracted saliency and obtains the saliency value. In other words, order the prominent objects. Then, the saliency value is notified to the search robot that copied each object.

  The search robot notified of the saliency value transmits a saliency signal corresponding to the saliency value. For example, when a saliency value is notified to each robot for the three images that are considered to be the most prominent among many robots, only the three robots that have been notified among the many robots are saliency. Send a signal.

  Many robots receive saliency signals transmitted by other robots and know the direction of radio waves and the strength of saliency. Then, the moving direction is determined based on the received saliency signal, and the vehicle moves a certain distance. After moving a certain distance, the surrounding image is acquired again and transmitted to the supervisory robot.

  FIG. 1B shows the position of the robot after moving a certain distance. Although it is a little, you can see that each robot is gathering. If this cycle is continued, the robots gather around the robots who have seen objects different from the surrounding scenery. FIG. 1C shows a state where robots are gathered at three locations. If the positions of the robots that perform such operations are displayed on the display device, it can be understood that the search robots have found something in the search area.

  As described above, according to the present invention, the search robot itself cannot determine whether or not the acquired image is remarkable. It is not until the saliency signal notified from the supervising robot knows that the object he saw is far from the surrounding landscape. Then, the individual robots gather the search robots that have seen the prominent objects, and show the search results.

  Moreover, in this invention, the instruction | indication of what is searched for in a search area is not given from anywhere. Therefore, I do not know what I found as a result of the search. However, each search robot has a relatively simple function of acquiring an image of its surroundings, sending it to the supervisory robot, and moving so as to gather in the saliency signal, and has a search function as a group. Has characteristics. Hereinafter, the search robot, the supervisory robot, and the display device will be described in detail.

Search Robot FIG. 2 illustrates an overview of the search robot. The search robot (10) has wheels (14), a position measuring device (16), a camera (18), and transceivers (20, 22) mounted on a main body (12). The main body (12) is equipped with a drive device and a control device for the wheels (14) as well as a battery.

  The wheel (14) is a moving means, but is not limited to a wheel, and may be a flying means such as a caterpillar, an air float, or a helicopter. Although the wheel (14) is depicted as being fixed, it goes without saying that the direction of travel can be changed.

  Here, the position measuring device (16) is equipped with a GPS (Global Positioning System). However, the present invention is not limited to this, and after the initial position is recorded, the position may be grasped by performing inertial movement using an accelerometer or the like. The position measuring device (16) outputs at least the coordinates of the point where the user is and the direction in which he is facing.

  The camera (18) is composed of a lens and an image conversion element, and captures surrounding images as electronic data. Accordingly, the neck portion (24) is connected to a motor for rotation via an appropriate speed conversion gear, and is assembled so as to be rotatable. In addition, there is an angle meter in the neck part, and it can measure how many times the neck is turned from the front.

  The rotating antenna (20) is mainly used to receive a saliency signal transmitted by another robot. By receiving while rotating, it can be determined from which direction the signal is transmitted.

  The fixed antenna (22) is mainly used when transmitting image data or transmitting a saliency signal by itself. However, the above rotating antenna can be used for transmission, and a fixed antenna can also be used for reception.

  FIG. 3 shows the internal configuration of the search robot. The fixed antenna (22), the rotating antenna (20), the position measuring device (16), the camera (18), and the wheel (14) are the same as in FIG. The control unit (50) controls the entire search robot, and a microcomputer (hereinafter referred to as “microcomputer”) is used. Therefore, the microcomputer operates by a program written in a built-in or external ROM. There may be a plurality of microcomputers.

  The memory (52) is connected to the control unit (50) and is used for storing necessary information. The position measuring device (16) is connected to the control unit (50), and after measuring its own position, sends measurement data to the control unit (50).

  The camera (18) includes a lens and an image sensor that converts an image formed through the lens into electronic data. A semiconductor device such as a CCD (Change-coupled device) or a CMOS (Complementary Metal-Oxide Semiconductor) is used as the imaging device. The electronic image data (hereinafter referred to as “pixel data”) is converted into image data that is easier to handle by the image processing unit (60). For example, there are standards such as MPG (Moving Picture Experts Group) and JPEG (Joint Photographic Experts Group).

  The converted image data is sent to the control unit. The image processing unit (60) is usually composed of a single independent large scale integrated circuit (LSI) chip, but may be created on the same chip as the control unit (50). The control unit (50) may perform data conversion performed by the image processing unit (60) by software.

  A drive motor (68) is connected to the wheel (14), and operations such as forward movement, reverse movement, and direction change are possible. The drive motor (68) is controlled by the drive controller (66). The drive controller (66) is also connected to the antenna motor (62) and the camera motor (64), and controls them. The drive control unit (66) is connected to the control unit (50), and rotates the wheel (14), the camera (18), and the rotating antenna (20) in accordance with instructions from the control unit (50).

  In FIG. 3, the double lines from the motors are physically connected. The drive control unit (66) is configured by a microcomputer different from the control unit (50), but may be created on the same chip as the control unit (50). Further, the control unit (50) may perform the process of the drive control unit (66).

  An angle sensor (58) is connected to the antenna motor (62), and notifies the controller (50) of the angle of the rotating antenna. The R transceiver (56) is connected to the rotating antenna and transmits a signal or notifies the control unit (50) of received information according to an instruction from the control unit (50).

  Further, the S transceiver (54) is connected to the fixed antenna (22), and similarly transmits a signal according to an instruction from the control unit (50) and notifies the control unit (50) of the received information. A battery not shown supplies necessary power to each part.

  The operation of the search robot having the above configuration will be described with reference to the configuration diagram of FIG. 3 and the flow of FIG. Referring to FIG. 4, this flow starts when the search is started (S1010). In addition, the point which starts search is demonstrated later. When the search is started, the search robot images the surroundings with the camera (18) (S1012).

  The control unit (50) instructs the drive control unit (66) to rotate the camera, and the drive control unit (66) rotates the camera motor (64) according to the instruction. The image processing unit (60) converts pixel data captured by the camera into image data and sends the image data to the control unit (50). At the same time, the camera angle sensor (59) notifies the controller (50) of the camera angle.

  The control unit (50) records the received image data and the camera angle in the memory (52) in association with each other. Then, when the camera rotates once, shooting is finished. Thus, the search robot has acquired data in which the surrounding direction and the image data are paired. Note that shooting may be performed continuously while the camera rotates once, or intermittent shooting may be performed. Further, it is possible to shoot in a predetermined range instead of shooting around one circumference. For example, only 60 degrees to the front, 60 degrees to the right, and 60 degrees to the left. Furthermore, an image in only one direction may be used.

  Next, the control unit (50) processes the acquired image data (S1014). In this step, the image data is processed for transmission. The image data is managed in units having a certain size, but not all information of the image data is necessary depending on the subsequent processing. The smaller the image data sent to the supervisor robot, the more efficient the communication between the robots.

  Therefore, this process compresses or deletes the image data until it reaches a predetermined size. Further, based on the angle information of the camera and the position information obtained from the position measuring device (16), the direction of the captured image data is calculated and associated with the image data. For example, if the front of the camera is 20 degrees east from the north according to the position information, and the acquired image data has a camera angle of 30 degrees, the image data is 50 degrees northeast from the current position. It can be seen that the image data is in the direction. This is mainly software processing of the control unit (50).

  When it is ready to send the image data, the image data is sent (S1016). For transmission of image data, the control unit (50) sends relevant data to the S transceiver (54), which is performed by the S transceiver (54). In addition, you may carry out by R receiver. More specifically, the control unit (50) prepares its own ID number, position information, image data with direction data, etc., and sends it to the S transceiver (54). The S transceiver (54) transmits these pieces of information to the supervisory robot using a predetermined band and modulation method.

  There is no particular limitation on the communication method, and it may be continuously transmitted while handshaking with the supervising robot, or may be divided into packets and transmitted while being divided. When the transmission is finished, it waits for the transmission of the saliency value. The signal is received by the S transceiver (54), but may be received by the R transceiver. After receiving the data from the search robot, the supervisory robot obtains the saliency value and notifies the search robot of the saliency value.

  When the saliency value from the supervisory robot is received, it is determined whether or not there is a saliency value addressed to itself (S1018). The supervisory robot obtains the saliency value based on the image data collected from the search robot, and searches the saliency value, its rank, the ID of the robot that transmitted the image to that value, the position information of the robot, etc. Notify Therefore, if it is determined whether or not the sent ID is the same as the own ID, it can be determined whether or not the notification is addressed to the user. In addition, all the searching robots can recognize the point that the saliency value is notified.

  If it is a notification addressed to itself (Y branch of S1018), a saliency signal is sent to the surroundings based on the notified saliency value. The saliency signal may be the magnitude of the signal, or may be a predetermined frequency or phase. At least the saliency signal is a signal that can be obtained by a predetermined method from the notified saliency value. Further, the saliency signal includes position information of the search robot that is transmitting the signal.

  A control part (50) knows what kind of signal is transmitted from the comparison table of the saliency value held beforehand and the signal to be output. Then, the control unit (50) instructs the S transmission / reception unit (54) to transmit a saliency signal (S1020).

  Next, the saliency signal of another search robot is received (S1022). This instructs the control unit (50) to rotate the rotating antenna to the drive control unit (66), and instructs the R transceiver (56) to receive a signal. The control unit (50) acquires information on the antenna angle from the antenna angle sensor (58) and information received from the R transceiver (56).

  FIG. 5 is an example of the saliency signal. Here, the case where three remarkable images are selected from images collected by the search robot will be described as an example. Assume that a search robot (hereinafter referred to as “search robot A”) obtains the reception result (70) of the saliency signal shown in FIG. 5 as a result of reception by the rotating antenna (20). The vertical axis represents intensity, and the horizontal axis represents frequency. It can be seen that there are three signals sufficiently larger than the noise level (78).

  The supervisory robot analyzed the image data collected from the search robot, and sent the saliency values and their ranks to the search robot that sent three outstanding images. The search robot notified of the saliency value transmits a saliency signal by a predetermined method. Here, the saliency signal is transmitted using the frequencies of f1, f2, and f3 in descending order of saliency.

  Therefore, the search robot A that has received FIG. 5 receives the signal of the frequency f1 as a signal from the search robot that has seen the image having the highest conspicuousness, and the position information of the search robot that has transmitted f1 from the signal. Get. Similarly, the positions of the search robots having the second and third saliency are known from the signals f2 and f3, respectively. Since the signal of f1 is smaller than the signal of f2, it can be estimated that the search robot that has seen the most prominent object is far from the search robot A.

  Returning to FIG. 4, when the saliency signal as shown in FIG. 5 cannot be received, the process returns to the surrounding photographing again (N branch in S1022). Alternatively, communication confirmation with the supervisor robot may be performed.

  Next, the movement position or movement direction is determined from other saliency signals (S1024). FIG. 6 shows the result obtained by the search robot A from the reception result of FIG. 5 on the map (80). The vertical axis of the dotted line shown on the map is the y direction, and the horizontal axis is the x direction. The y direction indicates the north direction, and the x direction indicates the east direction. At the center (82) of the map is the search robot A itself. The search robot that has transmitted the signal f1 on the assumption that the saliency value is the highest is the point 82. The search robots that transmitted the signals f2 and f3 are denoted by reference numerals 84 and 86, respectively.

  The next movement position or movement direction of the search robot A is determined by the positional relationship with the search robot that has transmitted these saliency signals. There are various ways of determination, but there is no particular limitation as long as the method proceeds to the search robot that is transmitting the saliency signal.

  For example, it is conceivable to determine the movement position so that the greater the saliency value and the closer the signal source is, the stronger the direction and the movement of a certain distance. Specifically, the weighting ρi is determined in advance according to the saliency value magnitude. i indicates the ranking of saliency. Therefore, ρ1 is assigned to the search robot with the highest saliency.

  Next, a distance ri with respect to those search robots is obtained. As shown in FIG. 6, the distance ri can be obtained because the coordinates of the opponent's position are known when the position of the user is the origin. Then, assuming that ρi / ri is an attractive force from each coordinate, the moving direction is determined according to this attractive force. Therefore, it suffices to move a certain distance in the direction of the vector of (Expression 1).

  Here, θ is given as an angle from the x direction of each search robot. This is merely an example, and other rules may be determined to determine the moving direction and moving distance. As described above, the moving direction and the moving distance (predetermined constant distance in the above description) can be obtained.

  Returning to FIG. 4, the next movement is performed (S1028). Since the search robot A knows its own position, the direction to be moved, and the distance, it moves toward that direction. The movement is performed by the control unit (50) instructing the drive control unit (66). It should be noted that if there is an obstacle during the movement, a detour is made as appropriate, but the explanation is omitted because it is not the gist of the present invention.

  When the movement of a certain distance is completed, the movement is stopped, and it is confirmed whether or not a search stop signal is issued (S1028). If the search end signal has been transmitted, the search ends (S1030). If the search stop signal has not been issued, the process repeats from the surrounding shooting (S1012) step. Although the reception of the search stop signal has been described as S1028, the search stop signal may be received at any step and may be interrupted by the microcomputer.

Another aspect is as follows. Let the kth robot saliency be p (k). Each robot generates a p signal (k) is when a predetermined magnitude p _th larger in proportion to the magnitude of p (k) s (k). For example, a radio wave having power p (k) and a frequency unique to each robot is generated. On the other hand, each robot receives a signal from another robot and determines its own action based on the result.

For example, suppose that the k-th robot receives signals from other robots, k1, k2,..., Ks-th robot, and unit vectors representing the directions of the respective robots are expressed as e _k (k1), e _{k. (k2), ···, e k} (ks), the power of the received signal _{r k (k1), r k} (k2), ···, and an _{r k} (ks). Due to the attenuation effect of the radio wave, the power r _k (k1) of the reception signal is smaller than the power p (k1) of the transmission signal, and the degree of attenuation increases as the distance between the robot that transmits the signal and the robot that receives the signal increases. At this time, the position of the robot is changed according to (Equation 2).

Here, q _k (t) represents the position of the k-th robot at time t. This expression represents that the robot moves at a speed of μ in the direction represented by the vector of (Equation 3). (Equation 3) is the numerator on the right side of (Equation 2).

  If such a process is performed, each search robot does not need to stop and can search while moving.

Supervisory robot In the present invention, at least one supervisory robot exists in the search robot. The appearance of the supervisory robot may be the same as that of the search robot shown in FIG. That is, a search robot changes from a search robot to a supervisory robot under certain activation conditions. Of course, there may be something specially built for the supervisor robot. Here, it is assumed that the supervisory robot is a supervisory robot under certain activation conditions. The constant activation condition may be given by the user or may automatically become a supervisory robot depending on the conditions satisfied during the control of the group robot.

  Since the supervising robot has the same configuration as the search robot, the appearance is the same as in FIG. 2, and the internal configuration is the same as in FIG. However, the supervisory robot performs a process specific to the supervisory robot using the supervisory robot software.

  FIG. 7 shows the flow of the supervisory robot. The following operations are mainly performed by software processing by the control unit (50). It is assumed that information has already been obtained regarding the location of the area to be searched. Specifically, the position coordinates of the four corners are known with the area to be searched as a rectangle.

  When the activity is started as the supervisory robot (S1040), first, the position of the search robot is confirmed (S1042). This may be notified of position information to all the searching robots all at once, or may be heard for each searching robot. These communications are performed using the supervisory robot's R transceiver (56), but the S transceiver (54) may be used, or a combination of these may be used.

  Next, the arrangement of the search robot is determined. In the present invention, a search is performed by gathering other search robots in a search robot that has seen an object having a different feature from the image data obtained by the search robot. Therefore, it is preferable that the search robots are scattered all over the search area at first. Therefore, the search robots are instructed so that the search robots are randomly arranged based on the information on the positions of the search robots and the search areas. For example, it arrange | positions like the state of Fig.1 (a). However, it does not have to be arranged randomly, and the search may be started from a regular arrangement.

  When the arrangement of the search robot is completed, a search start instruction is issued (S1046). This can be done using the S transceiver (54). Of course, an R transceiver (56) can also be used.

  Each search robot starts searching when receiving a search start instruction as described in FIG. When each search robot completes the search, image data and the like are transmitted and received (S1048). In order to receive data from a large number of search robots, it is received while confirming the other party by handshaking or the like.

  When data from all search robots has been received (Y branch of S1050), saliency is extracted based on these data (S1052). When the saliency extraction is completed, the saliency value can be obtained. Details will be described later.

  It is determined whether or not to end the search (S1054), and when the search is to be continued, the saliency value is notified to each search robot. The end of the search may be instructed by the user or may be determined from the condition of the set of search robots.

  Next, details of the saliency extraction (S1052) will be described. The saliency is the degree to which the data of interest is not similar to the surroundings among many data. To extract this, there are two steps: a step of extracting features of the image from the image acquired by the search robot and a step of extracting saliency based on the features. A generalized explanation will be given for each, and then a specific example will be explained.

Extraction of feature points A vector having the value of each pixel of an image composed of n pixels acquired by the camera as an element is represented by (Expression 4). And the feature shown by (Formula 6) is extracted using the vector (Formula 5) defined beforehand.

  By (Equation 6), features consisting of m numbers are extracted. Hereinafter, these are collectively referred to as feature data. (Equation 4) is image data taken by the search robot. The image data is obtained by arranging two-dimensional images in numerical form as a one-dimensional array. Therefore, one set can be considered as one image data, for example.

  (Equation 5) is a so-called filter. If there is a feature defined by f in the image data, it can be obtained as a numerical value by the inner product shown in equation (3). That is, one feature is one value, and m pieces can be extracted from one piece of image data. The user can freely decide what f is to be set and how it is used in the situation where the present invention is used.

Extraction of saliency The saliency is obtained by performing principal component analysis from the feature data obtained as described above, obtaining an eigenvector representing a highly similar view, and determining how far away from the eigenvector. . Consider the features extracted by the k-th robot with an m-dimensional vector (Equation 7).

  In the means for performing principal component analysis, an m × m matrix is calculated by (Equation 8). Then, an eigen equation (Equation 9) using this S is considered. Then, this eigen equation is solved to obtain the eigen vector of (Equation 10) and the eigen value of (Equation 11).

  The eigen equation can be solved using a known method. Note that the eigenvalues are arranged in descending order.

  Here, the eigenvector represents an axis (principal component) that is relevant among a large number of feature data, and the eigenvalue indicates how common the feature data is to each eigenvector (whether the variance is small). ). Therefore, the eigenvector v1, which is the largest eigenvalue μ1, best reflects the commonality of the feature data.

  As a specific example, it is assumed that the search robot searches the forest and takes an image of the surrounding area. Then, since most of the images should show the green of the plant, if the feature of “green” is included in the feature data, the eigenvector of “green” is extracted by the above calculation, and a large eigenvalue is obtained. It is to have.

  Next, the saliency is obtained. Of the m eigenvectors obtained by the principal component analysis means, 1 is selected from the eigenvectors having the larger eigenvalues, and this is expressed as (Equation 12).

  The characteristic obtained by the means for extracting the characteristic of the k-th robot is expressed by (Equation 13). Each feature data is projected onto an eigenvector, and the size is obtained. Specifically, (Equation 14) is calculated. The saliency p is calculated by (Equation 15).

  Since t ′ (k) is a mapping onto the eigenvector, the smaller it is in the direction orthogonal to the eigenvector, the smaller the value. That is, p (k) increases as it is orthogonal to the eigenvector and the distance from the eigenvector increases. That is, it is far from the same tendency of many other feature data.

  FIG. 8 is a diagram schematically showing this state. A certain eigenvector (92) has a very high correlation with the feature data (90). This is nothing but this eigenvector has a large eigenvalue. Here, the feature data (94) is far from this eigenvector.

  In FIG. 8, t (k) corresponds to reference numeral 95 and is a vector of feature data (94). T ′ (k) that is a projection onto the eigenvector (92) is a vector of the code (96). Therefore, the saliency p (k) is the square of the vector size of the code (97). This is nothing but the square of the distance from the eigenvector (92). The saliency of the feature data (94) is greater than the square of the distance from the eigenvector of other feature data. That is, it can be said that it is remarkable.

  As a more specific example, a very simplified model for the case where there are four search robots will be described. Assume that the four search robots are q1, q2, q3, and q4, respectively. Then, the position of the robot at a certain time is set as (Equation 16).

  FIG. 9A shows this state. Assume that each search robot has the image sensor shown in FIG. This is an image sensor in which elements that can detect each color of red (R), green (G), and blue (B) are arranged one by one. Accordingly, the pixel data output of each search robot is 3 bits (red, green, blue). It is assumed that the image processing unit () of each search robot transmits this pixel data as it is to the supervisor robot as image data. In addition, each search robot is in darkness. Then, the image data transmitted by each search robot is (Equation 17). In (Equation 17), the number of each search robot is a subscript. A vector that becomes a predetermined filter is expressed as (Equation 18).

f ⁽¹⁾ is a green filter, and f ⁽²⁾ is a red filter.
Next, it is assumed that images taken by the search robots q1, q2, q3, and q4 when the supervisory robot issues a search command are green, black, yellow, and red, respectively. That is, the image data is expressed as (19). Yellow is a complementary color of green, and green data is -1. Next, feature data is obtained from the image data and the filter. These are summarized in (Equation 20).

  Therefore, when the feature data (two-dimensional) extracted by each robot is collected, (Equation 21) is obtained. FIG. 9B shows the arrangement of feature data. The horizontal axis shows a red tendency and the vertical axis shows a green tendency. At this time, S in (Equation 8) becomes (Equation 22). The eigenvalue equation for the matrix of (Equation 22) is (Equation 23). When this equation is solved, two solutions shown in (Equation 24) are obtained.

That is, the first solution has an eigenvector of v ₁ = (0, 1) and an eigenvalue of μ ₁ = 2, and the second solution has an eigenvector of v ₂ = (1, 0) and an eigenvalue of μ ₂ = 1. is there. The eigenvector v ₁ = (0, 1) having the maximum eigenvalue is the main axis representing the principal component, and v ₂ = (1, 0) is the second main axis. It should be noted that v ₁ and v ₂ are horizontally written for convenience of description where they should be written as (number). The same applies hereinafter.

  As shown in FIG. 9C, the features extracted by the first, second, and third robots are arranged on the axis representing the first principal component, and the features of the fourth robot are separated from the axes. ing. The first main component is the vertical axis and is indicated by a thick arrow. Therefore, the features of the first, second and third robots are similar to each other (having the same tendency), and it can be determined that the fourth robot has different features. In other words, the green tendency is the first principal component, and the red tendency is the second principal component.

Next, the saliency is obtained. First, the eigenvector v ₁ = (0, 1) having the maximum eigenvalue is selected, and the characteristics of each robot are projected onto a straight line having the direction of v ₁ = (0, 1). That is, the vector t which is a feature is expressed as (Equation 26). The projection of (Equation 26) onto the eigenvector v ₁ is expressed by the equation (Equation 27). Therefore, the saliency of each robot is expressed as (Equation 28).

  That is, only the fourth robot has a saliency of 1. Indicates that the features extracted by this robot are distinct from others. This corresponds to the fact that the 4th robot is highly noticeable because the 1st to 3rd search robots are lined up in green, whereas only the 4th robot saw red. . Let p obtained in this way be the saliency value.

  Only the fourth robot outputs a signal corresponding to the size of 1, and the other robots receive it and proceed in that direction, so the position of the robot a little later is as shown in FIG. 1T, 2T, and 3T indicate the positions of the first to third robots after a little time has passed. In this case, the fourth does not move.

  The above is the detailed description of the extraction of saliency (S1052). FIG. 11 shows these in a flow. This flow starts from the extraction of saliency (S1052) (S1070).

  First, feature data is extracted by applying a filter vector to image data (S1072). This is a general expression, which corresponds to (Equation 6) and specifically (Equation 20). Next, principal component analysis is performed (S1074). This is a general expression, which corresponds to (Equation 8) and specifically (Equation 22).

  Next, the eigen equation is solved (S1076). This is a general expression and corresponds to (Equation 9), and specifically, (Equation 23). If there are m feature data, an m × m determinant is obtained. This solution is generally known. By solving the eigen equation, m eigen vectors and m corresponding eigen values can be obtained.

  Then, l (el) are selected from those having large eigenvalues. The number of eigenvalues selected here means how many similarities are taken into account when obtaining the saliency value of each search robot. For example, in the above specific description, the saliency is obtained with one similarity of “green”. However, when the number of feature data is larger, the saliency can be determined in consideration of a plurality of similarities.

  Next, the feature data is projected onto the eigenvector selected in S1078 (S1080). As can be seen from (Equation 14), all feature data are applied to the selected eigenvectors one after another. That is, if m feature data are extracted from image data from individual search robots, all the l eigenvectors are combined with the feature data and added together. By this step, t ′ (k) can be obtained for the k-th search robot.

  A saliency value is calculated based on the result of S1080 (S1082). This is a general expression, which corresponds to (Equation 15), and specifically, (Equation 28). As a result, one saliency value p (k) can be obtained for each search robot.

  It is determined whether or not the saliency values have been calculated for all search robots (S1084), and j are selected from those having a large saliency value. As a result, the saliency value, its rank, and the ID number of the search robot having the saliency value can be obtained. As described above, the step S1052 for extracting the saliency value ends, and the process exits from this routine (S1088). The above is the operation of the supervisory robot.

Display device Fig. 12 shows an overview of a display device. The display device includes at least a display portion (102) and an antenna (104). In addition, it has an operation part (106,108). The display unit (102) has cursors (110, 112). The display unit (102) is preferably composed of a liquid crystal panel, but an organic or inorganic EL panel, or a display device such as an SED or FED can be used. The cursors (110, 112) are not limited to those shown in the figure, and any cursors that can indicate coordinates on the display unit (102) may be used. For example, an arrow display or a cross display used in the current personal computer may be used.

  The operation units (106, 108) are used for operations such as moving the cursors (110, 112) and giving commands to the supervising robot and the search robot. The main purpose of the display device (100) is to acquire the position information of the search robot and display it on the display unit (102). This is because it is possible to discover something characteristic depending on how the search robots gather.

  It is also possible to acquire the image data of the robot designated by the cursor (110, 112) and display it on the screen of the display unit (102). As described above, the search robot, the supervisor robot, and the display device can discover characteristic objects that are different from the surroundings in the search area in a gathering manner of the search robots.

  The position information of the search robot has been described as an embodiment in which the position information is acquired from the search robot itself, but is not limited to this. For example, the search robot itself only transmits light, radio waves, and weak radiation, and the supervisor robot or display device receives these transmissions from a position overlooking the search robot, and grasps the movement of the search robot as a whole. Also good.

(Embodiment 2)
In this embodiment, feature data is calculated from image data captured by the search robot, and the feature data is transmitted to the supervisory robot. Finding feature data from a single piece of image data requires a considerable amount of calculation. Therefore, if each search robot calculates and transmits the feature data to the supervisory robot, the load on the supervisory robot is reduced.

  In FIG. 3, in the present embodiment, the image processing unit (60) performs the process up to the extraction of feature data. Therefore, the image processing unit (60) sends feature data instead of image data to the control unit (50).

  FIGS. 13 to 15 show details of the search flow of the search robot, the search flow of the supervising robot, and the saliency extraction of the search robot, respectively. FIG. 13 is substantially the same as FIG. 4, but differs in that image processing (S1014) is performed until feature data is extracted (denoted as S1013). Also, image data transmission (S1016) in FIG. (Denoted as S1015).

  In the image processing (S1013), in addition to the image processing in the first embodiment (S1014 in FIG. 4), a part of the process of the supervisory robot in the first embodiment is performed. Specifically, the step of extracting feature data from the image data (S1072) in the flow of extracting the saliency of the supervisor robot (FIG. 11) is performed. More specifically, the processing of (Equation 6) is performed. Accordingly, each search robot is notified in advance of the filter vector from the supervisor robot or the display device (100) of the user.

  FIG. 14 shows the flow of the supervisory robot. In the first embodiment, the feature data is acquired (S1047) in S1048 of FIG. Further, the extraction of saliency (S1052 in FIG. 7) is the extraction of saliency (S1051) in FIG. This is because the calling method is the same, but the processing is different from that of the first embodiment.

  FIG. 15 shows the flow of the processing in S1051. In the first embodiment, the flow corresponds to FIG. Since each search robot has already performed feature extraction processing on its own image data, the feature data extraction step is not necessary in the present embodiment. That is, the step of S1072 in FIG. 11 is not necessary for the supervisory robot of the present embodiment.

(Embodiment 3)
In the description of the embodiments so far, the filter vector for extracting the feature data from the image data acquired by the search robot has been given to the supervisor robot or the search robot in advance. Certainly, when the environment of the search area is known in advance, it is possible to design a filter vector for extracting features. It is also possible to schedule several search area environments and prepare several filter vectors in advance.

  However, even though the search areas are similar, they differ depending on the search area depending on the manner of occurrence of the disaster, etc., and there are cases in which feature data cannot be extracted well with a special filter vector such as a template. Therefore, in the present embodiment, the filter vector is created from the image data acquired by each search robot itself.

  In the present embodiment, the mode is almost the same as that of the second embodiment. That is, the search robot instructed to search images the surroundings according to the flow shown in FIG. 13, acquires image data, performs feature extraction, and transmits the feature data to the supervisory robot.

  The supervisory robot extracts the saliency from the feature data collected from each search robot, and notifies the search robot that has transmitted the feature data.

  The search robot notified of the saliency transmits a saliency signal corresponding to the saliency and receives the saliency signals of other search robots. Then, the next movement position is determined based on the received saliency signal and moved.

  The second embodiment is different from the second embodiment in that a search robot creates a filter vector by itself before extracting feature data.

  FIG. 16 shows the follow-up of the search robot according to the present embodiment. In step S1090, a process for creating a filter vector is newly entered before surrounding shooting (S1012). In the following, how to create a filter vector will be described in detail.

  An image composed of n pixels is captured by the sensor 510 at time t, and a vector in which the pixels of the image are arranged is represented by (Equation 29). Images are taken while moving the robot, and many such vectors are obtained.

  Next, an n × n matrix is calculated by (Equation 30), and m eigenvectors (Equation 32) are calculated from the larger eigenvalues by solving the eigenvalue equation of (Equation 31) to be filter vectors.

  The search robot first captures several surrounding images while moving a little distance, and extracts the main factors in the surrounding landscape by performing principal component analysis on the images. And let it be a filter vector. For example, a search robot placed in a forest could obtain a green landscape filter vector by this process.

  Once the filter vector is created, the same processing as in the second embodiment is performed. Therefore, the search robot captures a surrounding image, obtains image data u, and performs the following processing with the filter vector.

  This process calculates a feature consisting of m numbers. By the way, when the filter vector is calculated by such a process, feature extraction using a different filter vector is performed for each search robot. This is a point greatly different from the first or second embodiment.

  However, if many search robots are activated in a certain search area, slight differences in filter vectors of individual search robots are not a problem. This is because all search robots are arranged in a similar landscape within the same search area. Statistical processing of a large number of data smoothes out the difference of filter vectors for each search robot.

  The present embodiment is very useful in that it is not necessary to provide a filter vector suitable for the search area in advance. In particular, it has high applicability that feature data can be extracted from any search area.

The figure which illustrates the motion of each search robot of the present invention The figure which shows the general view of the search robot of this invention The figure which shows the structure of the search robot of this invention The figure which shows the flow of a search of the search robot of this invention Diagram showing examples of saliency signals The figure which shows the positional relationship with a search robot which a certain search robot transmitted the received saliency signal The figure which shows the flow of supervision operation of the supervision robot Diagram explaining saliency The figure which simplifies and demonstrates the motion of the search robot of this invention The figure which shows the image conversion element of the robot of description of FIG. Figure showing the flow of saliency value calculation Figure showing a display device The figure which shows the flow of the search robot of Embodiment 2. The figure which shows the flow of the supervisor robot of Embodiment 2. The figure which shows the flow which calculates | requires the remarkableness of the supervising robot of Embodiment 2. The figure which shows the flow of the search robot of Embodiment 3.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Search robot 16 Position measuring device 18 Camera 20 Rotating antenna 22 Fixed antenna 50 Control part

Claims (10)

At least one supervisory robot,
It has three or more search robots and a display device that displays the position information of each robot.
Each of the search robots is
Take a picture of your surroundings,
Send the captured image information to the supervisory robot,
When a saliency value is received from the supervisory robot, a saliency signal is transmitted,
Receive saliency signals from other search robots,
Moving based on the received saliency signal;
The supervisory robot is
Receiving the image information and the position information from each of the search robots;
Obtain a saliency value from the set of received image information,
A group motion control system comprising a plurality of robots that transmit saliency values to the search robot that has transmitted image information corresponding to the determined saliency values. A camera that captures surrounding images,
A mobile device that moves its housing and a signal transmitter that transmits a signal;
A signal receiver for receiving the signal;
An image processing unit for processing image information captured by the camera;
An image transmission processing unit that causes the transmitter to transmit data processed by the image processing unit;
A saliency transmission processing unit for transmitting a saliency signal by the transmitter when receiving a saliency value addressed to itself;
A search robot having a movement determination unit that receives a saliency signal transmitted from another robot and determines a movement direction based on the saliency signal. The search robot according to claim 2, further comprising a position measuring device that grasps its own position. The search robot according to claim 2, wherein the data processed by the image processing unit is image data. The search robot according to claim 2, wherein the data processed by the image processing unit is feature data extracted from image data.   6. The search robot according to claim 5, wherein a filter vector for extracting the feature data from the image data is created by itself. A receiver for receiving the signal;
A transmitter for transmitting a signal;
A saliency extraction unit for obtaining a saliency value based on image information received from another robot;
A supervisory robot having a saliency notification unit that transmits a saliency value to another robot that has transmitted image information having the saliency value extracted by the saliency extraction unit. The supervisory robot according to claim 7, wherein the saliency extraction unit obtains a saliency value based on feature data received from another robot. The supervisory robot according to claim 7, wherein the saliency extraction unit obtains saliency by principal component analysis of data received from another robot. A display and a receiver for receiving signals;
A transmitter for transmitting a signal;
A position display processing unit for displaying position information of a plurality of robots on the display unit;
An instruction transmission unit for issuing instructions to at least one robot;
An image display processing unit that receives image information from a specific unit and displays the image information on the display unit.