CA2444364A1 - A multi-agent system for automated image analysis and understanding - Google Patents

Abstract

A method for analyzing one or a plurality of digital images using a plurality of software agents, comprising: a program for initializing and assigning a plurality of agents to at least one of said images; means for said agents to perceive at least one of said images at one or a plurality of scale representations; and means for said agents to modify pixel values of said images.

Description

S A IVlulti-Agent Sys~rn ~r Au~rr~ated Image Analysis and nc9erstanding background of the Invcnti~~
The present invention relates to the field of image analysis and more specifically to methods for automated image segmentation, analysis and understanding by means of a novel lVIulti-Agent system.
There currently exists a broad spectrum of ignage analysis technologies and methods that try to provide automated image analysis. 1-Iowever, major problems arise from these methods as they are most commonly based on simple spectral thresholding techniques or variations thereof. There have been many attempts at providing efficient automated 1S image analysis methods, some of which have been more or less successful, such as multiscale analysis and deformable models. In recent attempts, there have been many developments in methods that combine already existing techniques. 1-Iowever, the main problem with the combination of existing methods consists in determining the optimal set of parameters and order in which th.e methods should be applied. With the recently added requirement of symbolic image interpretation in the industry as well as in academia, in addition to image segmentation, standard image analysis methods have even more shown to be inadequate. To efficiently address these issues, a new theoretical field known as Mufti-Agent systems has been adopted and formalised by the inventors specifically for the problems at hand. Although the basic concepts in Maxlti-Agent systems are known in 2S the art, the present invention brings forth completely novel methods therein for allowing robust and fully automated image segmentation, analysis and understanding.
Patent 5,978,520 provides a method and system for interactively generating vector data, more specifically to draw vector lines, from drawing data by means of a basic mufti-agent system. The provided mufti-agent system is based an the concept of a supervisor agent that enforces actions upon the agents it controls when problems or contradiction arise. In contrast to the invention described in 5,978,520, the present invention provides an automated system and methods for the automatic extraction and quantification of objects in images as well as symbolic image interpretation by means of a mufti-agent system composed of fully autonomous agents.
Summary ~f the fnvento~n An object of the present invention is to provide an image analysis and segmentation method and apparatus thereof for performing automated. image analysis in various fields of applications where extraction and quantification of objects in images is required and where precise and robust analyses are of primary importance.
Another object of the present invention is to provide a mufti-agent system that integrates novel concepts in order to achieve full automation and maximum precision in image analysis contexts where complex image structures are required to be analyzed.
The herein described invention therefore pr~vides a mufti-agent system that integrates the concepts of multiscale-based agent cooperation and sensing sub primitive agent specializations, dynamic agent parameterization as well as incremental feedback.
Overall there is provided methods and apparatus for first inputting digital images, then automatically analyzing the digital images by means of a specialized and autonomous mufti-agent system that is initialized and assigned to the considered images by a program, and finally for displaying to the user and for saving in repositories the results generated by the mufti-agent system.
Brief escripti~n ~f the ~rawinga The invention will be described in conjunction with certain drawings which are for the purpose of illustrating the preferred and alternate embodiments of the invention only, and not fox the purpose of limiting the same, and whereino Figure 1 displays the basic agent initialization and assignment process.
Figure 2 depicts the various agent components for an erribodiment.
Figure 3 displays the basic flow of an example agent initialization and assignment process for one embodiment.

2 Figure 4 displays the entire system's basic operational flow.
Figure 5 depicts the conccpt of an agent sensing an imagc by means of its sensors.
Figure 6 depicts the basic steps involved in the agent-to-agent communication process by means of the Receiver/Transmitter component.
Figure 7 depicts the concept of multiscaie agent specialization and cooperation.
Figure $ is a representation of the components involved in providing multiscale sensors.
Figures 9a - 9b represent the concept of multiresolution image sensing.
Figures l0a -10c depict the process of multiscale information gathering by an agent.
Figure 11 is a drawing of a typical satellite image landscape.
Figure 12a - 12b illustrates the extraction of water course ridges and the process of agent initialization on these ridges.
Figure 13 illustrates the initialization of Border Agents and the path following by Ridge Agents.
Figure 14a -14b illustrate the bridge detection process.
Figure 15a -15b represents the semantic interpretation of water courses.
Figure 16a -16b illustrate the detection of entities within water courses.
Figure 17 illustrates a basic configuration of an apparatus for analyzing images comprising N processing nodes.
Referring numerals comprised in the figures are here forth mentioned in the detailcd description within brackets such as: (2).
Detailed Description Although it is not the objective of the present invention to extensively describe hardware infrastructures that may be more optimal than standard pcrsonal computers as support to

3 the herein described mufti-agent system, we however describe a distributed hardware configuration that may be used in the present context.
Distributed Systems A mufti-agent system is inherently highly distributable; from the fact that individual or a group of agents can reside on separate machines and communicate with one another over a network connection. From this, various types of distributed architectures can be considered in the implementation of a mufti-agent system. Considering applications such as high resolution satellite image analysis, where considerable processing power is required to process the data in reasonable time frame, a distributed hardware platform may be required. A typical configuration is that of two or more networked computers i 5 where each computer (node) processes information using a program, and where a program communicates with another program residing on another computer through a communication interface. Patent 6;350,239 discloses a basic distributed software architecture for medical diagnostic systems, where distinct modules that perform well defined and separate functions are executed on separate computers. °The invention disclosed in 6,350,239 does not however address the challenging problem of distributing a single "module" over a plurality of nodes. Contrastingly, the herein described invention provides means for distributing a single "module" over a plurality of computers or processing nodes, where each node may be host to one or a plurality of software agents that each perform a subtask of the global task (module). Through communication and scheduling of events, the plurality of software agents will achieve the required end-result.
An example mufti-node apparatus is illustrated in Figure 17. Depending on the nature of the application, the distributed system can be either loosely or tightly coupled. In the former case, the system can be a cluster-based backbone, where each processing node is physically located near one another, in a rack mounted configuration for instance, and _ where each node is networked to a neighboring node using a high-speed network interface and cable (such as 1Gb fiber-optics interconnects). The use of high speed networking reduces the communication latency between any program and module required to exchange information. In the loosely coupled scenario, the backbone may

4 comprise networked computers physically remote to one another between which information can be exchanged using a networking protocol such as T~P/IP.
Although the present invention is detailed 'with an embodiment using a single personal computer as computational backbone, it is understood flat it can be extended to a snore sophisticated architecture such as a cluster system.
Entire System The entire system comprises two main segrnentse A first segment that integrates basic hardware and software components that form the basic backbone on which the multi-agent system relates, and a second segment that is the mufti-agent system itself which comprises a plurality of software agents and basic software components.
IS Referring to Figure l, the first process involved in the system consists in inputting one or a plurality of images by means of an input device (2). The latter can be a standard inputloutput device for reading digital data from a repository, or from an imaging device, and can comprise a communication interface to read the data from remote repositories over a network such as the Internet. Once the images) read by the device and stored in memory, the Mufti-Agent initialization and assignment program is instantiated.
The program instantiates one or a plurality of software agents and makes these software agent instances accessible in memory (4). Once the agents are instantiated, the program assigns the agents to the images that are required to be analyzed (8). In one embodiment, once the agents have been instantiated, they become entirely autonomous entities, making their own decisions and acting according to their goals and states (6). In one embodiment, the agents can be randomly assigned to the images, wherein the program positions each of the agents at a random coordinate within the image. In another embodiment, the agents are strategically positioned within the image according to a priori knowledge on the problem at hand and general image structures, as well as according to the agent's specialization. In an example embodiment, the initialization and assignment program invokes a pre-processing method for detecting local minima within the image as initial landmarks for positioning the agents. Referring to Figure 3, this initialization process consists in first detecting the minima within the image using the said pre-processing S

method (24). For cach of the said detected minima, spatial coordinates are extracted by means of the said method, wherein the said coordinates serve as the initial positioning coordinates, and are saved in memory (26). The initiation and assignment program is then input the set of coordinates, for which it automatically assigns the initial agents by instantiating each of the agents and registering the latter with their according pixel coordinate and value (28). From this point, the assigned agents will be autonomous and can move within the image to inspect other image regions and associated image pixels.
Referring to Figure 4, the entire system's overall operational flow comprises four main steps: first inputting a digital image to the system (30), then initiating the Mufti-Agent system {32), followed by the actual step of image analysis by means of the Mufti-Agent system (34), and finally, displaying the segmented image and associated results (36).
Multa-Agent System The current invention relates to a novel mufti-agent system devised for the automated analysis and segmentation of digital images.
Agent Architecture )3y definition a mufti-agent system comprises a plurality of agents. The present rnulti-agent system comprises a plurality of complex agents that have the means to receive information, take decisions, communicate, and take action, in an autonomous manner. To achieve this, a specific agent architecture that comprises a plurality of components has been devised. In one embodiment the, components are the following:
- Sensor(s) - Processor(s) - Actuator(s) - Receiver/Transmitter - Neural Processors) - State machine (s) - Memory unit{s) These components provide all the necessary means for creating autonomous and minimally "intelligent" agents. Each of these agents operates by first receiving stimuli from the external environment. Referring to Figure 2, an agent first receives input from either its sensor component (10) or its receiverltransmitter component (16).
The sensors provide the agent with means for sensing its physical environment, which is in the present invention one or a plurality of digital images. In this sense, the sensors provide the agent with a view of the digital image, wherein the said view is composed of processed or unprocessed pixel values that are within the range of the agent's sensors within the image. Referring to Figure 5, in one embodiment the agent has two circular sensors (38) that are positioned according to the agent's position within the image, as depicted by {40) which refers to an image region. Image objects are indicated by (42).
When the agent senses the image using its sensors, it has an appreciation of the image content based on the pixel values, from 0 to 255 for instance, comprised within its sensors. Again in Figure 5, the agent's sensors overlay portions of image objects that may have different pixel values (44 and 46). At this point, the agent receives image information from its sensors that indicates it is on image objects.
As opposed to receiving image information by means of sensors, the agent's Receiver/Transmitter component provides means for the agent to communicate with other agents and therefore receive and/or transmit valuable information. This communication can be for instance with objective of negotiating, collaborating, or perhaps simply update some information or knowledge cumulated by the entire mufti-agent system. In one embodiment, when an agent initiates communication with another agent, the said agent (48) first queries an Agent Registry (50) that holds a list of agents that are instantiated in the system along with their associated key or ID (52). The agent then uses the key or ID
to open a communication channel with the other agent (54). Finally, once the communication channel with the other agent is established, the agents are free to communicate (56). In another embodiment, the Receiver/Transmitter component is used to send information to or receive from a shared memory, where information can be gathered and accessed by all agents. This shared memory can be a database or a specific data structure residing in memory and made available to all agents.
~nce external stimuli are input by the agent, the following steps involve multiple components and require the agent to take action, and may also require the agent to make a decision before taking action. Referring to Figure 2, the information input by the sensors {10) is channeled to both car either the Neural Processor (18) and Processor (12) S components. On one hand, the Processor performs various type of processing operations on the input information, such as, without limitation, edge detection, contrast enhancement or computation of statistics. On the other hand, the Neural Processor component can perform classification tasks on the raw input information or on information processed by the Processor. In one embodiment, the Neural Processor identifies whether the input information is the edge of an object in the image. In general, the Neural Processor can identify the information input to it by means of classification or can incrementally generate a new knowledge base in conjunction with the Memory Unit (22). This incremental aspect of the Neural Processor provides the agent with means for learning on its own and can therefore adapt itself to new problems and new types of images. This can be achieved by first inputting the sensor information in the Neural Processor, then automatically classifying the input information and saving a descriptor of this information along with its associated class in the Memory Unit. In one embodiment, the Neural Processor is comprised of one or a plurality of Artificial Neural Networks, in which case the learning process further adapts the Networks' weights. In another embodiment, the Neural Processor comprises Self Organizing Networks such as Self Organizing Maps, Growing dell Structures, or any variants thereof. 'The self organizing networks can provide powerful means for automatically classifying and characterizing the visual information input by the agent's sensors.
An agent further comprises an Actuator component (14) and can also comprise a State Machine (20). On one hand, the Actuator is the component that controls the agent's process of taking action. It is a general component and can control the process of communicating, proper information dispatching, adapting sensors, or even the process of moving in the image and changing its content. Specific behavior or decision making can be achieved in conjunction with the State Machine which serves as the Agent's pre-defined mental states. In one embodiment, the agent's Actuator will force the adaptation of the Agent's sensors based on the output of the Processor, State Machine, Neural Processor, or any combination thereof. The sensor adaptation can comprise, without limitation, changing the size of the sensors so as to view more or less information at one instant within the image. Another adaptation can be to enforce the sensors to view the information at a specific scale within a multiscale view of the image. In another embodiment, the Agent will move in a specific direction in the image according to the Actuator's command based on the knowledge acquired by the other components.
This process of moving an agent in the image can take the form: move to coordinate (x,y) and sense image using sensors at these new coordinates. In which case, the Agent autonomously moves in the image. As the agents' general objectives are to analyze and segment images, the Actuator component is also responsible far the action of changing pixels within an image, as part of the image segmentation process. When an agent is in the mental state of "segmentation", as defined by its State Machine component (see below), the agent changes the value of the pixel on which it finds itself. The change in value can be of binary form (change any pixel to a 1 or a 0), or can also be of any other value. Each agent specialization may change the pixel values to their own predefined value, which results in associating these specific pixels to their own class.
For instance, a water agent can change the values of pixels it identifies as ''water" to a unique value of 255, where pixels segmented by other specialized agents will have values different than 255. In the present invention, the original input images are never directly modified, but instead copies of the images are created and used by the system. In such case, the segmented image that comprises modified pixel values will be done on a copied image, used by every agent in the system.
The agent's State Machine component provides means for pre-programming desired behaviors and patterns upon the activation of a specific state. For any given state the agent is in, if this particular state is coded in the State Machine component, then an action or a sequence of actions that is to be taken is directly determined from this component.
Upon definition of an action or set of actions, the State Machine will work in conjunction with the Actuator for the agent to execute the required actions. In specific embodiments the State Machine component is defined by Decision Trees and Finite State Machines. An example sub-sequence from the State Machine component could be as follows:
~ ~, IFsensor input information not cdass~ed by Neural Processor ~ Adapt sensors ~ Revalidate input ~ ...

The sub-sequence above provides a simple view of how the component can be involved in the execution of actions when the agent is in a specific state. In this particular example, the agent's Neural Processor was not able to identify the nature of the information input by the sensors and therefore finds itself in an "Information not Classified"
mental state.
This state is encoded by the State Machine and defines the various actions required to be 14 taken by the agent when such a state is encountered. Tlv~e example sub-sequence is with respect to the dynamic adaptation of the sensors, where when the information is not classified, the agent is required to change its sensors' configuration (adapt) and revalidate the information input by the later. In one embodiment, when the agent cannot readily identify the information as being part of an image object's edge, the sensors dynamically change the scale and resolution at which the information is considered, a method that may facilitate the identification process. Although the Example State Machine sub-sequence is of simple nature, it is understood that the complete decision tree or finite state machine may comprise a broad range of states and decision processes. Although not directly illustrated in Figure 2, the ReceiverlTransmitter component may also be 2~ controlled in conjunction with the State Maclsine, where for instance a simple incoming communication may trigger a predefined sequence of actions as defined by the component.
According to the herein ,described Agent Architecture, the present Multi-Agent system can be composed of specialized agents that make decisions and take actions specifically according to their specialization and in accordance with the inherent collaborative aspect of the system. From the previous description of the basic components that compose an agent, it is clear that an agent can integrate derived components (same components with modifications) as well as a plurality of other components. In one embodiment, each agent specialization has sensors that can differ in shape, sensitivity, specificity and behavior.
For instance, an agent that is specialized for the detection of object contours can have Neural Processor and Sensors that are specialized for the detection of edges.
In another embodiment, an agent can be specialized fox the analysis of high noise images, where the agent will be adapted to perceive the images as a multiscale representation so as to be robust to the noise and artifacts. A plurality of agent specializations can be readily IO

devised with the herein described agent architecture. It is therefore understood that the Tnvention is not limited to the above described embodimeruts of agent specialization.
Multiscale Perception used on the beforehand described agent architecture, an aspect of the present system is the muitiscale-based agent cooperation and image perception. The concept of rnultiscale perception herein considered comprises two main aspects: 1 ) multiscale perception achieved through the cooperation of agents specialized for distinct or multiple scale levels, and 2) multiscale perception achieved by means of multiscale and multiresolution agent sensors. In the first case, the general concept of scale can be applied to both semantic and spatial scales, wherein the second case only applies to spatial scale.
Cooperative Multiscaie Perception In this first case, a number of agents are specialized and adapted to analyze the image represented at certain scale levels, where for instance agents specialized for analyzing an image at Level 'N' (58) cooperate with one another and further cooperate with agents specialized for a level 'N-1' (60). In this context, agents analyzing an image at a scale level N can facilitate the analysis process by providing new information to agents at finer scale analyzing the base image data. As the agents at higher scale have a different and broader understanding on the nature of the object or region currently being analyzed, the provided information to the lower level agents will allow the latter to adapt their parameters and reorient their paths if required. The higher level agents can also enforce restrictions on the lower level agents, where the agents can be bound to a specific region for instance. This multiscale cooperation can also comprise mutual feedback mechanisms where the knowledge and information gathered by agents at higher level can facilitate the lower level analysis, and where the lower level information can improve and facilitate the higher scale analysis. This coevolution provides robust image analysis means as it simultaneously takes into account multiple scale representations of an image, allowing for increased robustness to noise and providing a global and local view of the inforanation. Figure 7 depicts this concept of rnultiscale cooperation where agents on multiple scales cooperate with one another in order to efficiently analyze the raw image lI

data. The herein described method of cooperative multiscale perception and image analysis can further comprise the processes of agent rc~producaion and agent cloning where for instance when an agent that analyzes the raw image data finds itself in an uncertain environment or is unsure on the action it should perform due to the degenerated information, the agent has the possibility to clone or reproduce itself at higher scales, or inversely from higher scale to lower scale. This requires the agent to first identify the scales at which it will instantiate new agents, then clone or reproduce itself to these scales, allow the new agents to analyze the image they find themselves on, and update itself according to the information and knowledge provided by the new agents.
Multiscale Sensor Perception In the case of sensor based rrmltiscale perception, where .only spatial scale is considered, the agents are composed of sensors adapted to provide a multiscale representation of the portion of the image being sensed. The agent therefore has the possibility of considering the image at one spatial scale level or at a plurality of levels within the multiscale representation. Referring to Figure 8, this mechanism comprises the steps of first reading the data that is in range of the sensors by means of the sensors(62), following this, the .
data is then input to the Processor component which computes a multiscale representation of the said data (64) using a method lmown in the art. The multiscale data representation can be input to the Neural Processor or further used by the Processor component. In one embodiment, the agent considers only one specific scale, level of the multiscale image representation. In another embodiment, the agent sequentially or simultaneously considers a plurality of scale levels. In yet another embodiment, the agent perceives the image at various spatial scales as well as in multiple resolutions, where the agent's sensors change in size to perceive a different Level of information. This type of adaptation is useful when an agent is :required to further validate the em~ironment in which it is found so as to increase or decrease its confidence of the structure it analyzes. Referring to Figure 9.a, the sensor data is input to eit:~er or both the Processor and the Neural Processor (66) where the data can be analyzed and processed, after which a response is sent to the Actuator that changes the size of the sensors (68) according to the later response. This process is repeated as required. This sensor adaptation is represented in Figure 9.b, where the initial sensor state is depicted by (T0), and where after adaptation the sensors are seen to be larger in size (7~), meaning that more image data is being read by the sensors. In one embodiment, this type of adaptation is used to perceive a larger image region to ensure the agent is precisely detecting a contour.
Agent N~otiation IO In order to resolve problems or contradictions as well as to facilitate the emergence of optimal solutions in the image segmentation process, t:he present system provides a means for Agent Negotiation. In a preferred embodiment, the agent negotiation is done by agents of different specialization that attempt to analyze the same image region. The objective is to allocate the image region to the agent that has maximum confidence on the IS segmented region. The process consists in first identifying the litigious region and for each agent to thereafter:
- Evaluate considered region according to specialization;
- Communicate with other agents in litigious state;
- ~7ote far winning agent.
~0 The first step of identifying the litigious region is achieved in one embodiment by a program that verifies if there is a plurality of agents that attempt to analyze a same image region. In another embodiment, an agent itself verifies if any other agents are analyzing the current region. In such case; the current agent evaluates the conf deuce that the said region is to be analyzed by itself: Follor-ving this evaluation, the agent communicates with 25 the other agents in the litigious region and asks for their confidence value. The communicating agent can thereafter award the said region to the agent with maximum confidence. In one embodiment, the conf deuce value is computed by the agent's Processor. In another embodiment, the confidence value is attributed by a classifier such as a neural network using the agent's Neural Processor, where the neural network's 30 activation value can be considered to weigh the confidence value.

Agent Adaptation. Collaboration and Reproduction To provide increased robustness az~d intelligence to the system, the devised agents within the present mufti-agent system have been incorporated means to dynamically adapt themselves to any image they analyze.
A first type of adaptation is t~~e context based agent adaptation, where an agent adapts its parameters and goals based on the information the agent gathers at a higher semantic or spatial scale or based on the information provided by another agent found at a higher semantic or spatial scale. °The agent first begins analyzing the image at its current scale level, then, to validate that the segmentation is precise and correct the agent gathers information from a higher scale which provides detail on the location and nature of the object being segmented. By having knowledge on the nature of the object, as provided from a higher scale analysis, the agent's behavior and actions can be dynamically modulated and adapted to improve the current segmentation, if required. As stated above, the gathering of information at a higher level can be achieved by the considered agent or by another agent at this higher level. In the embodiment of gathering higher Ieve1 information, the process involves the steps of 1) I~etertnining the scale level at which additional information should be gathered, 2) Proper agent initialization, 3) Information gathering, and 4) Agent adaptation. In a preferred embodiment, the adequate scale level is determined before-hand. 'The second step of agent initialization can take many forms, such as, without limitation, the agent moving itself to the higher scale, the agent cloning itself to the higher scale, and instantiating new agents (can be of different specialization) at the higher scale by reproduction or other means. In the first case where the agent at scale level i (74) moves to the scale Ievel n (76), the later analyzes the region of :interest at the higher scale to gain new knowledge on the feature oy- object being analyzed.
Referring to Figure IO.b, this process requires the agent to first identify the coordinate which it is currently at within the image (78), then; load the higher scale image representation from memory (80), set it as the current imaga~ of interest (82), begin analyzing the image at the previously identified coordinate (84), and finally update its memory and adapt its parameters (86}. In the other case, the agent clones itself at the higher level and continues the analysis through direct collaboration with its higher level clone(s). This is similar to th.e just above stated method, but differs with the aspect of collaboration. Instead of a sequential process where the agent gathers information then adapts itself, the cloning method provides a parallel and dynamic adaptation method where the lower level agent dynamically adapts itself from the feedback provided by its higher level clones. Referring to Figure 10.c, this method requires the agent to first instantiate a new agent of same specialization (88), copy its mental state and memory content to the new agent (90) so that the later becomes a clone (copy), position the clone at the proper coordinate at higher scale (92), have the cl~~ne analyze the image (94) and communicate the gathered information to the lower level agent (96). In a preferred embodiment, an agent can simply invoke its "clone" class method to clone itself, wherein the said class method is preprogrammed to generate a copy of an instance of an agent.
The other method provided for proper agent initialization is tl~e instantiation of new agents at the higher scale. As stated above, this can be achieved by simple instantiation or reproduction. In such case, new agents of same or different specialization will be instantiated and positioned at the proper coordinate on the higher scale image by the lower level agent. These new agents will thereafter analyze the image region and provide new information to the lower level agent by means oif communication. The overall process is identical to the previously defined method, wherein the only difference lies in the .type of agents that are instantiated. In the context of agent in.stantiation by means of reproduction, it is possible to instantiate new agents of specific specialization depending on the reproduction process of a specific agent. In the present invention, agent reproduction can be general or specialized, where for instance in one embodiment an agent specialized to detect object contours leas the possibility of reproducing itself in agents specialized in analyzing image background and object regions. ~y providing specialized reproduction schemes, agents have the possibility of collaborating and gaining new knowledge optimally and specifically for the tasks they are required to do.
Another type of adaptation is based on reinforcement. The agent begins a frst analysis, and then after a given number of pixels have been analyzed, the agent reiterates the analysis from the start. This reiteration allows for the agent to iteratively increase its knowledge on the information that is being analyzed and also offers the possibility of exploring differing analysis methods. This method allows an agent to incrementally S update its knowledge and understanding of the object or feature it is currently analyzing and can therefore optimally adapt its analysis parameters in accordance to its current knowledge. The adaptation involved can be with respect to the parameters relating to the Processor component or the tensors, where for instance the agent can modulate its noise sensitivity, adjust its threshold parameters, and vary the scale and resolution of its sensors.
Detailed description of the Invention in the preferred embodiment of Satellite Image Analysis A detailed description of the present invention will be further described hereunder using the example of detecting entities such as industrial buildings and bridges in satellite images, as well as further providing a symbolic interpretation of the image landscape.
Firstly, the system reads a digital satellite image by means of its input device, as depicted by numeral 2 in Figure 1. An illustration of a satellite image is depicted by Figure 11, which contains bridges, buildings, roads and water courses. Then, the Program (4) preprocesses the digital image with objective of extracting the predominant water 2U structures within the image which will thereafter serve as the initialization point for the multi-agent system. A possible method for extracting the main structures is to simply smooth the image at a high level by convolution with a (~aussian kernel followed by a threshold of pixels so as to oazly keep the pixels that are part of~ main water structures.
Then the ridges of these main structures are extracted by means of a ridge extraction operator. Figure 12.a depicts the extraction of the most predominant water courses along with their ridges. Referring to Figure 12.b, following these; preprocessing operations, the Program instantiates three agents specialized for the following of ridges, in accordance to the three ridge entrance points (98,100,102). The Program extracts the coordinates of these entrance points and assigns the agents respectively at these coordinates within the image. The objective of the ridge agents is to follow the ridge that serves as a path and thereafter collaborate with other agents if required. Once these agents have been initialized and assigned, the mufti-agent system becomes autonomous. In the next step, the ridge agents initialize be>rder agents that will analyze thc; border of the water structures. The border agents are not directly guided by a path but instead use their sensors to orient their selves in the proper direction. The border agents are specialized to respond to bimodal input from their sensors as they are expected to perceive pixel values within a specific range on one side and pixel values with a different intensity range on the other side. This is from the fact that pixels that are part of the water structures will have a different intensity then the land or other nonowater structures in the image.
Based on this cz praori knowledge, the border agent will be able to recognize water borders and move within the image in accordance, while at the same time segmenting the borders for thereafter allowing further analysis.
The next step consists in the agents to move within the :image and analyze its content.
Referring to Figure 13, during the ridge path following, the ridge agents will encounter a forking in the path ( 104,1 X6,108). ViThen a ridge agent identifies the problem, it generates offspring of same type in a number according to the number of branching paths.
These offspring will continue to follow ridges in an autonomous manner. At the same time, border agents analyze the image using their sensors and move along the water course borders. Refernng to figures 14.a and 14.b, the border agents will encounter a bridge (112) in their path which will be seen by the agent as a path discontinuity since the bridge line traverses the water and part of the bordering land (114). This discontinuity will be identified by the concerned agent (110) and will trigger that adaptation of the agent's sensors to verify the nature of this discontinuity. The agent's sensors will successively increase in size and scale representation until the border edge on the other side of the yet unknown discontinuity (116) is sensed by the agent. At this point, the agent reproduces itself in a new Line Agent that is initialized on the line discontinuity. The objective is to allow this new agent to gather more information on the latter. A.t this point, the Border agent has identified a discontinuity that crosses the border which provides an insight on the nature of the lattere can possibly be a building, a port, a. road or a bridge. The new line 3~? agent will provide the border agent with enough information to make the proper identification of the structure. Now that the line agent is initialized it begins the analysis and following of the line structure. This specific agent performs basic edge enhancement to compute the necessary information to move in the proper path direction and simultaneously analyses the pixel values on both sides of the line (118). By detecting and - analyzing these pixels, the agent can verify whether the pixels correspond to water or land. Since in the present case the pixels along the path correspond to water, the line agent communicates with the Border agent and informs it that the discontinuity structure corresponds to the entity "bridge". The line agent continues its path until it reaches the end of the line, where if it encounters another border agent ( 120), it can also inform it on the nature of the current line structure.
The next process in the analysis consists in providing a symbolic and semantic interpretation of the water Bourses. In such case, the agents are required to collaborate at various spatial scales in order to properly detect and understand the present structures, without being disrupted by noise, artifacts, or other dominant structures.
Relating to Figure 15, we can notice the presence of primary (122) and secondary (124) water courses that are to be detected, segmented and interpreted by the system.
First, in continuation with the previous processes, we see that the ridge agents come to a point where their paths meet (I26-128, 130-132). This contradiction triggers negotiation amongst the considered agents. In this case, as the agents correctly follow the same path, the negotiation will result in a fusion of both considered agents ( 126 and I28, 130 and 132). Referring to Figure l.S.b, the resulting fused agents (I34, 136) will contain the knowledge acquired throughout the analysis of both agents, which means that the agent will have in memory the path of the complete water course ridge. This fusion results in the generation of the symbolic concept of "same" mea~~ing that this path consists of a single water course. Now that there remains two fused ridge agents, another level of interpretation can be provided. Both agents (134 and 136) have the knowledge that there initially was a split in path and that the result was a fusion of agents since they were on a same path. From this info~nation, the agents' State Machines will provide the conclusion that both agents may be part of a same water course, and that the separation may be caused by an entity such as an Isle, platform, or ship, for instance. To validate this, both agents instantiate an agent specialized in detecting blab structures at high spatial scales.
Referring to Figure 16a, this is achieved through collaboration by first selecting a high spatial scale representation of the current image and instantiating both new agents (138, 140) at the coordinates of their respective "parents" in he new image. Their goal is to detect a blob structure ( 142) in the vicinity of their initial coordinates.
The two lower-scale parent agents restrict. the possible displacements of th.e higher-scale agents by confining them within the range of both agents (144), based on their knowledge that the structure is between them. Once initialized, the agents go about analyzing the region to detect and segment the "blob" structure. Further in the process, both blob agents will try -to segment the same pixels, which indicates that they are detecting the same structure.
From this, the lower-scale agents will be provided with the knowledge that this is in fact the same water course containing an entity. At this step, the system may chose to continue the blob detection in order to provide symbolic and semantic interpretation of this entity. In the present case, based on the shape and spectral intensity of the entity, the agents can conclude that it consists in an Isle or sand bank. In another scenario, this entity could be a ship, where in such case, the blob agent would detect that it is a ship based on the entity's specific shape by means of its state machine and memory unit. To achieve this identification, the agents are first required to precisely segment the structure and extract shape descriptors, such as maximum and minimum diameters, as well as spectral descriptors. By inputting this information in both or either their Neural Processor and State Machine, the agents will have the possibility of precisely identifying the analyzed ~0 structure and assign it a symbolic value, as mentioned previously..
Throughout these processes the border agents go upon segmenting the water borders.
Now that the ridge agents have identified a single water course, the bolder agents collaborate with the later to segment the whole water ccmrse. The objective is to relate both borders to the appropri<~te water course, to quantif~,~ and analyze the water course and to thereafter identify connecting courses. Quantifying and analyzing the water course requires the agents to simultaneously perform the analysis at multiple spatial scales so as to gather the most precise and robust information, allowing for further detecting structures within and on the periphery of the water course. First, one of the remaining ridge agents positions itself at the beginning of the ridge path and displaces itself at the same coordinate in a higher spatial scale representation. Its objective is to inform the agents to which ridge and water course they belong to and to instantiate Water Agents that will analyze the water course. From its higher scale perception of the structures, the ridge agent readily sees that the border agents directly belong to the same water course.
As the ridge agent moves along the ridge path it instantiates water agents.
These new agents are specialized to analyze water structures by means of their multiscale sensors.

.5 These agents' sensors simultaneously perceive the locally sensed pixels at multiple spatial scales. This multiscalc view generates a multiscale "signature9' that is directly analyzed by their Processor and Neural Processor components. The Neural Processor will classify the signatures according to these two classes wager and non-water.
This simple binary classification allows the agent to take the appropriate actions upon the classification output. If the Neural Processor identifies a signature as non-water, then the agent may be in the presence of an entity such as a ship. In a specific embodiment, the water agent can reproduce itself in agents specialized to segment such entities. The water agents are constrained by the borders previously segmented by the border agents, which prevents them from segmenting and analyzing structures that may have similar signatures 1S as water pixels. Furthermore, since the water agents sense the image at multiple scales, they are r~bust to noise present within the image. At the end of the process, the water agents will have segmented the complete water course and other entities will have been further segmented and analyzed by other agents. The collaboration of border agents and water agents also further allaws for the segmentation and identification of bordering structures such as buildings. In this case, the buildings will be seen as small discontinuities by the border agents, while at the same time perceived as non=water entities by the water agents. The water agents can therefore instantiate other specialized agents to segment and analyze these bordering entities. .fit the same time, the border agents can constrain the analysis domain of these new agents.
Overall, by letting evolve the previously described processes, the entire image and image structures will be precisely segmented, quantified and characterized. The agents will go upon achieving their goals and the overall autonomous collaboration will result in the complete quantification and understanding of the therein contained structures.
The final step of the overall process consists in storing the segmented image as well as the symbolic interpretation of every identified structure in a repository such as a database.
The assignment program can execute this task by accessing the information contained in memory and appropriately storing it in a database. This stored information can be used by other programs, such as data-mining programs and display program, and can be directly interpreted by users.

While the invention has been described in connection with specific embodiments thereof, it will be understood that it :is capable of further modifications and dais application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains and as may be applied to the essential features hereinbefore set forth, and as follows in the scope of the appended claims.
2,1

Claims (15)

WHAT IS CLAIMED IS:

1. A method for analyzing one or a plurality of digital images using a plurality of software agents, comprising:
- A program for initializing and assigning a plurality of agents to at least one of said images;
- Means for said agents to perceive at least one of said images at one or a plurality of scale representations; and - Means for said agents to modify pixel values of said images.

2. The method as claimed in claim 1, wherein each said agents further comprise:
- Sensors to locally sense information comprised in said images;
- Decision making means for allowing said agents to take action in accordance to specific mental states;
- Processing means for processing said information perceived by said sensors.

3. The method as claimed in claim 1, wherein said step of assigning further comprises the steps of:
- Preprocessing at least one of said images;
- Producing spatial coordinates from results of said step of preprocessing;
- Assigning said agents according to said spatial coordinates.

4. The method as claimed in claim 2, wherein said agents have the capability to further adapt said.sensors by changing configuration of said sensors.

5. The method as claimed in claim 4, wherein said step of changing configuration of said sensors comprises at least changing size of said sensors.

6. The method as claimed in anyone of claims 4 and 5, wherein said step of changing configuration of said sensors comprises changing resolution of said sensors.

7. The method as claimed in claim 1, wherein said means to perceive comprises:
- Analyzing said images at a first scale representation by means of at least one of said agents;
- Analyzing said images at a second scale representation by means of at least one other of said agents; and - Exchanging information between said agents at said first and second scale representations.

8. The method as claimed in claim 7, wherein said step of exchanging information further allows for at least one of said agents to adapt itself.

9. The method as claimed in claim 1, wherein said scale is a spatial scale.

10. The method as claimed in claim 1, wherein said scale is a semantic scale.

11. The method as claimed in any one of claims 2, 4, 5,6, wherein said sensors provide a spatial multiscale representation of said image, wherein said multiscale representation simultaneously provides said agents with image information at every considered spatial scale.

12. The method as claimed in claim 11, wherein said multiscale information provides a multiscale signature and wherein said signature further allows said agent to classify said information.

13. The method as claimed in claim 12, wherein said step of classifying information is achieved by means artificial neural networks.

14. A method for contour recognition in one or a plurality of digital images by means of one or a plurality of software agents, comprising the steps of:
- Initializing and assigning at least one of said agents to an image, said agents comprising at least one sensor;
- Said agents recognizing said contours in said image by means of said sensors; and - Said agents drawing said recognized contours in said image.

15. An apparatus for analyzing one or a plurality of digital images, comprising:
- Image data input means;
- At least one processing node, said processing node comprising memory and processing unit;
- A method for analyzing said images, said method as claimed in claim 1; and - Display means for displaying said images.