JP4474577B2 - Experience mapping device - Google Patents

Description

  The present invention relates to a configuration of an experience mapping apparatus for reconstructing recorded information such as video accompanying a user's experience and presenting the content experienced by the user to the user or another user.

  With sensor elements and microprocessors becoming smaller and cheaper and the development of microfabrication technology, it is going to be practical to place networked sensor groups, so-called “ubiquitous sensors”, in everyday living spaces. Yes.

On the other hand, there have already been several reports on applied technology using such “ubiquitous sensor” technology (see, for example, Non-Patent Document 1).
Tetsuya Matsuguchi, Yasuyuki Sumi, and Kenji Mase. "Deciphering interactions from spatio-temporal data." ISPJ SIGNotes Human Interface, (102), 2002.

  In the environment where the “ubiquitous sensor” as described above is realized, it is possible to sense and record various activities and experiences of users who act in the environment.

  In that case, for example, when the user wants to look back on his / her activity and talk about his / her past experience, he / she assists in visually recalling his / her own experience based on the data recorded as described above. Useful if possible.

  An object of the present invention is to provide an experience mapping apparatus that can systematize visual / audio information experienced by a user and present it to a user as an experience map.

In order to achieve such an object, the experience mapping apparatus of the present invention has a range corresponding to an observer's visual field in an area where a plurality of objects each having an optically identifiable identification mark are arranged. the imaging means is mounted to image, comprising a storage device for storing a photographed image information associated with the sequence information on the time axis the image includes sequential photographing an object being in accordance with actions of the observer, each predetermined For each first time slice having a time interval, the object for which the identification mark is detected so that the object for which the identification mark is detected in the captured image information can be distinguished from the object for which the identification mark is not detected. By assigning a specific value to, when defining the observation vector for the first time slice, from the set of observation vectors, it is temporally over a time longer than a predetermined time. Extracting means for extracting the mass at which observation vector chunks of the observation vector in contact, based on the similarity between the mass observation vector representing the observation vector chunks, video clips representing, respectively observation vector chunks among the captured image information By associating each with an attractive force or a repulsive force, video clips representing different groups of observation vectors on the time axis correspond to groups of objects with which the observer interacts repeatedly in time , forming clusters. Display means for displaying as described above.

Preferably, the lump observation vector is a logical sum of the elements of all the observation vectors included in the observation vector lump, each element corresponding to the lump observation vector.
Preferably, the extraction unit collects the first time slice so as to be the second time slice having a certain time interval longer than the predetermined time, and thereby, from the set of observation vectors, for each second time slice. An observation vector block that is a block of observation vectors is extracted .

Preferably, the extraction unit collects first time slices that are close in time corresponding to continuous observation of the same object, thereby obtaining an observation vector block that is a block of observation vectors from the set of observation vectors. Extract .

Preferably, the attractive force or the repulsive force has a predetermined effective distance, and further includes an input means for the user to select a video clip and input a signal indicating a destination, and the display means is a video selected by the user. in response to moving between other video clips by moving a clip along with a video clip repel each other with a repulsive to each other at the effective distance, form a video clip clusters with mutually attractive force within the effective distance To display.

  Embodiments of the present invention will be described below with reference to the drawings.

[System configuration of the present invention]
FIG. 1 is a conceptual diagram showing an example of the experience capture system 20 of the present invention.

  As shown in FIG. 1, the original observer 2 is wearing a portable experience capture system 20. The experience capture system 20 includes a wearable (hereinafter referred to as “wearable”) computer 100, a video camera (including an infrared-sensitive camera) 142, and a microphone (not shown in FIG. 1).

  Further, in an environment, an object that is an object of interaction is previously provided with an LED (Light Emitting Diode) tag 10.1-10. It is assumed that n (n: natural number) is attached. LED tag emitting infrared rays (hereinafter referred to as “IR tag”) 10.1-10. Each of n emits a unique identification signal by infrared rays. Such a unique identification signal can be read by the above-described infrared-sensing camera (hereinafter referred to as “IR camera”).

  The experience capture system 20 uses the audio data collected by the microphone, the video data captured by the video camera 142, and the IR tag data captured by the IR camera throughout the data acquisition period (capture period). Record. Only IR tags that can be optically detected within the field of view of the IR camera are recorded in the system 20 at the point where the observer 2 is present.

  The computer 100 is not particularly limited. For example, the computer 100 can be configured to be connected to a LAN (Local Area Network) wirelessly.

[Hardware configuration of experience capture system 20]
FIG. 2 is a block diagram showing the configuration of the experience capture system 20.

  Referring to FIG. 2, a computer 100 of the experience capture system 20 includes an external medium drive 116 for reading information on an external medium 150 such as a memory card, and a CPU (Central Processing Unit) 110 connected to the bus BS1. A memory 112 including a ROM (Read Only Memory) and a RAM (Random Access Memory), a direct access memory device, for example, a hard disk 114, and a communication interface 118 for exchanging data with the LAN using wireless connection or the like And a timer 122 for providing the time used by the system.

  The computer 100 further includes an external device interface 120 for receiving data from the input device 160 that is connected to the computer 100 as needed and used for data input by the user. The external device interface 120 receives image data and audio data from the video camera 142 and the microphone 144 attached to the headset 140 attached to the head of the observer 2. Furthermore, the headset 140 may be provided with a headset display 146 for displaying an image from the video camera 142 to the observer 2 as necessary. Further, it is assumed that the video camera 142 is provided with not only a camera system for capturing a visible light image but also an IR camera system for capturing an infrared image as described above.

  The external medium 150 is a medium capable of recording information such as programs installed in the computer main body. In addition, the captured data can be recorded as necessary.

  Further, by using the hard disk 114 or other large-capacity external medium as the image data recording medium, the computer 100 in the experience capture system 20 can basically be a general-purpose mobile computer. Of course, the computer 100 in the experience capture system 20 may be configured with dedicated hardware instead of a general-purpose mobile computer.

[Hardware configuration of experience mapping apparatus 1000]
FIG. 3 is a conceptual diagram showing an example of the experience mapping apparatus 1000 of the present invention.

  Referring to FIG. 3, experience mapping apparatus 1000 includes a computer 300 for accessing voice data and image data experienced by observer 2 based on the operation of observer 2 or a third party.

  Referring to FIG. 3, a computer 300 includes a CD-ROM drive 308 for reading information on a CD-ROM (Compact Disc Read-Only Memory) and an external medium drive for reading / writing information from / to an external medium 150. A computer main body 302 provided with 306, a display 304 as a display device connected to the computer main body 302, a keyboard 310 and a mouse 312 as input devices also connected to the computer main body 302, and a microphone 332 as a voice input device. And a speaker 334 as an audio output device.

  FIG. 4 is a block diagram showing the hardware configuration of the computer 300. As shown in FIG.

  As shown in FIG. 4, in addition to the CD-ROM drive 308 and the external medium drive 306, the computer main body 302 constituting the computer 300 includes a CPU 320 connected to the bus BS2 and a memory 322 including ROM and RAM, respectively. And a direct access memory device such as a hard disk 324 and an interface 328 for exchanging data with the microphone 332 or the speaker 334.

  The interface 328 is a server that stores the data acquired by the experience capture system 20 so as to be accessible via the LAN, for example, directly with the experience capture system 20 via the LAN 410 or depending on the system configuration. It can also be used to communicate with (not shown).

  In the following description, when the experience mapping apparatus 1000 operates, the data acquired by the experience capture system 20 is described as being stored in the hard disk 324 via the LAN or the external medium 150. .

  Note that the CD-ROM 318 may be another medium, such as a DVD-ROM (Digital Versatile Disc), as long as it can record information such as a program installed in the computer main body. The computer main body 302 is provided with a drive device capable of reading such other media.

  The main part of the experience mapping apparatus 1000 of the present invention is composed of computer hardware and software executed by the CPU 320. Generally, such software is stored and distributed in a storage medium such as a CD-ROM 318, read from the storage medium by a CD-ROM drive 308 or the like, and temporarily stored in the hard disk 324. Alternatively, when the device is connected to the network, it is temporarily copied from the server on the network to the hard disk 324. Then, the data is further read from the hard disk 324 to the RAM in the memory 322 and executed by the CPU 320. In the case of network connection, the program may be directly loaded into the RAM and executed without being stored in the hard disk 324.

  The computer hardware itself and its operating principle shown in FIGS. 3 and 4 are general. Therefore, the most essential part of the present invention is software stored in a storage medium such as the CD-ROM 318 and the hard disk 324.

As a general tendency, various program modules are prepared as a part of a computer operating system, and an application program generally calls a module in a predetermined arrangement and advances the processing when necessary. In such a case, the software itself for realizing the experience mapping apparatus 1000 does not include such modules, and the experience mapping apparatus 1000 is realized only when the computer cooperates with the operating system. However, as long as a general platform is used, it is not necessary to distribute software including such modules, and the software itself not including these modules and the recording medium storing the software (and the software distributes on the network). Data signal) can be considered to constitute the embodiment.
(Outline of operation of the present invention)
As will be described below, the purpose of the experience capture system 20 and the experience mapping apparatus 1000 for presenting data captured by the experience capture system 20 is to automatically or semi-automatically capture the mass Are classified into “situations”, and can be searched after the fact regardless of time. Such classification is performed because a large amount of captured raw data is not efficient to search for the captured data of interest manually and along the time axis.

  Also, the experience mapping apparatus of the present invention makes it possible to access other related captured data based on the “similarity” relationship between captured data, as will be defined later, and to store the memory of the observer 2 For the purpose of assisting the recall, the “situation” classified so as to have the relationship of “similarity” and the captured data are correlated with each other. Such a relationship of “similarity” is related to the ubiquitous IR tags 10.1 to 10.6 attached to the object in the environment where the observer 2 is active. By cooperating with n, observation data captured by the head-mounted camera are correlated with each other.

  By making it possible to measure such “similarity”, the present invention provides a new classification method called “classification based on the concept of“ situation ”” without being bound by the time axis, and is captured. Video, audio and other data can be accessed.

(Details of operation of the present invention)
In the present invention, the above-mentioned “situation” is defined from the viewpoint of the object, and the captured data is divided into discrete “chunks”, and similarities are observed between the observation vector and the chunks. A degree metric is defined and ultimately such a similarity metric is used to form a “situation” (group of chunks).

(1. Definition of “situation” from the viewpoint of the object)
In the present invention, a “situation” is defined as an object (or group of objects) that is repeated in time and observed by the observer 2. Such a group of objects is one in which an observer repeatedly interacts over a period of time.

  An object is referred to as “interacting” when its tag is taken by an IR camera at each moment on the time axis. Such a group of objects is considered to have significance in terms of the entire experience of the observer 2 by repeating such interaction.

(2. Divide the capture data into discrete chunks)
A portion of the captured data is provided by a head-mounted IR camera, and such a portion identifies which IR tag is present in the observer's 2 field of view at some point in time.

  FIG. 5 is a diagram showing a part of the capture data acquired in this way.

  As shown in FIG. 5, a part of captured data that identifies an object to be observed based on an IR camera is, in the form of its raw data, a list in which objects that the user interacted with are arranged on a time axis. It has become. One section from when the observation object in the field of view begins to exist until the object disappears from the field of view is referred to as a “record”. In addition, since image data and audio data are acquired in parallel, the observation of the object is associated with the image and audio data via the time data of the start time and end time of this record.

  In other words, at a given moment, the user is looking at a set of objects and, in a sense, interacting with such a set of objects. And these objects are IR tags 10.1-10. can be identified by n. Each IR tag 10.1-10. As described above, n has a unique integer identifier.

  FIG. 6 is a graph showing the observation state of one observer 2 according to time. In FIG. 6, the vertical axis represents IR tags 10.1-10. The identification number of n is shown, and the horizontal axis shows the elapsed time of observation.

  In this situation, from the viewpoint of sampling data, it is more convenient to focus on a small time interval (time slice, for example, but not limited to 1 second) rather than considering an infinitely small time. It is. The set of all objects observed by the user during a time slice is called the observation vector in that time slice.

  FIG. 7 is a conceptual diagram showing a specific configuration of the observation vector.

  In the “observation vector”, an element corresponding to an object to be observed is “1” and an element corresponding to an object not to be observed is “0” in a time slice. It is a vector like this. Of course, as long as it is a finite value, the value of the element corresponding to the object to be observed may be a value other than 1.

  In FIG. 6, one example of an observation vector is shown by a vertical rectangle. As can be seen from FIG. 6, the observation vector includes the identifiers of all IR tags that the observer 2 is viewing in the time slice.

  In FIG. 6, in a vertical rectangle corresponding to one observation vector, each position either contains one dot or is empty. The inclusion of a dot indicates that the object represented by that position in the observation vector is being observed in that time slice. On the other hand, being empty means that the object represented by the position in the observation vector is not observed in the time slice.

  In FIG. 6, a change in the set of observed objects can be found as time progresses. In this way, the observation vector changes with time. Furthermore, it can be seen that there are certain patterns that are repeated in the observation vector. This expresses that the user is in such a “situation”.

  After the observation along the time axis as shown in FIG. 6 is performed, all sets of observation vectors are divided into “chunks”. One block corresponds to a sequence of observation vectors that are close in time. The operation of extracting such a lump can be performed by the following two methods.

  FIG. 8 is a diagram showing a first method for extracting such a lump.

  As shown in FIG. 8, one method of extracting a lump is to perform lump extraction at the level of a time slice (in this case, although not particularly limited, for example, 5 seconds). In other words, the captured observation data is divided into small time slices of the same size, and each time slice is made into one lump that is used in the next step. The advantage of this method is that such processing is simple and that raw data can be processed directly.

  FIG. 9 is a diagram showing a second method of extracting such a lump.

  As shown in FIG. 9, the second method for extracting chunks divides the acquired observation data into temporally close chunks corresponding to observations of the same object, rather than time slices of the same size. That's what it means.

  In this second method, the user typically interacts with a particular set of objects for a period of time, which means that the observation vectors are identical or similar to each other. This corresponds to the extraction of a lump assuming that the period of time represents a part of a “lump” representing one “situation”. And the interaction with other different sets of objects corresponds to the expression of a “chunk” of another situation. The advantage of such a method is that it does not split temporally adjacent “lumps” corresponding to the same or similar observations, increasing the temporal consistency of the clustered parts. This can increase the “semantic” value of the units in each cluster.

  In this way, by extracting “lumps” of observation vectors that are temporally adjacent to each other (hereinafter referred to as “observation vector chunks”), as a set of “observation vector chunks” that are the same or similar to each other, Can be defined.

  The second method of extracting observation vector blocks requires comparison of observation vectors. The contents will be described below.

(3. Measurement of similarity between observation vectors and between observation vector chunks)
The similarity metric is necessary to compare the similarity between one observation vector and another observation vector.

  Further, a similar similarity metric method defines a “bulk observation vector” corresponding to one observation vector block as a logical sum of all the observation vectors included in the observation vector block, It can also be used to compare observation vector clusters.

  For example, when given observation vectors are (0, 1, 0, 1) and (0, 0, 1, 1) in one observation vector block, the logical sum of these two vectors is (0 , 1, 1, 1). This is a block observation vector corresponding to the entire observation vector block.

  The observation vector for a specific time interval is a binary vector indicating whether or not the object is observed for all objects that can be considered. The following similarity measure can be used to compare such observation vectors.

(3-1. Similarity based on the number of common components)
In this comparison method, the number of identical elements between two observation vectors is counted in order to obtain a similarity score. The score thus obtained is divided by the number of elements to obtain a normalized similarity score.

(3-2. Similarity based on the number of different components)
This simply corresponds to obtaining a complementary score of the similarity score based on the number of 3-1 common components.

(3-3. Tanimoto similarity)
Literature:. ". Self-Organizing Maps " Teuvvo Kohonen Springer-Verlag Berlin Heidelberg, 1995, as shown in Pp.16-17, metering of similarity S _T proposed by Tanimoto has two observation vector a , B can be defined as follows.

S _T (a, b) = (a · b) / ((a · a) + (b · b) − (a · b))
Here, (a · b) represents the inner product of the vectors a and b.

  This Tanimoto similarity essentially corresponds to finding the ratio of the number of common elements to the number of different elements.

(3-4. Weighted similarity measure)
The similarity metric as described above means a metric value that is not weighted. In other words, all elements in each vector are treated with the same weight. The problem with using a similarity vector that is not weighted is that the similarity between elements in one observation vector cannot be considered.

  The element of the observation vector is formed based on an IR tag that can be observed by the head mounted IR camera. IR tags that are spatially close or semantically close can be interchanged. For example, if one object has two IR tags a and b mounted on it, the similarity between the two vectors (a, 0) and (0, b) will be very high. . This is because these tags are very close in meaning.

  However, if the tags a and b are spatially or semantically separated, for example if they are mounted on or physically separated from different objects, (a, 0 ) And (0, b) have a low similarity.

  Therefore, if elements of such observation vectors are spatially close or semantically close in advance, these elements are common between the two observation vectors. Can be weighted to increase the similarity.

  Further, in the calculation of the inner product in the Tanimoto similarity, for example, when the element corresponding to the tag a is “1”, the element corresponding to the tag b is also “1” regardless of the raw data of observation. By calculating the inner product, the similarity can be weighted.

  Pre-assigning weights as described above based on the spatial or semantic content of tags a and b allows for individual fine-tuning of the metrics for comparison in individual specific cases. It is what.

(Formation of “situation (chunk group)” using similarity measure)
When an observation vector block is extracted based on the similarity between observation vectors as described above, each observation vector block is a series of experiences that contain some common information that has been observed during a certain period of experience. Represent a lump of.

  Since each observation vector block is generated at an identifiable time, audio or video data can be associated with each observation vector block.

  Further, the similarity between the observation vector blocks can be compared based on the above-described similarity of the block observation vectors.

  If these pieces of information are given, groups of similar observation vector clusters can be formed. Each group of observation vector clusters similar to each other corresponds to a “situation”. This corresponds to the fact that “situation” means “a group of objects in which it is repeated in time that an observer interacts over a certain time interval”.

  The similarity between observation vector chunks can be defined by the similarity metric described in the steps as described above. When a similarity metric is given, a well-known method can be used as a method for clustering video clips corresponding to observation vector chunks.

  For example, it is possible to use a clustering method based on a self-organized map or an attractive or repulsive magnetic model.

  That is, in the latter case, for example, for each of the clips of video and audio data (referred to as “video clips”) corresponding to a predetermined time slice, attraction works between the clips corresponding to the same situation, and corresponds to different situations. When a two-dimensional display is performed on the assumption that repulsive force acts between clips, clips corresponding to the same situation are clustered on this display.

  FIG. 10 is a diagram illustrating an example of clustering display (map display) using a magnetic force model that performs interaction.

  In FIG. 10, each video clip is represented by a still image represented.

  Although not particularly limited, in the example shown in FIG. 10, the force acting between the video clips is limited to act within a predetermined radius in order to form a cluster. This has the disadvantage that a cluster of video clips is not automatically formed as a whole, but it is possible to perform a reciprocal correlation search between the dragged data. It has the advantage of being possible. Of course, the video clip may automatically generate a cluster corresponding to the situation without limiting the effective radius of the force acting between the video clips. The force acting between the video clips corresponding to the similarity is classified as attractive if the similarity is higher than a certain threshold, and repulsive if the similarity is lower than a certain threshold.

  Data consisting of observation vectors and video clips (experience corpus data) corresponding to the experience is divided into time slices having the same size. For each time slice, the observation vector and the head-mounted camera are used. A video frame representing the moving image corresponding to each time slice among the moving images (for example, the video frame at the center of the time slice) is extracted.

  As shown in FIG. 10, the video frames are initially randomly distributed on the two-dimensional map. Later, the interaction of attractive forces brings together similar observations to form each “situation”.

  Each video frame has an observation vector associated with it. That is, the object observed during the period of the experience is acquired by the IR tag sensor, and the object is associated with the observation vector.

  In the experience map shown in FIG. 10, it is assumed that the force acting on each frame has a radius that can be affected, and the similarity metric of Tanimoto is close to the frame under consideration and the adjacent influence range. It is used to measure the similarity between all frames. The user can drag and move any frame on the map, and can observe how the dragged frame attracts or repels other frames. Then, the user can promote that the clustering is performed semi-automatically by moving all the frames existing in the map by a minute amount.

  In FIG. 10, the layout is interactively formed by dragging the frames in the map, and similar frames are clustered with each other. This is because an attractive force is generated depending on a similar observation vector. Such a map can therefore be searched in a two-dimensional space by a force with a limited effective or repulsive effective radius acting on an observation vector visualized as a video frame. In the example of FIG. 10, each still image represents one time slice, and video clips corresponding to similar observation vectors are mutually related even if they are not temporally related. Inquire and form a cluster of situations. Video clips that correspond to different situations repel each other. For example, if the lower left image in FIG. 10 is dragged and moved to the right cluster (indicated by an arrow in the figure), these observation vectors are dissimilar, so the cluster Will not be a single cluster.

  Visualization of such a group of observation vectors can access and view image data captured in a situation-based manner without being along the time axis as shown in FIG. 10, for example. It can be used as an interface. It can also be used to search for similar relationships in captured image data. The similarity relationship between observation vector chunks can also be used as a memory recall aid to remind the user of a captured video clip that is similar to the particular video clip under consideration. .

  In the description of FIG. 10, the video frame representing the moving image is extracted in the case where the acquired observation data is divided into time slices of the same size. However, the time frame corresponding to the observation of the same object is used. It is also possible to divide into chunks adjacent to and extract a video frame representing a moving image of the chunk.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

It is a conceptual diagram which shows an example of the experience capture system 20 of this invention. It is a figure which shows the structure of the experience capture system 20 in a block diagram format. It is a conceptual diagram which shows an example of the experience mapping apparatus 1000 of this invention. FIG. 2 is a block diagram showing a hardware configuration of a computer 300. It is a figure which shows a part of capture data acquired. The observation state of one observer 2 according to time is shown by a graph. It is a conceptual diagram which shows the specific structure of an observation vector. It is a figure which shows the 1st method of extracting a lump. It is a figure which shows the 2nd method of extracting a lump. It is a figure which shows the example of the clustering display (map display) using the magnetic force model which performs interaction.

Explanation of symbols

  2 observers, 10.1-10. n IR tag, 20 experience capture system, 100 computer, 110 CPU, 112 memory, 114 hard disk, 116 external media drive, 118 communication interface, 120 external device interface, 122 timer, 140 headset, 142 video camera, 144 microphone, 146 Headset type display, 150 external medium, 300 computer, 302 computer main body, 304 display, 306 external medium drive, 308 CD-ROM drive, 310 keyboard, 312 mouse, 318 CD-ROM, 320 CPU, 322 memory, 324 hard disk, 328 interface, 1000 experience mapping device.

Claims (5)

In an area where a plurality of objects each having an optically identifiable identification mark are arranged, an imaging means attached to image a range corresponding to the visual field of the observer according to the behavior of the observer A storage device for storing captured image information in which images including the objects to be sequentially captured are associated with sequence information on a time axis;
For each of the first time slices each having a predetermined time interval, the identification is performed so that the object in which the identification mark is detected and the object in which the identification mark is not detected can be distinguished in the captured image information. When defining an observation vector for the first time slice by assigning a specific value to the object for which the sign is detected,
Extraction means for extracting from the set of observation vectors an observation vector block that is a block of the observation vectors that are temporally adjacent over a time longer than the predetermined time ;
Based on the similarity between the mass observation vectors representing the observation vector mass, the observer can associate an attractive force or a repulsive force with each video clip representing the observation vector mass in the captured image information. The video clip corresponding to the group of objects interacting repeatedly in time and representing the different observation vector clusters on the time axis, and display means for displaying to form a cluster, Experience mapping device. The experience mapping apparatus according to claim 1, wherein each of the lump observation vectors is a logical sum of the elements of all the observation vectors included in the observation vector lump corresponding to the lump observation vector. The extraction means collects the first time slice so as to be a second time slice having a certain time interval longer than the predetermined time, thereby obtaining the second time slice from the set of observation vectors. The experience mapping apparatus according to claim 2 , wherein an observation vector block that is a block of the observation vectors is extracted . The extraction means collects the first time slices that are close in time corresponding to continuous observation of the same object , so that an observation vector block that is a block of the observation vectors is collected from the set of observation vectors. The experience mapping device according to claim 2, wherein the device is extracted . The attractive force or repulsive force has a predetermined effective distance;
An input means for the user to select the video clip and input a signal indicating the destination;
It said display means in response to said user to move between the other of the video clip by moving the video clip selected, together with the video clip with repulsive to each other at said effective distance repel each other, The experience mapping apparatus according to claim 2 , wherein the video clips that are mutually attractive within the effective distance are displayed to form the cluster.