JPH11120162A - Thought support system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a thought support system which provides a support environment of thought by exteriorizing user's thought concept and operating it with a finger. SOLUTION: A parallel processing computer 10 in a thought support system simulates the relation between conceptual elements which constitute conceptual space with a spring system of material points model. A conceptual structure and a finger icon which moves, interlocking with the movement of a finger are presented as a view point position tacking time division stereoscopic picture on a projector screen 6. By dynamic sense presenting devices 2 and 4, a user can directly operate conceptual elements on virtual space by using the finger icon. Also, connection strength among the conceptual elements is intuitively presented as operational reaction force to the user through the devices 2 and 4.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a thinking support system, and more particularly to a thinking support system that supports a thinking activity by a user's hand operation.

[0002]

2. Description of the Related Art Conventionally, there is a KJ method as one of methods for supporting human thinking activities. According to the KJ method, concept elements (expressions of ideas and pieces of knowledge in words) are put together on a card, and the ideas are put together while the user himself composes an array of them in the course of thinking.

[0003] At present, research is being conducted on a thinking support system that supports a thinking activity of a user by automatically arranging conceptual elements using a computer.

"Thinking support system by spatially arranging text objects" by Kado, Hori, Osuka et al.
Journal of the Japanese Society for Artificial Intelligence, Vol. 9, No. 1, pp. 139
147, (1994) disclose a thinking support system CAT1, which is one of them.

Further, "Communication Support Method by Visualizing Thinking Space" by Kaku, Ogawa, Hori, Osuka, Mase, et al., IEICE Transactions A, Vol. J79-
A, No. 2, pp. 251 to 260, (1996) disclose a thinking support system CSS that inherits the results of the thinking support system CAT1.

[0006] Tamura, "Concept Map Generation System SPRINGS Based on Intersymbol Dynamics", IPSJ Transactions, Vol. 33, no. 4, pp. 465-4
70, (1992), discloses a concept map generation system SPRING.

The flow of thinking support based on a thinking support system using these computers will be briefly described with reference to FIG.
In the following, the thinking space is referred to as "a mental space composed of fragments of ideas and knowledge raised in consciousness due to thought activities and their connection", and the conceptual structure is referred to as "representing similar concepts "A spatial knowledge expression in which symbols are arranged in the spatial vicinity".

FIG. 6 is a conceptual diagram showing the flow of thinking support in a conventional thinking support system using a computer. As shown in FIG. 6, first, a concept element group having some relation to obtain a new idea is defined as an object, and the whole relation is extracted by a computer from those feature amounts (step S1 in FIG. 6). Then, a conceptual structure is geometrically displayed on a two-dimensional plane (step S2 in FIG. 6). Using this result, the user can obtain a new idea from the existing concept (step S3 in FIG. 6).

[0009]

As described above, in any of the conventional thinking support systems, the evaluation of the conceptual structure displayed geometrically and the new thinking from this conceptual structure (steps in FIG. 6). S3) is completely left to the user.

That is, although the conventional thinking support system attempts to model and visualize the conceptual structure, the user must perform all the tasks of developing thinking using the results.

Further, the operation of the conceptual structure in the conventional thinking support system is merely an operation of a parameter which is indirectly related to spatial information such as a keyword or similarity of an object, and is being integrated in, for example, a thinking process. It was difficult to manipulate only part of the conceptual structure.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and has as its object to embody a conceptual structure, to directly operate the embodied conceptual structure and to realize a new conception process from an existing concept. An object of the present invention is to provide a thinking support system that supports information presentation.

[0013]

According to a first aspect of the present invention, there is provided a thinking support system comprising: a materializing means for embodying a conceptual structure in a thinking space; and a user manually operating the materialized conceptual structure in a virtual space. The conceptual structure is composed of a plurality of conceptual elements, and the concrete-forming means visually presents the concrete-structured conceptual structure in the virtual space in accordance with the viewpoint position of the user. It includes a presentation unit and a force sense presentation unit that presents, to the user, an operation reaction force based on the strength of connection between conceptual elements to be operated in response to an operation in the virtual space.

According to a second aspect of the present invention, there is provided the thinking support system according to the first aspect, wherein the plurality of conceptual elements are conceptual elements in a virtual space, and the connection between conceptual elements is defined as a spring. Is expressed by simulation.

[0015]

BEST MODE FOR CARRYING OUT THE INVENTION

[First Embodiment] A thinking support system according to a first embodiment of the present invention will be described.

The thinking support system according to the first embodiment of the present invention embodies the concept (non-shape) of the user's thinking as multi-sensory information, and operates the embodied thinking concept to perform the thinking activity of the user. (S3 shown in FIG. 6).

In the first embodiment, the concept structure is regarded as a structure in which concept elements defined from text and accompanying keywords are combined in a net-like manner, and the relation between the concept elements, which are constituent elements, is represented by a spring mass point model. Simulate. Then, this spring mass point model is visually and forcefully reproduced in the virtual space. As a result, the relationship between intangible conceptual elements is presented to the user, and further, the conceptual structure can be directly operated.

An example of the configuration of the thinking support system according to the first embodiment of the present invention (hereinafter referred to as thinking support system 1000) will be described with reference to FIG.

FIG. 1 is an overall configuration diagram showing a specific example of a thinking support system 1000 according to Embodiment 1 of the present invention. The thinking support system 1000 includes the haptic device 2
And 4, a VR system comprising a projector screen 6 which is a three-dimensional image presentation device and a head position tracking device 8.

The projector screen 6 is, for example,
This is a rear projection type diffusion display surface on which a time-division stereoscopic image is projected. Specifically, the conceptual structure is displayed in a three-dimensional space, and a hand icon (not shown) that is linked to the movement of the user's hand is displayed. In the following, the concept space means a space in which the concept structure is arranged, that is, a space in which concept elements are arranged as mass points. In the first embodiment of the present invention, the concept space is represented by a three-dimensional linear space.

The head position tracking device 8 measures the position and posture of the user's head. Based on the measured user's viewpoint position, the appearance of the conceptual structure from the viewpoint position is reproduced on the projector screen 6.

The user directly operates the conceptual structure in the virtual space using the haptic devices 2 and 4. Specifically, by operating the grips of the haptic devices 2 and 4, the hand icon displayed in the virtual space is moved. Further, by superimposing the hand icon on the conceptual element and operating a switch (not shown) at the tip of the grip, the conceptual element is gripped and released. The applicant has
"Cooperative object operation device (Japanese Patent Application No. 9-60973)"
Such a haptic device has been proposed.

The user can change the position and speed information of the grasped conceptual element. In addition, the force reaction presentation devices 2 and 4 forcefully present the operation reaction force to the user. Here, the operation reaction force refers to the strength of the connection between conceptual elements, and in the first embodiment of the present invention, as described later, the strength of the connection between the mass points in the spring mass model and the viscosity obtained from the space. Embody as power. The degrees of freedom of the haptic devices 2 and 4 are 6 (reaction forces in the X, Y, and Z axes and rotational torque around each axis), and these operation reaction forces are independently presented to the user.

As described above, the thinking support system 1000 according to the first embodiment of the present invention includes two force sense presentation devices, and is configured so that the user can operate the conceptual structure with both arms. As a result, it is possible to realize a natural operation environment in which one hand holds a reference conceptual element and the other conceptual element is arranged while continuously comparing the feels of other conceptual elements. .

The thinking support system 1000 further includes a parallel processing computer 10 and an I / O controller 14.

The processing unit in the parallel processing computer 10 is composed of a control system of the force sense presentation devices 2 and 4, a simulation system using a spring mass point model, and a graphic processing system. More specifically, the local simulation unit 20, the plurality of spring mass point simulators 22, the main process unit 24, and the drawing process unit 26
Is provided.

The I / O controller 14 is a low-level control device for the hardware of the haptic devices 2 and 4, and is connected to the parallel processing computer 10 via the signal line LINE1.

The local simulation section 20 performs a high-level simulation of the haptic devices 2 and 4 that require high-speed calculations.

The plurality of spring mass simulators 22 are simulators for simulating a conceptual structure of a complicated network structure with a dynamic system model. By integrating calculation results of individual simulations, a plurality of The structure is embodied.

The main processing unit 24 integrally performs the simulation of the spring mass point system.

The drawing process unit 26 displays, based on the position and orientation of the user's head measured by the head position tracking device 8, the appearance of the conceptual structure as viewed from the measured user's viewpoint on the projector screen 6. Perform processing to reproduce.

The multi-thread process unit 28 performs a process (process division) for executing each of the plurality of spring mass point system simulators 22, the main process unit 24, and the drawing process unit 26 in parallel. This makes it possible to reproduce the natural movement of the spring mass point model in real time processing.

Next, a process of simulating the conceptual structure in the first embodiment of the present invention using a spring mass point model will be described with reference to FIG.

FIG. 2 is a conceptual diagram for explaining the correspondence between the conceptual structure and the spring mass point model in the first embodiment of the present invention.

As shown in FIG. 2, in the virtual space, the conceptual structure is composed of a plurality of mass points and a spring connecting them (FIG. 2).
The symbol SP) in the above. The mass points in FIG. 2 correspond to concept element i, concept element j, concept element k, concept element m, and concept element n, respectively. A spring in the virtual space is defined only from the similarity between two conceptual elements. Concept elements (mass points) are respectively connected by springs. Here, the similarity is a similarity between two conceptual elements that is subjectively given by a numerical value.

When there are a plurality of conceptual elements, the user arbitrarily sets only the pairs whose similarities can be set subjectively.

As described above, based on the information of the similarity set between a plurality of concept elements, a network structure as a spring mass point system is defined, and a dynamic simulation is executed. As a result, the entire dynamic structure converges to a state where the elastic energy is locally minimized.

Considering this converged state in the concept space, it is shown that the relatively close concept elements are similar concepts while locally minimizing the overall similarity distortion. .

Further, when the user operates the concept structure (that is, gives a disturbance to the concept space), the user may converge to another minimum solution. Therefore, the user
It is possible to evaluate the state of conceptual elements in the conceptual structure while temporarily applying external distortion to the conceptual structure. In addition,
The symbols in the figure will be further described with reference to Table 1.

Table 1 is a table showing concrete correspondence between physical quantities of a dynamic system composed of a spring mass point system model and elements of a conceptual space. In Table 1, subscripts i and j represent mass points (concepts) corresponding to the respective elements, and t represents a time series of the simulation.

[0041]

[Table 1]

In Table 1, | P _i -P _j | indicates the corrected similarity between concept elements i and j in the concept structure in the concept space, and corresponds to the spring length in the dynamic system. In the first embodiment, the absolute positional information P _i between conceptual elements in the conceptual space has no objective meaning, and the relative positional relationship | P _i −P _j | Indicates the similarity between the two.

L _ij is two conceptual elements i in the conceptual space,
Indicates the degree of similarity between j, and corresponds to the natural length in a dynamical system. Specifically, a spring is defined in which the reciprocal of the degree of similarity defined only from the relationship between target concept elements is a natural length L _ij .
If there is no disturbance, the relationship between concept elements is kept at the natural length L _ij . In the concept space, between the concept elements extending or contracting beyond the natural length L _ij , the similarity is distorted in the overall structure, and the haptic devices 2 and 4 shown in FIG.
Presents this distortion to the user as an operation reaction force.

K _ij indicates the importance of the similarity between the concept elements i and j in the concept space (the degree of maintaining the similarity).
In a dynamical system, it is made to correspond to a spring constant.

D _ij represents the degree of change in the degree of similarity between concept elements i and j in the concept space (the degree of resistance to a change in the distance between concept elements), and is made to correspond to the spring friction coefficient in a dynamic system. In the initial stage of thinking, the concept is set so that the conceptual element can move freely by suppressing the friction coefficient D _ij, and if the user thinks that the thinking has progressed to a certain extent and the user does not want to break the relationship between certain conceptual elements, By increasing the friction coefficient D _ij , the relative relationship between these conceptual elements can be maintained even with respect to disturbance.

[0046] m _i indicates the superiority-hierarchical order of concepts containing i in concept space to correspond to the mass in the dynamical system. In this way, the hierarchical relation of conceptual elements is represented by the mass of the mass point. By increasing the mass of conceptual elements that are determined to be important, it is possible to suppress a change in position due to a small disturbance.

P _{i, t} indicates the position of the concept element i at the time t in the concept space, and corresponds to the current position of the mass point in the dynamical system. P _{i, 0} indicates the initial concept structure of the user in the concept space, and corresponds to the initial position of the mass point in the dynamical system.

V _{i, t} and a _{i, t} indicate the strength of the distortion of the concept element i and the superiority of the concept element i at the time t in the concept space, and correspond to the velocity and acceleration of the mass point in the dynamical system. Let it.

D indicates the degree of change tolerance of the entire structure in the conceptual space, and corresponds to the friction coefficient of the space in the dynamical system.

F _i-ext represents a distortion applied to the concept element i from outside in the concept space, and is made to correspond to an external force in a dynamic system.

It is also possible to further enhance the hierarchical structure and introduce a function of fixing a certain concept element in the concept space as a reference concept element in forming the concept structure into the operation process. In addition, the initial arrangement of concept elements is determined by the user subjectively arranging concept elements at the start of thinking.
The relative positional relationship between conceptual elements in the initial arrangement greatly affects the result of subsequent convergence.

When a simulation is executed based on these, the conceptual element moves by an elastic force of a spring or an external force input from the outside by an operation, and eventually converges to a minimum value of energy and stops moving. At this time, the motion speed and the acceleration state of the concept element are intuitively presented to the user as the degree of distortion of the similarity of the concept defined in Table 1 and the expression of superiority by the visual information by the operation input.

Further, by directly operating the conceptual structure in the virtual space, the user can recognize the difference in operation feeling. For example, when the operational feeling of a certain mass point (concept element) is very light, the concept element may be light, the spring constant of the connected spring may be small, and the viscosity may be low. Based on the correspondence, this is a conceptual element with low dominance or a similarity with low importance.

By repeating such highly intuitive operations and manipulating the mass and the spring constant as the concept is put together, finally, the “hard” which suppresses the distortion of the similarity relation of the entire concept model It is possible to form a conceptual structure.

Here, the operation reaction force given to the user by the direct operation of the conceptual structure will be described.

The operation reaction force presented when the user is operating a certain concept element is calculated based on the following parameters. As described above, the conceptual structure basically corresponds to FIG.
As shown in (1), it is composed of a plurality of mass points corresponding to each conceptual element and a plurality of springs connecting them.

Here, the concept element j performing free motion
The elastic force F _ij that acts on the concept element i via a spring from
It is given by the following equation.

[0058]

(Equation 1)

The external forces acting on the conceptual element i are an elastic force, a viscous force from space, an inertia and an external force represented by the equation (2), and a motion equation is given by the equation (3) using these.

[0060]

(Equation 2)

By using the acceleration a _{i, t + 1} of the concept element i, the speed represented by the equation (4) and the position represented by the equation (5) can be approximately obtained by the difference calculation.

[0062]

(Equation 3)

The above processing is performed for all concept elements not operated by the user. For the concept element that is directly operated by the user, the information of the position, speed, and acceleration in the next step of the simulation is determined by the motion information of the user's hand.

[0064]

(Equation 4)

Here, P _hand represents the position of the user's hand, and v _hand represents the speed of the user's hand.

From the equation of motion of the operated concept element,
An operation reaction force to be presented to the operating hand is expressed by the following equation (8).

[0067]

(Equation 5)

Here, a _hand represents the acceleration of the user's hand. FIG. 3 is a conceptual diagram showing a flow of thinking support in the thinking support system 1000 according to Embodiment 1 of the present invention.

As shown in FIG. 3, after extracting the concept elements (step S10), the relevance between any two concept elements is calculated (step S11). Further, based on this result, the relevance of the entire concept space is calculated (step S12).

Then, these are embodied in accordance with these (step S13). Specifically, the concept space is displayed on the projector screen 6 shown in FIG. 1 as a viewpoint position tracking type time-division stereoscopic image. Further, the operation reaction force represented by Expression (8) is presented to the user via the six-axis force sense presentation devices 2 and 4.

As described above, the conceptual structure and the hand icon that moves in accordance with the movement of the hand of the user are displayed on the projector screen 6. The hand icon is superimposed on the concept element, and the force sense presentation devices 2 and 4 are displayed. By operating the switch at the tip of the grip (step S14), the concept element is grasped and released.

By performing processing for changing the concept elements or the weights between the concept elements (step S15), the relevance of the entire concept space is newly corrected (steps S11 and S12). Then, further concreteization based on the new concept space is performed (step S13).

Using such a thinking support system 1000, it is also possible to support group communication. In communication in a group, by combining a plurality of the thinking support systems 1000, it is possible to mutually understand the structure of the thinking space of the other party and to fuse them.

FIGS. 4 and 5 show Embodiment 1 of the present invention.
FIG. 4 is a conceptual diagram for explaining operations related to a plurality of conceptual structures in FIG. 4. FIG. 4 shows an operation for combining two groups of conceptual structures, and FIG. 5 shows an operation for separating into two groups of conceptual structures. .

In FIG. 4, A and B show conceptual structures, respectively. In FIG. 5, A1 and A2
Indicates sub-conceptual structures formed separately from one conceptual structure. 4 and 5, C
1, C2,... Represent concept elements, H1 and H2, respectively.
Indicates a hand icon.

Referring to FIG. 4, each participant first establishes its own conceptual structure to some extent prior to the discussion. Thereby, the conceptual structures A and B are embodied.
Then, common concept elements of the respective conceptual structures A and B are joined by a spring mass point model. By fusing the two concept structures to minimize distortion, it is possible to bridge the difference between the concepts of both.

Alternatively, referring to FIG. 5, a certain conceptual structure is divided (separated) at a weakly connected portion to obtain sub-conceptual structures A1 and A2. This makes it possible to appropriately classify common concept elements that are confused with respect to a certain concept structure, and to reestablish communication.

[0078]

As described above, in the thinking support system according to the first embodiment of the present invention, an intangible concept structure can be displayed three-dimensionally, so that the degree of recognition of its own concept is improved. be able to.

Further, by simulating the conceptual structure in the virtual space with the spring-mass system model, it is possible to observe a state in which the structure transitions to a local convergence state, and to recognize partial features of the structure. Information that is important in doing so.

Further, since it is possible to directly operate individual concept elements constituting the concept structure in three dimensions, operability can be improved.

Further, by presenting the concept element to the user with a feeling of operation, it is possible to guide a new idea from the operation feeling.

Touching is not only a mechanical change in the touched object, but also a highly interactive action of locally presenting multidimensional information to the user. If the concept elements are moved directly and the parameters are intuitively grasped as an operation reaction force while grasping the entire structure as visual information, the distortion of the conceptual structure is performed by trial and error by operation, like unraveling the entangled string. Intuitively and consciously perform adjustment operations, and the user directly manipulates the embodied conceptual structure, so that the operational sensation of the structure can be fed back to thinking, and new ideas can be obtained from the process. Brought.

As described above, in the present invention, the sense of operation is regarded as a part of the thinking activity as it is, and the construction of a complex conceptual element, which has been difficult in the past, is intuitively performed with the aid of the sense of operation. Make it possible. Therefore, in the individual thinking activity, the user can intuitively recognize the conceptual structure of his / her own thinking space from the feel of the structure itself, and this is a clue for obtaining a new idea. On the other hand, in group communication, by combining a plurality of the thinking support systems 1000, it is possible to mutually understand the structure of the other person's thinking space and form a new complex conceptual structure by fusing them together. It becomes possible.

[Brief description of the drawings]

FIG. 1 is an overall configuration diagram showing a specific example of a thinking support system 1000 according to Embodiment 1 of the present invention.

FIG. 2 is a conceptual diagram for explaining a correspondence between a conceptual structure and a spring mass point model according to Embodiment 1 of the present invention.

FIG. 3 is a thinking support system 10 according to the first embodiment of the present invention.
It is a conceptual diagram which shows the flow of the thinking support in 00.

FIG. 4 is a conceptual diagram for describing operations related to a plurality of conceptual structures according to the first embodiment of the present invention.

FIG. 5 is a conceptual diagram for explaining operations related to a plurality of conceptual structures according to the first embodiment of the present invention.

FIG. 6 is a conceptual diagram showing a flow of thinking support in a conventional thinking support system using a computer.

[Explanation of symbols]

 2,4 Force sense presentation device 6 Projector screen 8 Head position tracking device 10 Parallel processing computer 14 I / O controller 20 Local simulation unit 22 Simulator 24 Main process unit 26 Drawing process unit 28 Multithread process unit 1000 Thinking support system

 ────────────────────────────────────────────────── ─── Continued on the front page (72) Inventor Yasuyuki Kado 5 Sanhiradani, Inaya, Seika-cho, Soraku-cho, Kyoto Prefecture AT / IR Intelligent Motion Imaging Communications Laboratory (72) Inventor Tsutomu Miyazato Kyoto Soraku 5 Shiraya, Inaya, Subsection, Seika-cho, Gunma ATI Intelligent Motion Imaging Communications Laboratory (72) Inventor Kenji Mase, 5 Shiraya, Inaya, Seika-cho, Kyoto, Soraku-gun, Kyoto 5 Aida, Inc. Intelligent Imaging Communications Laboratory

Claims (2)

[Claims] 1. A system comprising: a concrete means for embodying a conceptual structure in a thinking space; and an operating means by which a user manually operates the conceptualized conceptual structure in a virtual space. A visual presentation means for visually presenting the embodied concept structure in the virtual space in accordance with a viewpoint position of a user; and A thinking support system, comprising: a force sense presentation unit that presents an operation reaction force to the user based on the strength of connection between conceptual elements to be operated in response to the operation. 2. The thinking support according to claim 1, wherein the plurality of concept elements are expressed in the virtual space by using the concept elements as mass points and simulating a connection between the concept elements with a spring. system.