JP2001100879A - Information reading system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information reading system capable of transmitting three kinds of sense information of the sense of sight, the sense of hearing and the sense of touch while including the information transmission of 'the sense of touch', which was not possible conventionally, while utilizing a computer and a computer network. SOLUTION: Concerning the system capable of displaying a virtual space on a network terminal while using data on the network, the data contain data of sense of touch and sense of power and the network terminal is provided with a device, capable of presenting the data of the sense of touch and the sense of power to an operator.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to an information browsing system, and more particularly to an information browsing technology using a terminal such as a computer connected to a network, and an information browsing system applicable to a sense presentation system using virtual reality. About.

[0002]

2. Description of the Related Art In recent years, with the spread of Internet technology and the spread of low-cost, high-performance personal computers, various types of information have been provided via networks.

[0003] For example, in addition to the conventional method of providing information which has been unilaterally distributed from an information provider, a new information provision from the viewer to the provider and a request from the viewer are reflected. In addition, a bidirectional information providing method capable of transmitting information with higher density and higher utility value by selecting information to be provided is performed in real time.

According to such a two-way information providing method, it is possible to provide information more efficiently and accurately than the existing information transmitting means (for example, by writing).

A computer and an operator (here,
The communication means of the information viewer is usually performed by operating a keyboard and a mouse connected to the computer, the information is viewed by a monitor output connected to the computer, and the image information is viewed by the viewer. Information is transmitted only by the thing.

[0006] That is, the input of information from the viewer side,
This is performed using a keyboard or a mouse, and the information provider only outputs image information (and audio) in response to the result.

[0007]

The conventional method as described above is very effective in obtaining information from visual and auditory sensory organs in terms of human senses such as characters, photographs and voices called multimedia. It is.

[0008] However, the conventional method as described above cannot transmit any information regarding "tactile sense" which plays an important role in human information recognition and judgment ability.

Here, the tactile sensation is an important sensation that enables not only grasping / manipulation of an object but also recognition of detailed physical properties.

[0010] That is, the tactile sensation (tactile sensation) when grasping or pushing an object is a very effective sensation that can judge even the surface and internal structure of the object.

In particular, in the medical field, it is needless to say that identification of the affected part and determination of the degree of progress of the symptoms by palpation are very effective means in the practice of medical treatment.

However, since tactile sensation cannot preserve its senses, the only way to learn is to remember its "senses" through experience.

For this purpose, it is necessary for the operator to repeatedly touch the object, and to remember the tactile sensation (tactile sensation), which requires much time and experience.

Furthermore, in order to transmit "sense" to other people as information, it is necessary to transmit the character information as an abstract expression.
There was only a means, and the transmission of the sensation itself was impossible.

Further, the tactile sense is transmitted in relation to the visual sense (image information) or the auditory sense (speech) rather than the independent information transfer force, so that the information transfer force to the receiver (viewer) is dramatically improved. I do.

However, while it is clear that realizing multi-modal communication of visual and tactile sensations on a computer network is effective, it has not been studied so far. Met.

Accordingly, the present invention has been made in view of the above-mentioned circumstances, and includes a computer, a computer network, and a method of visual, auditory, and tactile sensation including information transmission of “tactile sensation” that was impossible in the past. It is an object of the present invention to provide an information browsing system capable of transmitting the three kinds of sensory information.

[0018]

According to the present invention, in order to solve the above-mentioned problems, (1) a system capable of displaying a virtual space on a network terminal using data on a network, Is provided, wherein the network terminal includes a device capable of presenting the tactile / force sense data to an operator.

According to the present invention, in order to solve the above problems, (2) a system capable of displaying image information on a network terminal using data on the network, wherein the network terminal Photographing means capable of photographing the image, a feature extracting means for extracting features of the facial expression of the operator photographed by the photographing means, and a psychological state of the operator is estimated based on the features extracted by the feature extracting means. And a psychological state estimating means.

According to the present invention, in order to solve the above-mentioned problems, there is provided (3) a system capable of displaying a virtual space by communication between a plurality of network terminals using data on a network. At least one of the network terminals includes a device capable of operating a virtual space, and the operation of the virtual space performed by the one network terminal is observable from at least one other network terminal of the plurality of network terminals. An information browsing system is provided.

[0021]

DESCRIPTION OF THE PREFERRED EMBODIMENTS First, an outline of the present invention will be described.

Recently, applying simulation technology by applying robotics and virtual reality technology,
Efforts have been made to improve operability by transmitting tactile sensations.

Here, it is possible to obtain a sense of operation at the time of operation by drawing a virtual object on the monitor of the computer and operating the object with a manipulator connected to the computer.

In addition, adaptation has been made such that a plurality of persons simultaneously enter the virtual space to perform operations or to perform a certain work jointly.

The advantage of these adaptations is that in order to work in cooperation, it is necessary to recognize and grasp both pieces of information, and the information providing side and the obtaining side are not uniquely determined.

According to the present invention, by using the tactile sense presentation device used in the virtual reality area and the cooperative virtual space, information obtained by adding visual, auditory, and tactile data to operation of browsing information data in the information browsing system. It is intended to build a browsing system.

The cooperative virtual space used here is a virtual space constructed based on data held at a plurality of places (for example, a plurality of computers connected to a network).

Data for this purpose usually includes position coordinates, size, shape, orientation, etc. of the object existing in the virtual space.

Also, data that is frequently used includes data representing an object by a wire frame model in a three-dimensional virtual space. However, the present invention is not limited to this, as long as data at a plurality of locations can be used. In each case, only data at a single location (eg, in a server computer) may be used.

The browsing information data used here is data referred to by a client computer in order to create a cooperative virtual space as described above. For example, the above-described wire frame model data and the like correspond to the browsing information data. I do.

The tactile sensation presentation device used here is, specifically, an operator moving a virtual device in the form of a hand or an appliance in the cooperative virtual space and touching a virtual object in the virtual space. In such a case, the tactile and force sense data including repulsive force and resistance are fed back to the operation unit to give the operator a feeling as if he / she touched an actual object.

Here, the tactile and force sensation data are, specifically, data reflecting the hardness and softness of the virtual object, and in the later-described embodiment of the present invention, the elasticity value of each object is calculated. When the pointer touches an object, the tactile sensation providing device gives feedback of repulsion or resistance.

Further, in an embodiment described later of the present invention,
The operation of the tactile sense presentation device is configured to be reflected in the virtual space.

For example, when a virtual object is pushed in the virtual space by operating the tactile sense presentation device, a depression corresponding to the elasticity is formed on the surface of the object in the virtual space.

In a later-described embodiment of the present invention, the virtual device displayed in the virtual space is movable with a tactile sense presentation device, and is displayed in the form of a hand or an appliance.

As described above, the information browsing system of the present invention realizes a function that a plurality of viewers can browse and cooperate simultaneously, which is not available in the conventional information browsing method.

Further, the information browsing system of the present invention simply presents bidirectional information such as presenting browsing information using the emotion or psychological state of the viewer as a variable, which was not possible in the past. , Rather than presenting one-sided information such as disclosure of information, rather than interactive and appropriate information transmission /
It can also be used as a means of obtaining.

FIG. 6 shows a specific example of a tactile presentation device (force display) used in the present invention.

As shown in FIG. 6, the tactile presentation device (force display) includes a manipulator 1 including three link mechanisms 2 having the same configuration.

The tip of each link mechanism 2 is a joint 4
Are connected to one moving plate 6, and the base end of each link mechanism 2 is supported by a base 10 via a support portion 8.

The support portions 8 are arranged at the respective vertices of an equilateral triangle virtually formed on the base 10.

As a result, the three link mechanisms 2 are connected to the base 1
From the support portion 8 disposed at each vertex position of a virtual equilateral triangle on the moving object 0 to the moving plate 6, it is constructed in a substantially triangular pyramid shape as a whole.

The tactile sensation providing device (force display) further includes, in addition to the three link mechanisms 2 described above, a sensor for detecting the position or displacement of a predetermined movable part of each link mechanism 2, and a link mechanism. Actuators 42 and 46 that can drive the movable plate 2, and a control unit that detects the position or posture of the moving plate 6 by the sensors 40 and 44 and controls the actuator to control the position or posture of the moving plate 6. , The sensors 40 and 44 and the actuators 42 and 46 are both disposed on the support portion 8 of the link mechanism 2.

Each of the support portions 8 constituting the base ends of the three link mechanisms 2 has a fixed block 12 fixed to the base 10 and one base part rotating shaft parallel to the base 10. A base 16 rotatably attached to the fixed block 12 through the base 16 is provided.

The base 40 is provided with the above-described sensor 40,
44 and actuators 42 and 46 are arranged in a concentrated manner.

Further, each of the three link mechanisms 2 has a first link 2 rotatably attached to the base 16 via a link rotation axis perpendicular to the base rotation axis.
0, a second link 24 having one end rotatably attached to the base 16 via a link rotation shaft coaxial with the link rotation shaft 18 of the first link 20, and a first link. 20
One end is rotatably attached to the other end of the first link 20 via a link rotation shaft parallel to the link rotation shaft 26 of
And a third link 28 parallel to the second link 24;
The other end of the second link 24 is rotatably attached to the intermediate portion via a link rotation axis 30 parallel to the link rotation axis of the first link 20, and the link rotation of the first link 20 is performed. One end is rotatably attached to the other end of the third link 28 via a link rotation shaft 32 parallel to the moving shaft, and includes a fourth link 34 parallel to the first link 20. I have.

The other end of the fourth link 34 is connected to the moving plate 6 via the joint 4.

Each of the three link mechanisms 2 has a parallel link composed of the first link 20 and the fourth link 34 and a parallel link composed of the second link 24 and the third link 28, respectively. A pair of parallel link structures, two degrees of freedom in the translation direction realized by the degrees of freedom of rotation of each link rotation axis including the link rotation axes 26, 30, 32, and one base part rotation axis. And one degree of freedom in the rotation direction.

The joint 4 for connecting the tip of each of the three link mechanisms 2 (specifically, the other end of each of the fourth links 34) to the moving plate 6 has three degrees of freedom around the rotation axis. have.

Accordingly, by connecting the tips of the three link mechanisms 2 (the other ends of the respective fourth links 34) to the moving plate 6 via the joint 4, the moving plate 6 has six degrees of freedom. Can be moved in space.

A gripping member for operating the manipulator 1 is fixed to the moving plate 6 by the operator actually holding it with his / her hand.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(First Embodiment) FIG. 1 is a diagram for explaining an outline of a device configuration of an information browsing system according to a first embodiment of the present invention.

FIG. 1 is a diagram showing a connection state of a computer used as a terminal connected to a network.

In FIG. 1, 1a is a server computer, 1b
1e is a computer of each facility, 1f-1i is a client computer, 1j-1n is a network line,
o to 1r indicate tactile sense presentation devices, and 1s to 1v indicate cameras.

Here, the server computer 1a, the computers 1b to 1e of each facility and the client computers 1f to 1i are connected to the network lines 1j to 1 respectively.
n, data communication between each computer
Exchange is possible.

Each client computer 1f-1i
Have a cooperative virtual space construction (display) function unit, respectively, and can start the information browsing system according to the present invention.

The server computer 1a has a system management function unit.

The computers 1b to 1e of each facility
Has a browsing data section for storing browsing data for creating a collaborative virtual space, and the client computer 1
f-1i create a virtual space using the browsing data stored in the browsing data section.

If the designated virtual space creation data does not exist in the server computer 1a, the information browsing according to the present invention is performed on the computers 1b to 1e of the other facilities connected to the network. The system management function unit realizes a function of automatically searching the system and automatically acquiring data from another server computer holding necessary data.

Then, the client computers 1f-
Tactile sense presentation devices 1o to 1r can be connected to 1i.

The tactile sense presentation devices 1o to 1r can be used when the browsing information including the virtual space includes tactile or force sense data.

Here, the tactile sense presentation devices 1o to 1r are:
For example, a structure as shown in FIG. 6 is used, and when the object presented as the browsing information includes information on hardness, softness, or tactile sensation, a resistance when contacting the object is detected. Works to represent.

That is, when the viewer touches the object using the tactile sense presentation devices 1o to 1r, the sense can be recognized.

For contact with the object or operation of the object,
This is performed by relying on a virtual device which is displayed together with the operation target in the virtual space and moves in response to the tactile sense presentation devices 1o to 1r.

This tactile sensation presentation device has the shape of a hand or an appliance in accordance with the content of the browsing information. As described above, the virtual device displayed in the virtual space is:
It is movable with a tactile presentation device, and is displayed in the form of a hand or an appliance.

The viewer can operate a virtual object to be viewed using these virtual devices that move in the same manner as the movements of the tactile sense presentation devices 1o to 1r.

This operation can be confirmed on separate client computers 1f to 1i.

Further, it is possible to perform not only a simple browsing but also a work of cooperatively operating one virtual object in cooperation with each other.

In the case of browsing using the tactile sense presentation devices 1o to 1r, if the browsing information, that is, the mechanical attribute of the virtual object does not exist in the server computer 1a, the information is transmitted from another server computer having necessary data. Obtains automatically.

By distributing the information in a distributed manner, the information that must be stored by the main server computer can be reduced.

The cameras 1s-1v are connected to the client computers 1f-1i.

Each of the cameras 1s to 1v captures an image of a face at the time of operation of an operator who performs browsing.

If the browsing information that the operator is browsing is read from the expression of the operator and the psychological state is read, it is more useful to change a more detailed part of the content. In the case of such browsing information, based on this photographed image, facial expression features that can identify the operator's psychological state are automatically extracted, and this data is browsed via a network. The information is transmitted as detailed search information to the server computer 1a that has obtained the information or the computers 1b to 1e of each facility.

Based on the transmitted information, the optimum information to be presented to the operator as the browse information is searched, and the browse information presented in the cooperative virtual space is changed.

(Second Embodiment) FIG. 2 is a diagram for explaining an outline of the relationship between the functional parts in FIG. 1 as a second embodiment of the information browsing system according to the present invention.

That is, FIG. 2 shows a schematic configuration of the relationship between the functional units realized by the server computer 1a, the computers 1b to 1e of each facility, and the client computers 1f to 1i connected to the network shown in FIG. FIG.

Hereinafter, each function will be described in detail.

In FIG. 2, reference numeral 2a denotes a system management function unit.
b is a standard browsing data section, 2c is a standard haptic generation data section,
2d is a virtual space distribution function unit, 2e is an autonomous search function unit, 2
f is a browsing data part, 2g is a unique browsing data part for virtual space generation, 2h is a unique tactile generation data part for virtual space generation, 2
i is a cooperative virtual space construction function unit, 2j is a virtual space generation function unit, 2k is a sense control function unit, 21 is a feature extraction function unit, 2
m is a virtual space data distribution function unit, 2n is a tactile presentation device, 2o is a camera, 2p is a position detection function unit, 2q is a reaction force output function unit, 2r is a mental state estimation function unit, 2s is a sensor, and 2t is an actuator. Is shown.

The processing flow is as follows.

First, the viewer logs in to the client computers 1f to 1i shown in FIG. 1 and activates the application.
The virtual space data distribution function unit 2m in the cooperative virtual space construction function unit 2i is connected to the virtual space distribution function unit 2d in the server computer 1a shown in FIG. 1 to enable browsing information to be obtained.

At this time, the client computers 1f to
On the 1i side, all browsing information managed in the server computer 1a can be grasped.

Therefore, the viewer selects the information he / she wants to view, and obtains the browse information from his / her own client computer 1f-1i.

Here, if the obtained browse information is in the standard browse data section 2b of the system management function section 2a in the server computer 1a, the server computer 1a transmits the data to the client computers 1f to 1i.

However, when the browsing information to be obtained is not in the standard browsing data section 2b in the server computer 1a, the server computer 1a sends the information from the computers 1b to 1e of each facility shown in FIG. Is issued to obtain the unique browsing data for virtual space generation stored in the unique browsing data unit for virtual space generation 2g of the browsing data unit 2f corresponding to.

The tactile or haptic information is added to the standard browsing data stored in the standard browsing data unit 2b and the virtual space generating unique browsing data stored in the virtual space generating unique browsing data unit 2g. When there is an addition instruction, the standard haptic generation data stored in the standard haptic generation data unit 2c in the system management function unit 2a is transmitted to the client computers 1f to 1i again.

Similarly, if there is no desired tactile or force information in the standard tactile generation data stored in the standard tactile generation data 2 part c, the computer 1b to 1e of each facility will The unique tactile data for virtual space generation stored in the unique tactile data unit for virtual space generation 2h is obtained and transmitted to the client computers 1f to 1i via the server computer 1a.

A series of these data distribution operations are all performed by the system management function unit 2a in the server computer 1a.

Next, in the client computers 1f to 1i which have obtained the browsing information, the obtained browsing information is drawn in the cooperative virtual space, and the information can be browsed.

The obtained data is passed through the virtual space data distribution function unit 2m to the virtual space generation function unit 2g, where the contents described in the browsing information are displayed in the virtual space.

The display method of this virtual space is also described in the browsing data, and a unique screen is drawn according to the contents of each data.

Further, if tactile or force sensation information is added to the acquired data, the information is passed to the sensation control function unit 2k, and the tactile or force sensation can be presented.

This can be confirmed by displaying a virtual device such as a hand or a treatment tool in the cooperative virtual space.

At this time, the position detection function unit 2p in the sensation control function unit 2k obtains a signal from the sensor 2s of the tactile sense presentation device 2n connected thereto, and a virtual device displayed in the cooperative virtual space. When it is determined that the browsing information has been operated by using, the degree of the operation is transferred from the position detection function unit 2p to the reaction force output function unit 2q, and the standard tactile sense generation data unit 2g obtained as the browsing data is transmitted to The torque calculated based on the stored standard tactile sense data or the unique tactile sense data for virtual space creation stored in the unique tactile sense data unit for virtual space creation 2h is a reaction force output function unit 2q. From the tactile sense presentation device 2n
By being output with respect to t, the viewer can recognize the tactile or force sensation that changes with the operation, and can recognize the browsing information in more detail.

In the present invention, the cooperative virtual space for displaying the browse information described above means a virtual space where a plurality of people can share the same environment at the same time.

If the viewer logs in a new client computer 1f to 1i and another viewer has already viewed the browsing information to be viewed, the consent of the viewer who is browsing in advance is obtained. Thus, it is possible to browse the same screen as the viewer who is currently browsing in advance.

As described above, when other browsing data is selected, a box on which the user can newly select information to be browsed is displayed on the screen.

The processing for obtaining data in such a situation is all performed by the system management function unit 2a of the server computer 1a, and a state without any inconvenience is maintained.

In the case of such a so-called cooperative work,
Virtual devices are simultaneously drawn in the cooperative virtual space of the client computers 1f to 1i, and information can be browsed while observing and recognizing the operation of the partner.

The client computers 1f-1
A camera 2o that captures the viewer's face is connected to i, and this image is input to a feature extraction function unit 21 in the cooperative virtual space construction function unit 2i.

When data contents are selected by using a change in the mental state of the operator as browsing data, the feature of the image is extracted by the feature extraction function unit 21 based on the image data. The result is passed to the mental state estimation function unit 2r, where the mental state of the operator is estimated.

Then, the result is transmitted from the virtual space data distribution function unit 2m to the system management function unit 2a, and is referred to via the standard browse data unit 2b or the autonomous search function unit 2e. Is updated.

At this time, the image from the camera 2o is always
If it is obtained and it is not necessary to perform the above processing, it is passed as it is from the virtual space data distribution function unit 2m to the system management function unit 2a of the server computer 1a. When it is desired to check the viewer's face, it is possible to display the face in the cooperative virtual space.

Next, in the third, fourth, and fifth embodiments, the above functions will be described based on more specific adaptations.

(Third Embodiment) FIG. 3 is a diagram for explaining an outline of a device configuration in a third embodiment in which the information browsing system according to the present invention is applied to palpation diagnosis in the medical field.

In FIG. 3, 3a is a server computer, 3b
To 3e are computers of each facility, 3f to 3i are client computers, 3j to 3n are network lines, 3
o to 3r indicate tactile presentation devices, and 3s to 3v indicate cameras.

FIG. 7 shows a drawing image 3w of a cooperative virtual space drawn on a client computer in the third embodiment in which the information browsing system according to the present invention is applied to palpation diagnosis in the medical field.

The doctor who wants to browse the information displays the cooperative virtual space using the client computer 3f, and obtains the patient's diagnostic data from the server computer 3a.

In this case, the diagnostic data obtained from the server computer 3f is examination data taken by CT or MRI.

The diagnostic data obtained from the server computer 3f also includes three-dimensional model data constructed from a plurality of two-dimensional tomographic images taken by CT or MRI.

The obtained diagnosis data is displayed as a virtual laparotomy model in the cooperative virtual space on the client computer 3f, and the organ arrangement of the patient is three-dimensionally displayed as shown in FIG.

On this screen, observation is performed mainly on the organ where the affected part exists.

From the present diagnostic data, it is possible to visually identify the diseased part. However, if it is desired to more accurately identify the affected part, drive the tactile sensation presentation device and simulate the palpation operation of the affected part. Is possible.

The diagnostic data is accompanied by tactile or force sensation data unique to the data.

In this case, as the tactile / force sense data, anatomically known data may be held in advance, but the ultrasonic wave is applied to the subject and the echo is analyzed to analyze the elasticity / force. Viscosity data may be obtained.

Thus, using the tactile presentation device,
It is possible to operate the browsing information, and a virtual hand or treatment tool that operates in synchronization with the tactile sense presentation device is simultaneously drawn in the cooperative virtual space.

Here, the display image of the cooperative virtual space to be drawn is like 3w shown in FIG.

FIG. 7 is a diagram showing an example in which a cooperative virtual space is displayed on a display of a client computer.

Reference numerals A1 and A2 in FIG. 7 are the organs of the patient created in the cooperative virtual space.

The figure in which the arrow is arranged in parallel with the shape of the hand denoted by reference numeral A3 is a virtual device that can move in response to the operation of the tactile sense presentation device.

Further, reference numeral A4 is fixed to the patient,
This is an area that indicates where the virtual device is located and in which direction in the coordinate system with the center of the affected part as the origin.

A reference mark A5 is a plane mark displayed when the virtual device comes into contact with the organ of the patient, and has a form capable of imaging the tangent plane between the organ and the virtual device.

By displaying this plane mark,
The operator can know that the organ and the virtual device have come into contact with each other, and when the virtual device is moved in the direction in which the organ is pressed, a reaction force is presented by the sensation presentation device, and a realistic operation can be performed.

Reference numeral A6 is an area where a message from the browsing system is displayed, and displays an elasticity value of an organ, a coordinate value of a virtual device, a warning from the system, and the like.

Reference numeral A7 is a reference information area.

The reference symbol A8 indicates that the outer elliptical figure is a cross section of the patient's body, and the inner circle indicates the position of the organ currently displayed.

[0127] Reference numeral A9 is an area where an image or the like of an organ actually taken is displayed.

[0128] Reference numeral A10 is an area in which a depression on the surface of the organ is schematically displayed as a graph when the organ is pressed by the virtual device.

When the organ is not pressed, a horizontal straight line is displayed. However, when the organ is pressed and dented, the straight line changes to the cross-sectional shape of the dent.

Then, it is possible to perform a simulation for checking the affected part using the virtual hand or the treatment tool (reference numeral A3) in the display image 3w of the cooperative virtual space drawn as described above.

In the cooperative virtual space, CT or MRI raw data and
An operation position is displayed so that the tactile sense presentation device can currently determine which position is currently being operated, and the screen is configured to be easy for a doctor to determine.

When the patient is to be verified with reference to the opinion of another doctor, the patient can be jointly verified using another client computer connected via a network. is there.

Here, it becomes possible for a plurality of doctors to log in to the same cooperative virtual space and to have a discussion while being remote from each other.

In this case, it is possible to simultaneously observe each other's hands or treatment tools operating in synchronization with each other's tactile sensation presentation devices in the virtual space, and share the positions and operations indicated by each other as information. It is possible to

Further, the viewers who are cooperating with each other display an image of the face photographed by the camera connected to the client computer in the cooperative virtual space.
Operation while confirming each other is possible.

(Fourth Embodiment) FIG. 4 is a diagram for explaining an outline of a device configuration in a fourth embodiment in which the information browsing system according to the present invention is applied to psychological counseling in the medical field.

In FIG. 4, reference numeral 4a denotes a server computer and 4b
4e to 4e are computers at each facility, 4f to 4i are client computers, 4j to 4n are network lines,
o to 4r indicate tactile presentation devices, and 4s to 4v indicate cameras.

An operator who wants to view information (in this case, a patient) uses the client computer 4f to display a cooperative virtual space.

[0139] Then, the server computer 4a performs a browsing information search capable of performing psychological counseling to obtain data.

Information for counseling is described in this data item, and it is structured to answer questions in order.

[0141] In the question, items whose mental state can be grasped by changing the expression of the patient in response to the question are described.

The information browsing system obtains, from the camera 4s connected to the client computer 4f, the facial expression of the patient at the time of answering the question, passes it to the feature extraction function unit 21 shown in FIG. The function unit 2r grasps the mental state at the time of answer as data, and sequentially transfers the result to the server computer 4a.

When a change in the expression that can identify the mental state is recognized, it is used when creating counseling result data.

Here, the mental state is estimated from the expression,
When changing the acquired browsing data, the following points are featured in the face of the operator who has photographed.

(1) Direction of line of sight: From this, the image part of interest to the operator is inferred.

(2) Fixed degree of sight line: If the sight line does not move for a predetermined time or more, it is determined that the image part where the sight line is placed is an important object of interest.

(3) Shape of Mouth: If it is determined in (1) and (2) that there is significant interest in the image area,
Judge the shape of the mouth.

At this time, when it is determined that the mouth is a strongly connected mouth by comparing it with the reference image model, it is determined that the operator's mental state is in negative emotion.

That is, it is determined that the operator has a negative emotion with respect to the contents of the image area.

(Fifth Embodiment) FIG. 5 is a diagram for explaining an outline of a device configuration according to a fifth embodiment in which the information browsing system according to the present invention is applied to browsing natural materials.

In FIG. 5, 5a is a server computer, 5b
5e to 5e are computers of each facility; 5f to 5i are client computers; 5j to 5n are network lines;
o to 5r indicate tactile sensation presentation devices, and 5s to 5v indicate cameras.

A viewer who wants to browse information displays a cooperative virtual space using the client computer 5f.

A viewer who wishes to browse information obtains desired museum data from the server computer 5a.

In this case, the standard browsing data 2b shown in FIG.
If there is no desired museum data, the computer 5 of each facility (here, a museum that owns the museum data)
Obtain browsing information from b to 5e.

As a result, the client computer 5
f, the browsing information desired by the viewer is displayed, and if tactile or haptic information is added to the browsing data, the natural history material is cooperatively virtualized using the tactile presentation device 5o. It is possible to recognize the texture using virtual cues drawn in the space.

For example, in the case of a natural history material in which the manufacturing method changes according to the age of the ancient pottery and the like, and the texture of the surface is greatly different, and the age is determined, the information browsing system of the present invention is used. Is effective for acquiring their knowledge.

In the case of exhibits such as museums, it is usually strictly forbidden to touch the exhibits, but in the case of touching, the knowledge acquisition or learning effect is enhanced by touching. The information browsing system of the present invention is very effective.

The configuration of the embodiment of the present invention is not limited to the configurations of the first to fifth embodiments described so far.

In the present invention, the device configuration is based on a network.

[0160] The network referred to here is intended to connect computers scattered in distant places. However, if a network configuration is simply established between computers in adjacent local sites, The application of the present invention can be operated and used between the computers.

If desired data is stored in the client computer, it may be used as an information browsing system solely by the client computer.

Furthermore, in the information browsing system of the present invention, it is impossible to present tactile or force information, but a tactile presentation device is a keyboard or mouse which is an input operation device of a conventional standard computer. It is also possible in the configuration of the present invention to realize a configuration in which the information to be transmitted is changed to character or position information instead of the signal input from the device.

In the present specification described in the above-described embodiment, in addition to claims 1 to 3 described in the claims, the inventions shown below as supplementary notes 1 to 6 are described. include.

(Supplementary Note 1) A system management function unit provided in any one of the computers connected to the network, a browsing data unit provided in any one of the computers and storing browsing data, A plurality of computers, a cooperative virtual space construction function unit configured to construct a cooperative virtual space, and a tactile presentation device capable of providing a tactile or force sensation that is connected to each of the plurality of computers in the computer group, A camera that is connected to each of the plurality of computers in the group of computers and captures the face of the system operator, and is assigned to a virtual object existing in the cooperative virtual space, and has at least visual and tactile sensation capable of apparently simultaneous communication or An information browsing system comprising a haptic data unit.

(Operation) A plurality of computers in a group of computers connected to the network, that is, computers actually operated by a viewer (hereinafter, client computers) are connected to a computer having a system management function unit (hereinafter, server computer). And start browsing.

Using the system management function unit, the server computer prompts the provision of information from itself or a computer of each facility having browsing information connected to the network according to the examination of the browsing items or the intention of the browsing user. .

The browsing data stored in the browsing data section is visually presented to the viewer using the cooperative virtual space.

Further, when the browsing data includes tactile presentation information, that is, data for generating tactile or force sensation, the tactile presentation device connected to the client computer transmits a signal from the drive mechanism of the tactile presentation device. Then, the tactile or haptic information arranged in the browse information is presented to the viewer.

At this time, the actual operation of the tactile sensation providing device, the state of change of the browsing information in the cooperative virtual space, and the new information obtained by the operation correspond.

The camera photographs the face of the system operator.

(Effect) According to the present invention, not only visual information, which is a general form of conventional browsing information, but also tactile or force sensation information is presented to the viewer, so that more dense information can be presented. Will be possible.

Further, according to the present invention, by using a virtual space, it is possible to present browsing data in a spatially expanded world, and to obtain information from a more realistic or immersive world. Become.

In the virtual space, a common virtual world can be shared among a plurality of client computers in a coordinated manner.

That is, according to the present invention, it is possible to browse the common browsing data at the same time, and it becomes possible to share information.

In the present invention, by coordinating the third perception of touch in this space, the information density is further increased in the present invention.

According to the present invention, a huge amount of browsing information
Each of them is stored in a computer of each facility connected to the network, and the server computer only serves as a contact point with the client computer.

Therefore, there is an advantage that the information stored in the server computer is very small.

Further, according to the present invention, for the viewer,
Because it is possible to communicate directly with the optimal facility that has the information you want to view, efficient information can be obtained.

Further, the present invention has a camera capable of photographing the face of the operator, and recognizes the expression of the operator.
Judgment can be made as data for browsing information that changes browsing information according to the psychological state of the operator.

(Supplementary Note 2) A system management function unit provided in any of the computers connected to the network includes a standard browsing data unit for storing standard browsing data, and a standard haptic generating data for storing standard haptic generating data. Unit, a virtual space distribution function unit for distributing virtual space data, and autonomy for retrieving necessary data from another computer including the browsing data unit and transferring the data to a virtual space generation function unit of the cooperative virtual space construction function unit. 2. The information browsing system according to claim 1, comprising a search function unit.

(Operation) The standard browsing data stored in the standard browsing data section is data for generating a standard virtual space in the information browsing system of the present invention.

This is a menu screen which is first seen when a viewer logs in to the information browsing system of the present invention. The network screen is searched for more information to be obtained by relying on the items shown on this screen. You can also look over.

When all the browsing information is available in the server computer, browsing is performed using a virtual space generated based on the standard browsing data.

When it is desired to provide information including presentation of a sense in the browse information, the sense is presented using the standard tactile generation data stored in the standard tactile generation data section.

The standard tactile sense data section stores data for driving the tactile sense presenting device.

The virtual space distribution function unit updates the browsing data and transmits the position coordinate information of the tactile sensation providing device in the virtual space, that is, the motion trajectory data to a plurality of client computers browsing in the cooperative virtual space. for,
Transfer over the network.

From within the standard virtual space, the viewer can move to the computer of each facility to obtain local browsing information of each facility in order to obtain more detailed information.

At this time, the display is changed to the virtual space menu prepared separately by each facility, and the information from the server computer of each facility is used as standard virtual space auxiliary data prepared by the server computer. 2
There is a street.

The acquisition of the latter auxiliary data is automatically performed by the autonomous search function unit of the system management function unit.

The selection judgment is also made by the system management function unit.

Here, it is determined how to complete the virtual space.

Even in a situation where a plurality of pieces of data are required to create a collaborative virtual space and each of them is scattered at completely different facilities, the autonomous search function unit determines the data contents and determines the most suitable facility that holds the data. Is selected, and cooperation for obtaining desired data is urged, and the obtaining and creating are completed.

(Effects) According to the present invention, the information stored in the server computer or database of each facility is only data for generating a virtual space, and this function does not have a function of generating a virtual space. Since all the data is stored in the server computer, if the data stored in the server computer alone is not completed as the final browsing information, the browsing information is completed using the additional data of each facility as described above. .

As a result, according to the present invention, the information can be distributed and the server computer or the server computer or database of each facility can effectively use the resources of the data holding mechanism.

(Supplementary Note 3) The browsing data section provided in any of the computers connected to the network includes a virtual space generating unique browsing data section for storing virtual space generating unique browsing data, and a virtual space generating unique browsing data section. The information browsing system according to claim 1, comprising a unique tactile sensation generation data section for generating a virtual space, which stores the unique tactile sensation generation data.

(Operation) The unique viewing data for creating a virtual space stored in the unique viewing data section for creating a virtual space is data for creating a virtual space prepared by each facility independently.

Based on this, the information browsing system changes the display of the virtual space.

When the browsing information includes an item for adding information for generating a sense, and when the standard tactile generation data of the server computer is insufficient or more special, the virtual space is used. The unique tactile sensation generation data for virtual space generation stored in the unique tactile sensation generation data unit for generation is used to present a sense to a viewer.

Then, these data are distributed to the server computer via the virtual space data distribution function unit, constructed as individual browsing data of each facility, and distributed to the client computer, that is, the viewer.

(Effect) In the present invention, the information held by the computer or database of each facility is only data for generating a virtual space, and has no function of generating a virtual space.

Therefore, in the present invention, if the data stored in the server computer alone is not completed as final browsing information, the browsing information is completed using the additional data of each facility as described above.

As a result, according to the present invention, the information can be distributed and the server computer or the computer or database of each facility can effectively utilize the resources. It is not necessary to pay attention to changes in the browsing format itself, and data can be promptly distributed.

(Supplementary Note 4) A cooperative virtual space construction function unit provided in a plurality of computers in the group of computers connected to the network includes a virtual space generation function unit,
The information browsing system according to claim 1, further comprising a sense control function unit, a feature extraction function unit that extracts a feature of an image from the camera, and a virtual space data distribution function unit.

(Operation) The cooperative virtual space construction function unit of the client computer draws a virtual space in front of the viewer by the virtual space generation function unit based on information from the server computer.

Here, it is assumed that a virtual space that is standard in the information browsing system of the present invention is drawn.

In this virtual space, a model to be browsed three-dimensionally is drawn.

The browsing data has been distributed to the client computer through exchange between the system management function unit of the server computer and the virtual space data distribution function unit of both the client computer.

If the browsing data includes tactile information, that is, tactile or force sensation generating data, the tactile sensation generating unit sends a signal to the drive mechanism (actuator) of the tactile sensation providing device connected to the client computer. Is transmitted, the tactile sense presentation device is driven to present the tactile or force sense information arranged in the browse information to the viewer.

Further, the information browsing system of the present invention has a camera for photographing the operation status of the viewer.

This camera is connected to a client computer used by a viewer, and inputs an image signal from the camera to the client computer.

On the client computer side, the feature amount of the image is calculated by the feature extracting function unit,
Save as data.

[0212] The sense control function unit performs processing for presenting a tactile sensation to the viewer when the item for providing the sense information is included in the browse information.

[0213] It controls the presentation of the tactile sensation and generates the drive signal of the tactile sensation presentation device.

Further, the emotion / psychological state of the operator is estimated from the expression, base data for browsing information using the data is created, and the base data is distributed to the server computer by the virtual space data distribution function unit. The browsing data so that the browsing information can be changed using the emotional / psychological state of the person.

(Effect) According to the present invention, spatial expanse can be recognized by using a virtual space as a method of browsing information, so that a three-dimensional grasp becomes possible.

In the present invention, even when information held by a plurality of facilities is integrated and completed as one piece of information, the final form of the information is formed in the virtual space, and Can be delivered to

An advantage of the present invention is that the method of publishing information up to now has been limited to text information or text information as an image instead of a document, but an information browsing system using a virtual space as described above. This means that not only the above information but also tactile information can be disclosed.

At the same time, it is important that the present invention can realize high-dimensional information disclosure in response to temporal changes.

The change in tactile sensation corresponding to this time change can also be realized by the information browsing system using the virtual space as described above.

Further, according to the present invention, it becomes possible to share operation information of the same tactile sensation providing device between client computers connected to a network.

Thus, when working in cooperation,
Both or a plurality of clients can make a common operation status determination.

Further, according to the present invention, information can be presented in both directions such as presentation of browsing information using the emotion or psychological state of the viewer as a variable, which was not possible in the past. Instead of such a unilateral dimension, it can also be used as an interactive and appropriate means for transmitting and obtaining information such as an appropriate information search that grasps the state of the viewer.

(Supplementary Note 5) The sense control function unit connected to each of the plurality of computers connected to the network includes a position detection function unit that detects an operation state of the tactile sense presentation device, and an operation in the virtual space. Depending on the situation, the emotional / psychological state of the operator based on data from the reaction force output function unit that performs reaction force feedback to the tactile sense presentation device and the feature extraction function unit of the cooperative virtual space construction function unit 3. The information browsing system according to claim 1, comprising a psychological state estimating function unit for estimating the information.

(Function) The reaction force output function unit generates a signal for driving an actuator installed in the tactile sense presentation device.

Further, the reaction force output function unit, when operating the browsing information using the tactile presentation device, determines the actual operation of the tactile presentation device, the state of change of the browsing information in the cooperative virtual space, and Resonant with new information obtained by operation.

[0226] The psychological state estimating function unit specifies the emotion and psychological state of the current operator, that is, the viewer, using the data from the feature extracting function unit of the cooperative virtual space construction function unit.

(Effects) According to the present invention, the contents of the browsing information can be more accurately and realistically recognized by transmitting a feeling accompanying the operation.

Further, in the present invention, the visual presentation state of the tactile sensation or the force sensation is resonated to create a situation in which there is no contradiction in the recognition of the operation, and the immersion feeling is deepened.

Further, according to the present invention, it is possible to present information in both directions, such as presenting browsing information using the emotion and psychological state of the viewer as variables. Instead, it can be used as an interactive and appropriate means of transmitting and obtaining information, such as searching for appropriate information by grasping the state of the viewer.

(Supplementary Note 6) The tactile sensation providing devices connected to the plurality of computers connected to the network each include a sensor for detecting the position of a hand or finger that moves in response to an operation of an operator, and a sensor for detecting the position of the hand or finger. The information browsing system according to claim 1, further comprising an actuator that performs a reaction force feedback with respect to the position of the hand / finger to be performed.

(Function) The sensor detects spatial position information of a part that freely moves with the movement of the operator's finger on the tactile sense presentation device.

The actuator restricts its free movement to generate operational resistance.

Further, the actuator changes momentarily according to the actual operation of the tactile presentation device, the state of change of the browse information in the cooperative virtual space, and the operation when the browsing information is operated using the tactile presentation device. Resonate with new information.

(Effect) In the present invention, the position coordinates of the finger in the real space are compared with the position coordinates of the finger in the virtual space.
When the movement of the finger in the virtual space corresponds to the operation of the viewing target, the actuator drives the actuator to generate a force that hinders the free movement.

Thus, according to the present invention, the sensation received by the operator is an operation resistance, and it is possible to obtain a sensation as if the operator is physically operating the object to be viewed in the real space.

[0236]

Therefore, as described above, according to the present invention, the visual, auditory, and tactile sensations, including the transmission of "tactile" information, which were not possible in the past, using a computer and a computer network. An information browsing system capable of transmitting three kinds of sensory information can be provided.

[Brief description of the drawings]

FIG. 1 is a first view of an information browsing system according to the present invention.
It is a figure explaining the outline of a device composition in an embodiment.

FIG. 2 is a second view of the information browsing system according to the present invention.
FIG. 2 is a diagram for explaining an outline of a relationship between respective functional parts in FIG. 1 as an embodiment.

FIG. 3 is a diagram schematically illustrating a device configuration according to a third embodiment in which the information browsing system according to the present invention is applied to palpation diagnosis in the medical field.

FIG. 4 is a diagram schematically illustrating a device configuration according to a fourth embodiment in which the information browsing system according to the present invention is applied to psychological counseling in the medical field.

FIG. 5 is a diagram for explaining an outline of a device configuration in a fifth embodiment in which the information browsing system according to the present invention is applied to browsing natural materials.

FIG. 6 is a perspective view showing a specific example of a tactile presentation device (force display) used in the present invention.

FIG. 7 is a diagram showing a drawing image of a cooperative virtual space drawn on a client computer in a third embodiment in which the information browsing system according to the present invention is applied to palpation diagnosis in the medical field.

[Explanation of symbols]

1a: Server computer, 1b to 1e: Computer of each facility, 1f to 1i: Client computer, 1j to 1n: Network line, 1o to 1r: Tactile presentation device, 1s to 1v: Camera, 2a: System management function unit, 2b ... Standard browsing data section, 2c ... Standard tactile sense data section, 2d ... Virtual space distribution function section, 2e ... Autonomous search function section, 2f ... Browsing data section, 2g ... Original browsing data section for virtual space generation, 2h ... Virtual space Unique tactile sense generation data section for generation, 2i ... cooperative virtual space construction function section, 2j ... virtual space generation function section, 2k ... sense control function section, 2l ... feature extraction function section, 2m ... virtual space data distribution function section, 2n ... Tactile presentation device, 2o… Camera, 2p… Position detection function part, 2q… Reaction force output function part, 2r… Psychological state estimation function part, 2s… Sensors, 2t actuators, 3a server computers, 3b-3e computers at each facility, 3f-3i client computers, 3j-3n network lines, 3o-3r ... tactile presentation devices, 3s-3v cameras, 3w ... 4a: server computer, 4b-4e: computer at each facility, 4f-4i: client computer, 4j-4n: network line, 4o-4r: tactile presentation device, 4s: client computer 5v ... server computer, 5b-5e ... computer at each facility, 5f-5i ... client computer, 5j-5n ... network line, 5o-5r ... tactile presentation device, 5s-5v ... camera, 1 ... manipulator 2 ... 3 link mechanisms, 4 ... joint, 6 ... moving plate, 8 ... support, 10 ... base, 12 ... fixed block, 16 ... base, 20 ... first link, 24 ... second link, 26: link rotation axis, 28: third link, 30: link rotation axis, 32: link rotation axis, 34: fourth link.

 ────────────────────────────────────────────────── ─── Continuing from the front page (72) Inventor Takao Shiba ▲ ▼ 2-43-2 Hatagaya, Shibuya-ku, Tokyo F-term in Olympus Optical Co., Ltd. 5B049 CC02 DD01 EE03 EE05 EE07 EE12 EE41 FF03 FF04 GG04 GG07 5B075 KK03 KK07 KK13 KK33 KK37 ND03 ND08 ND20 ND23 PP02 PP03 PP10 PP13 PP30 PQ02 PQ13 PQ46 PQ60 PQ64 PQ69 UU40 5E501 AA13 AB15 BA14 BA15 CA10 CB14 FA36

Claims (3)

[Claims] 1. Using data on a network,
A system capable of displaying a virtual space on a network terminal, wherein the data includes tactile and force data, and the network terminal includes a device capable of presenting the tactile / force data to an operator. An information browsing system characterized by the following. 2. Using data on a network,
A system capable of displaying image information on a network terminal, the network terminal comprising: a photographing unit capable of photographing an operator; a feature extracting unit for extracting a characteristic of an expression of the operator photographed by the photographing unit; Based on the features extracted by the feature extraction means,
An information browsing system, comprising: psychological state estimating means for estimating a mental state of an operator. 3. Using data on a network,
A system capable of displaying a virtual space through communication between a plurality of network terminals, wherein at least one of the plurality of network terminals includes a device capable of operating the virtual space, and the virtual space performed by the one network terminal Is provided so as to be observable from at least one other network terminal of the plurality of network terminals.