WO2002032135A1

WO2002032135A1 - Method and device for communication systems

Info

Publication number: WO2002032135A1
Application number: PCT/SE2001/002173
Authority: WO
Inventors: Per Nystedt
Original assignee: Telia Ab (Publ)
Priority date: 2000-10-10
Filing date: 2001-10-08
Publication date: 2002-04-18
Also published as: SE0003690L; SE0003690D0

Abstract

The present invention concerns a method and a device for communication systems, in order to make the visualisation of, for example, three-dimensional images incorporated into the local surroundings possible for communicating parties. Images are taken with the aid of a camera of the surroundings at each communicating party. The images are transmitted between the parties and projected in combination with the surroundings the each party sees around him or her. Furthermore, the parties have the possibility of interacting with the said images, whereby other parties experience the interaction as if they were interacting with the real object.

Description

METHOD AND DEVICE FOR COMMUNICATION SYSTEMS.

TECHNICAL AREA

The present invention concerns systems for transmission of three-dimensional images between different parties in a communication system. Its use takes place in internal systems and/or in traditional telecommunication systems, for example, fixed telephony and mobile telephony.

TECHNICAL PROBLEM

The need exists during communication between different individuals to observe one and the same view/object/document from a common viewpoint. This observation must be possible even if the individuals who are involved in the observation are located at a distance from each other. This means either that the persons can be present at the same geographical location, or that the persons are present at different geographical locations. The communication between the observers may take place with the aid of telecommunication and/or radio communication. Three-dimensional observation of the object/text must hereby be possible for all the parties participating in the communication.

Furthermore, it is desired that the parties who participate in a conference, communication, etc., should have the possibility to give visual indications in the document or against the object in question. These visual indications can be carried out by using a finger, pointing object, etc. In order for these indications to be clear, it is necessary for the person who is pointing to have the possibility of pointing in the three-dimensional image that this person has in front of him or her. Thus, an observer must have the possibility of seeing not only the real object in its own surroundings, but also images constructed in three dimensions that can be transmitted from another person. The local real surroundings must thus be linked with the received surroundings as one integrated three-dimensional image that each contracting participant in the conference/communication has the possibility of using in the common communication.

The need of being able to interact between different parties in a communication when observing three-dimensional images in a situation in which the parties are located in different locations has existed for a long time without any solution to the problem being revealed. It is the aim of the invention to solve the problems described above. PRIOR ART

Communication between different individuals via telecommunication systems of different types is already known. Examples of such telecommunication systems include teleconferencing systems that are combined with image transmission. Images of participants and objects/documents are transmitted two-dimensionally in these known systems.

Furthermore, arrangements of different types are available for producing three-dimensional images. These systems are based on the transmission of two images of one and the same object and their subsequent display for each eye of a person, whereby an illusion of three- dimensional vision is obtained. Examples of how this is carried out are spectacles in which each image is projected in order to be seen by each eye. These spectacles can provide by different technologies the possibility of seeing not only the local surroundings but also the transferred image. The amount of the local surroundings relative to the amount of the transmitted image that is observed can be adjusted according to the desires of the observer. It is thus possible to observe only the transmitted image or only the local surroundings, or to observe a continuous integration of the two surroundings. The possibility of interaction between communicators is not provided by the known systems.

SOLUTION

The present invention concerns a method and device to make the visualisation of images possible for communicating parties, incorporated into the local surroundings. The parties use visualisation devices that project the images received at the receiver in combination with the local surroundings for achieving this purpose. Furthermore, each party is allowed to interact in the images that are created whereby other parties experience the said interaction as if the interaction were carried out with the object in reality.

Images are recorded using cameras that are located in a manner that is equivalent to the location of the eyes in a human. Each camera is hereby, for example, placed on individuals beside the eyes of the said individual. The relevant images are projected in front of the eyes of each person, onto, for example, special spectacles for this purpose. The local surroundings are presented in combination with the transmitted images whereby hands, pointing objects, etc., that are introduced in front of a person act against the image in a manner that is experienced as if the person acted against the object in question. The presentation of images occurs simultaneously and in real time for each party.

ADVANTAGES

One advantage of the present invention is that participants located at a distance from each other are allowed to observe one and the same object in the same view as a person who observes the object in reality. Furthermore, people who only see a three-dimensional image of the object are given the possibility of interacting with other persons who see the said object. In this way, it is possible for the persons to, for example, point at features of the said object on a three-dimensional image created at the observer. In that the pointing is transmitted to other observers/displayers, these are allowed to experience the said pointing against the object as if the pointing were to occur against the real object.

DESCRIPTION OF THE DRAWINGS

Figure 1 Overview :

Two persons A and B use the system, person A (closest in the figure) holds a piece of paper.

The small figure (lower left corner) illustrates use over large distances.

Figure 2 Overview: Two persons A and B use the system, person A (closest in the figure) holds a piece of paper. The small figure (lower left corner) illustrates use over large distances. 2A This is what person A sees WITHOUT spectacles

2B This is what person B sees WITHOUT spectacles

2C This is what person A sees WITH spectacles

2D This is what person B see WITH spectacles

Figure 3 The figure shows how persons that are part of the field of view of each party are experienced by the other party. The figure refers to persons a and b present at party A and B, respectively. PREFERRED EMBODIMENTS

The invention will be described below based on the drawings, and the reference figures in them.

The present invention concerns a method and a device for a communication system to make possible visualisation of three-dimensional images at communicating parties incorporated into the local surroundings. Images are taken of the surroundings of each communicating party with the aid of cameras. The images are transferred between the parties and projected in combination with the surroundings that each party sees around them. Furthermore, the parties have the possibility of interacting with the said images, whereby other parties experience the interaction as if they were interacting with the object in reality.

In a first example, two communicating persons are provided with spectacles (the use of spectacles is only one example of how the invention may be implemented, which is why other solutions can be used within the framework of the innovative concept) onto which images from the other party are projected at the same time as the local surroundings can be observed. Furthermore, cameras are applied to each pair of spectacles in a manner that is equivalent to the eyes of a person. The positioning of the cameras may be carried out in different methods whereby the recreation of three-dimensional images is to be preferred. The images that are recorded by the cameras are transmitted to the other party and projected onto his or her spectacles whereby the view seen by the first person is transmitted to the second person. In that the spectacles in a first variant are made transparent, both the local and the transmitted surroundings will be integrated to a single image, which is displayed three-dimensionally. In further embodiments, the transmitted images are displayed for short intervals of time while the local surroundings become apparent during the corresponding interruptions. Furthermore, visualisation can be achieved by both the local and the received surroundings being displayed on the spectacles. One advantage of the latter method is that all parties see exactly identical surroundings, and that the local cameras can be used to enlarge or reduce the local image. This means that a greater degree of control is available in this case, something that may be an advantage in certain circumstances.

A further example is supervision during, for example, surgery, where a supervisor is located at a distance from/at another geographic location than the supervised. The supervised is in this case located at the operating table. The supervised sees through his or her spectacles the object for the procedure, and this image is transmitted to the supervisor. The supervisor, who observes the transmitted image through his or her spectacles, can thus provide instructions during the surgery by pointing, gesticulating and by giving verbal information. Thus it is possible to use the invention in particular situations of an educational nature where there are difficulties associated with all participants observing one and the same view. Further examples are the demonstration by a lecturer/teacher of complicated operations, procedures during surgery, objects, technical equipment, etc., where it is difficult for the participants to participate simultaneously and see what is displayed. In that the images are displayed for all participating parties simultaneously and in real-time, the possibility is given to point and demonstrate without, for example, disturbing surgery that is carried out.

According to Figure 1 , two parties A and B communicate with each other. The communication can take place either via direct radio communication or over various types of communication such as telecommunication networks, mobile telephone networks, etc. Person A in Figure 3 has a piece of paper with text in their hand in front of them, a box lies in front of Person B. That which person A sees, that is, a piece of paper in the local surroundings, is recorded by person A's spectacles. Furthermore, Person B's spectacle record what this person sees, that is, the local surroundings and, among other things, a box. The recording occurs in such a manner that the image can be experienced as three-dimensional when it is recreated. The spectacles of both person A and of Person B transmit the recorded image to the spectacles of the other person. (A's spectacles transmit to B's spectacles, while B's spectacles transmit to A's spectacles.) The transmitted image is projected in the spectacles of each receiver together with the local view of the receiver. This means that A's piece of paper and B's box are displayed together for both person A and person B. Thus it is possible to point to both real and to imaginary objects that are observed through the said spectacles. A piece of paper and a box have been observed in the example, but the object can be replaced by different types of objects without limitation.

A and B have in their surroundings a person a and b, respectively, whom they observe in reality. The image of these persons are recorded by A's and by B's spectacles G through the camera devices that are arranged on the spectacles and that record the local surroundings and transmit the image of these to the other person. The transmitted image is projected onto the spectacles of the receiver such that the persons a and b are displayed together in a three- dimensional image at A and at B. The persons A and B can, through the use of pointing tools or their own hands or fingers, point both at a and at b, something that is experienced by the other party as if the pointing had occurred in reality. Thus it is possible to point at both real and imaginary objects that are observed through the said spectacles G. People have been observed in the example, but people can be exchanged for objects of different types without limitation. The advantage of the invention is that all observers, in this case A and B, see the same view at the same time and in real-time, whereby misunderstanding due to the fact that persons A and B not only do not see the same thing, but also misunderstand each other's references, or see different views of the same object in cases where they are situated at one and the same place during observation of the object, can be avoided. In cases in which the persons are located at different geographical locations during observation of the object, the risk for misunderstanding as a result of the fact that a verbal description is based upon the impression of the other person, or due to the fact that a transmitted two-dimensional image does not provide the required information, is further reduced. The invention is not limited to the embodiment described above or to the subsequent claims, and it can be subjected to modifications within the framework of the innovative concept.

Claims

PATENT CLAIMS

1. Device for communication systems that makes the visualisation of three-dimensional images incorporated into the local surroundings possible for communicating parties, characterised in that the parties are arranged to use display arrangements that project an image received from a first party in combination with the surroundings present at a second party to form a combined image, that the second party in an equivalent manner is arranged to receive an image and combine his or her local surroundings with this image to form a corresponding three-dimensional image, and that the first and the second parties are arranged to interact in the created images against real objects that exist at the corresponding party.

2. Device according to claim 1, characterised in that the images of the local surroundings at the first and second parties are recorded by cameras whose positions are equivalent to the eyes of the each party, that the images are projected in front of the corresponding eye of the second or the first party, respectively, which is arranged to create a three-dimensional image of the surroundings of each party at each receiver.

3. Device according to claim 1 or 2, characterised in that a real object in the world of the first party is arranged to be displayed three-dimensionally at the second party, who is arranged to point and demonstrate in the three-dimensional image created, with the aid of hands and/or pointing tools, whereby the first party is arranged to experience the said pointing as if the pointing were carried out against the real object.

4. Method for communication systems that makes the visualisation of three-dimensional images incorporated into the local surroundings possible for communicating parties, characterised in that the parties are provided with display arrangements that project images received in combination with the local surroundings, whereby the parties are allowed to interact in the images created at each party or against real objects that exist at the corresponding party.

5. Method according to claim 4, characterised in that the images of the local surroundings at each party are recorded by cameras whose positions are equivalent to the eyes of each party, that the images are projected in front of the corresponding eye of the party or parties who receive the recorded images and combine these images with the local surroundings, whereby a three-dimensional image of the local world and the received world is created.

6. Method according to claim 4 or 5, characterised in that a real object at one of the parties is displayed three-dimensionally at other parties, which other parties are allowed to point, with the aid of hands or pointing tools, at features in the received three-dimensional image, whereby the one party that possesses the real object experiences the said pointing as if the pointing pointed at the real object.

7. Method according to any one of the claims 4-6 characterised in that the imaging at each party occurs simultaneously and in real-time.