JP5186238B2 - Virtual communication system - Google Patents

Description

The present invention relates to a virtual communication system through perception, and more particularly to a virtual communication system that enables communication while feeling the presence of a partner between separated persons.

Conventionally, various things are used as a communication means between persons who are separated.
In order to convey each other's thoughts and feelings as words, for example, a mobile phone conveys words through hearing as speech information, and for example, E-mail conveys words through vision. Recently, like a video phone, it is also possible to dodge conversations while reflecting each other's appearance and facial expression on the screen, and it is possible to communicate as if they were facing each other in a pseudo-face-to-face manner. It ’s on stage.

More specifically, in addition to the functions of a normal telephone, a videophone has an imaging unit that images itself, a display unit that displays a partner, and a transmission / reception unit that transmits and receives imaging information. The transmitted image is transmitted to the other party by the transmission / reception means and displayed on the display means of the other party. At the same time, the imaging information of the other party is received by the transmission / reception means and displayed on the display means. Realized.

However, even if the persons who are separated from each other are facing each other in a pseudo manner, it is impossible to communicate while actually touching the other party and feeling the presence of the other party. In addition to verbal communication, communicating with confirmation of each other's appearance, facial color, facial expression, etc., allows the original meaning contained in the word to be transmitted more accurately than in the case of simple verbal communication. It is expected that more accurate communication will be realized if it is possible to communicate while actually feeling the existence of the other party.

For this purpose, for example, when the other party makes a movement that touches himself / herself, it is desired that a tactile sensation or a warm feeling corresponding to the movement is transmitted to him / her in a pseudo manner.
In addition, in the case of a videophone, since an imaging camera cannot be installed behind the screen as a display means, the direction of the line of sight of the screen displaying the other party does not match the imaging direction of the imaging camera that images itself For this reason, even in a mode in which separated persons are facing each other, it is difficult to communicate in a state in which they are staring at each other, that is, in a state where each other's eyes are in line, and communication while feeling the existence of the other party It's hard to say. In particular, when either one moves, the line of sight is further distant, which can be an obstacle to smooth communication.

In view of the above technical problems, the object of the present invention is to perform communication while feeling the presence of the other person more closely by perceiving the other person's tactile sensation or warmth in a pseudo manner. Is to provide a virtual communication system capable of

An object of the present invention is to provide a virtual communication system capable of performing communication in a state in which the eyes of each other are aligned while pseudo-perception of the other party's tactile sensation or thermal sensation between persons who are separated from each other. It is in.

In order to achieve the above object, the virtual communication system of the present invention includes:
A virtual communication system through perception,
Have at least two communication booths separated from each other;
Each communication booth
A display panel arranged so as to be able to face a user in the booth, the transparent display panel being visible to the user from behind the display panel;
An imaging means installed on the opposite side of the display panel from the user in the booth and capable of imaging the user from behind the display panel;
Transmitting means for transmitting the image of the user imaged by the imaging means to the other booth;
Receiving means for receiving an image of the user from the other booth,
The perspective display panel is provided with a free-form surface forming part that forms a free-form surface when pressed by a user,
Furthermore, free curved surface detection means for detecting a free curved surface formed in the free curved surface forming unit,
Transmitting means for transmitting the free curved surface information detected by the free curved surface detecting means to the other booth;
Receiving means for receiving free-form surface information from the other booth;
Based on the free-form surface information received by the receiving unit, the free-form surface restoring unit for restoring the free-form surface is associated with the image of the other booth user displayed on the display panel.

According to the virtual communication system of the present invention having the above configuration, the user in each communication booth is imaged by the imaging means, and the user's image is transmitted to the other party's communication booth and displayed on the display panel. Thus, in each communication booth, the users in the booth and the users displayed on the display panel can be face-to-face.
In this case, the display panel is a perspective display panel through which the user can see through, and the user is imaged by the imaging means from behind the display panel, so that the user can display the display panel as long as both users are stationary. And the imaging direction in which this user is imaged by the imaging means are not greatly deviated, so that it is possible to make a pseudo-face-to-face meeting in a state where the eyes are aligned to some extent.

In such a state, when one user places a hand on the display panel and presses it, the free curved surface formed on the free curved surface portion is detected by the free curved surface detection means, and this free curved surface information is transmitted to the other booth. Then, in the other booth, the free curved surface is restored at the place where the free curved surface is restored by restoring the free curved surface in association with the image of one user displayed on the display panel by the free curved surface restoring means. As a result of the simulation of the movement of one user's hand that is separated and the free-form surface being restored, the trace of one user's hand is reproduced, and the user of the other booth is at the position where the free-form surface is restored. For example, by touching hands, it is possible to realize a state in which remote users are touching each other as if they are touching each other, thereby making it easier to feel the presence of the other party. While it is possible to perform communication.

Further, each of the communication booths has a circular cross section,
Furthermore, face front position detection means for detecting the front position of the face of the user located at the center of the communication booth,
Transmitting means for transmitting the front position information of the user's face detected by the face front position detecting means to the other booth;
Receiving means for receiving front face position information of the user's face from the other booth;
A guide that guides the imaging means and has a circular cross section, and is arranged concentrically with the circular cross section of the communication booth;
Control means for moving the imaging means of the other booth along the guide of the other booth based on the face front position information detected by the face front position detection means of one booth,
Accordingly, it is preferable that the line of sight of both users always be in alignment by controlling the imaging position of the other user displayed for this user according to the movement of the face of one user.

Furthermore, the face front position detection means may include a gaze direction detection means for detecting the gaze direction of the user in the booth and a position detection means for detecting the position of the user's eyes.
In addition, the guide may include a first guide that guides the imaging unit along an arc surface of the guide, and a second guide that guides the first guide in a vertical direction.
Furthermore, the display may be a liquid crystal display.
Furthermore, the free-form surface forming portion has a plurality of elongated member bundles whose side surfaces are held in close contact with each other and can be independently moved in the longitudinal direction,
Each of the plurality of elongated members has the same length, and is formed to form a flat surface at both ends in a state of being bundled with each other,
Thereby, by applying a pressing force to one end surface of the plurality of elongated member bundles, a free curved surface corresponding to the pressing force may be formed on the other end surface of the plurality of elongated member bundles.
In addition, the free curved surface detection means may detect the amount of displacement of the other end of each of the plurality of elongated members.
Further, the perspective display panel is provided with a through hole at a position corresponding to the height of the user's hand,
The through-holes are formed through a support member that bundles and supports the plurality of elongated members from the periphery so that one end surface of the plurality of elongated member bundles is flush with the transparent display panel surface facing the user. It is good to fit inside.

Furthermore, a temperature detection means is provided on one end surface of the plurality of elongated member bundles,
Transmitting means for transmitting temperature information of the user's hand detected by the temperature detecting means to the other booth;
Receiving means for receiving temperature information of the user's hand transmitted from the other booth;
A temperature adjusting means for reproducing the temperature of the user's hand based on the received temperature information of the user's hand may be provided at the other end of the plurality of elongated member bundles.
Furthermore, the temperature detecting means may be in the form of a heat-sensitive seal provided so as to cover one end face of the plurality of elongated member bundles.

Embodiments of the present invention will be described below in detail with reference to the drawings.
As shown in FIG. 1, the virtual communication system 10 has two communication booths 12 (12A, 12B) spaced apart from each other. Each communication booth 12 has a cylindrical shape, has a size that allows a user to enter inside and communicate with other users, and is provided with an antenna 13 so that wireless communication can be performed between the booths 12.

Each communication booth 12 is provided with a display panel 14 over the entire height of the booth 12. The display panel 14 has a planar shape that is large enough to display the whole body of the user. Thereby, it is comprised so that the user in the communication booth 12 can face the other party displayed on the display panel 14.
The display panel 14 is a so-called perspective display panel 14, and when viewed from the front of the display panel 14, an image displayed on the panel 14 can be viewed as the display panel 14. The front side of the display panel 14 can be seen through from behind the display panel 14, and a user located on the front side of the display panel 14 can be photographed from behind the display panel 14 by the imaging means 16 described later. It is possible.

Such a display panel 14 can be realized by a so-called liquid crystal or organic EL display panel 14.
In addition, as a method for realizing imaging and display by seeing through a subject only from the back side, if organic EL is used, the screen reaction speed is faster than that of liquid crystal. It is possible to achieve this by adjusting the display timing of the image to be displayed on the screen and the imaging timing by the imaging means behind, and time-sharing so that the timings do not overlap. That is, in more detail, a process of switching both timings is performed at a switching speed that cannot be perceived by the user. In other words, the display timing of the display panel 14 may be changed so that, when switching, the image display frame is displayed without causing the user to perceive the state where the display panel becomes black in a transparent or translucent state. Imaging can be performed at a timing during reproduction. In this case, in the image picked up by the image pickup means, the video only at the timing when the display panel becomes transparent or the like is extracted, and the still image joining process is performed. At this time, it is desirable that the display panel to be employed employs organic EL whose screen reaction speed is faster than that of liquid crystal.

As shown in FIG. 2, the display panel 14 forms the front surface of the box 11 arranged in the booth 12 near the user, and a free curved surface forming unit 18 described later at a position corresponding to the height of the user's hand. Further, the line-of-sight direction detection means 20 and the position detection means 22 are provided at a height that does not interfere with the imaging of the imaging means 16, and the imaging means 16 is provided on the rear surface of the box 11.
The free curved surface forming portion 18 is formed by the end surfaces 26 of the plurality of elongated members 24. The plurality of elongate members 24 are provided with ring-shaped support members 28 for bundling and supporting the plurality of elongate members 24 from the periphery of the plurality of elongate members 24, so that the plurality of elongate members 24 are side by side with each other. They are held in close contact and can be moved independently in the longitudinal direction. The material of the elongated member 24 may be made of, for example, metal or resin as long as it can be moved in the longitudinal direction accordingly when pressed by the user. The plurality of elongate members 24 have the same length and are positioned so as to form flat surfaces at both ends in a bundled state. The cross-sectional shape and size of the elongated members 24 and the number of the elongated members 24 are determined by applying a pressing force to the one end surface 26 of the bundle of the plurality of elongated members 24, so that the other end surface 26 has a free curved surface according to the pressing force. It may be appropriately selected so as to be formed, but it is preferable to provide several hundred elongated members 24 having a circular cross section with a diameter of about several millimeters in consideration of simulating the movement of the user's hand or the trace of the hand.

The perspective display panel 14 is provided with a through-hole 30 at a position corresponding to the height of a user's hand sitting on a chair or standing in the booth 12, and one end face 26 of the plurality of elongated members 24 is It is fitted in the through hole 30 via the support member 28 so as to be flush with the surface of the transparent display panel 14 facing the user.

Further, a free curved surface detection means 32 is provided to detect the displacement amount in the longitudinal direction of each of the plurality of elongated members 24. This free curved surface detection means 32 may be a normal displacement sensor, for example.
As shown in FIG. 3, a temperature detection unit 34 is provided on a surface (a surface facing the perspective display panel 14) formed at one end of the bundle of the plurality of elongated members 24. It is good to be a heat-sensitive seal provided so as to cover.

A free curved surface restoring portion 19 having a configuration similar to that of the free curved surface forming portion 18 is provided adjacent to the free curved surface forming portion 18 and is formed on one end of a plurality of elongated members 24 bundles of the free curved surface forming portion 18 ( A temperature adjusting means 36 is provided on the surface facing the perspective display panel 14, and the temperature adjusting means 36 is based on the temperature information of the user's hand detected by the temperature detecting means 34 of the other booth 12. The temperature of the user's hand in the booth 12 is reproduced. Thereby, when the user in the booth 12 presses the free curved surface forming unit 18 by hand, the free curved surface forming unit 18 traces the movement of the hand and the trace of the hand, and detects the temperature of the user's hand. I can communicate the feeling of warmth.
The imaging means 16 is composed of a still camera or the like, and images a user in the booth 12 through the display panel 14. The imaging means 16 is a guide 13 having a circular cross section and is guided by a guide 13 arranged concentrically with the circular cross section of the booth 12. The imaging means 16 is supported by the guide 13 so as to be rotatable in any direction, up, down, left, and right, so that the user in the booth 12 can be imaged from an arbitrary direction.
More specifically, the guide 13 includes a first guide 15 that guides the imaging unit 16 along the arc surface, a second guide 17 that guides the first guide 15 in the vertical direction, and the imaging unit 16 as the first guide. 15 includes a drive unit (not shown) that moves along the first guide 15 and a drive unit (not shown) that moves the first guide 15 along the second guide 17. Based on the above, the imaging means 16 of the other booth 12 can be moved along the guide in a circumferential direction and a vertical direction that are concentric with the circular cross section of the booth 12. Any of the driving units may be a commonly used motor. The imaging unit 16 and the first guide 15 have an engaging relationship that allows relative movement with respect to the first guide 15 and the second guide 17, respectively.

The line-of-sight detection means 20 detects the user's line-of-sight direction and transfers the detection signal to the arithmetic processing unit 50 (FIG. 4).
The position detection means 22 detects the position of the user's eyes and transfers the detection signal to the arithmetic processing unit 50. By detecting such a user's line-of-sight direction and eye position, it is possible to specify the front position of the user's face, and when the user moves the face, it can be detected.

As shown in FIG. 4, the functional configuration of the virtual communication system 10 includes an arithmetic processing unit 50, a display control unit 52, a guide driving unit 54, a temperature adjustment unit 56, and an imaging processing unit that are connected to each other via a system bus. 58, a free-form surface restoration unit 60, a gaze direction detection unit 62, an eye position detection unit 64, a free-form surface detection unit 66, a temperature detection unit 68, an external interface unit 70, an input unit 72, a storage unit 74, a communication unit 76, and I / O78.
The arithmetic processing unit 50 is a CPU that controls the entire virtual communication system 10 and performs information processing necessary for virtual communication.
The storage unit 74 is provided with a ROM that stores a program or data executed by the arithmetic processing unit 50, a RAM that constitutes the storage unit 74 necessary for information processing of the CPU, and the like. The input unit 72 receives an instruction from the user and transfers the instruction to the arithmetic processing unit 50.

The display control unit 52 converts the data sent from the arithmetic processing unit 50 into image data displayed on the display panel 14.
The external interface unit 70 is responsible for an interface with an external system. As the external system, a mobile phone, a mobile terminal, a wireless LAN terminal, and the like, as well as specifying whether or not both parties are eligible to communicate with each other An identification electronic terminal (electronic authentication key terminal) can be connected.
The guide driving unit 54 sends a control signal for driving the imaging unit 16 and the first guide 15 from the arithmetic processing unit 50 through the guide driving unit 54.
The temperature detection unit 68 sends the temperature data detected by the temperature detection unit 34 to the arithmetic processing unit 50 through the temperature detection unit 68.
The temperature adjusting unit 56 sends a control signal for controlling the temperature adjusting unit 36 from the arithmetic processing unit 50 through the temperature adjusting unit 56 based on the temperature data detected by the temperature detecting unit 34.
The imaging processing unit 58 processes the imaging data captured by the imaging unit 16 and sends it to the arithmetic processing unit 50.
The line-of-sight direction detection unit 62 sends the line-of-sight direction data detected by the line-of-sight direction detection means 20 to the arithmetic processing unit 50 through the line-of-sight direction detection unit 62.
The eye position detector 64 sends the position data detected by the position detector 22 to the arithmetic processor 50 through the eye position detector 64.
The free curved surface detection unit 66 sends the free curved surface data detected by the free curved surface detection means 32 to the arithmetic processing unit 50 through the free curved surface detection unit 66.

The free curved surface restoration unit 60 sends a control signal from the arithmetic processing unit 50 to the free curved surface restoration unit through the free curved surface restoration unit 60 based on the free curved surface data sent to the computation processing unit 50 through the free curved surface detection unit 66. ing.
The communication unit 76 receives data from the other booth 12 and sends the data to the arithmetic processing unit 50, and transmits data from the arithmetic processing unit 50 to the other booth 12.

The operation of the virtual communication system 10 having the above configuration will be described below by taking as an example a case where virtual communication is performed between the booth 12A and the booth 12B.
First, the users X and Y enter the booth 12A and the booth 12B, respectively, and are positioned almost at the center of each booth 12. The user may be sitting on a chair provided in the booth 12 or standing.
Next, in the booth 12A, the user X is imaged from behind the display panel 14 by the imaging means 16, and the imaging data is transmitted from the communication unit 76 to the booth 12B. On the other hand, in the booth 12B, similarly, the imaging unit 16 images the user Y from behind the display panel 14, and the imaging data is transmitted from the communication unit 76 to the booth 12A. As a result, in the booth 12A, a mode in which the user X is displayed on the display panel 14 and the user Y is displayed in the booth 12B in a pseudo-facing manner with the user X displayed on the display panel 14 is formed. .

In this case, for example, it is assumed that the user X changes the face direction in the booth 12A. At this time, the gaze direction of the user X is detected by the gaze direction detection means 20, and the position of the eyes of the user X is detected by the position detection means 22, thereby detecting the front position of the face of the user X, The front position data is transmitted from the communication unit 76 to the booth 12B.
Next, in the booth 12B, the front position data received by the communication unit 76 is sent to the calculation processing unit 50, and the calculation processing unit 50 moves the imaging means 16 in the booth 12B along the guide 13 according to the front position data. Move Y to the imaging position for imaging.

More specifically, from the state in which the user X and the user Y are facing each other in a quasi-facing manner, the user X has the front position of the face centered on the user X in the booth 12A and the perpendicular direction is the rotation axis direction. Suppose that it is rotated α degrees in the clockwise direction. In this case, if the imaging direction of the user Y is not changed, the user Y and the user X displayed on the display panel 14 in the booth 12A are separated from each other. Therefore, by rotating the imaging direction of the user Y by α degrees counterclockwise around the user Y in the booth 12B, the user Y and the user X can keep their eyes in parallel and keep the eyes in line. It becomes possible.

Through the control described above, even if the user X changes the face direction, it is possible to communicate in a state where the line of sight of the user X and the line of sight of the user Y always match.
Next, for example, it is assumed that the user X touches and pushes the free curved surface forming unit 18 provided on the display panel 14 in the booth 12A. That is, it is assumed that a pressing force is applied to the one end surface 26 of the bundle of the plurality of elongated members 24 toward the back of the display panel 14. Thereby, each of the plurality of elongate members 24 is displaced in the longitudinal direction independently of each other while maintaining the state of being bundled together according to the pressing force applied to itself. The displacement of each elongated member 24 is detected by the free curved surface detector 66. At the same time, the temperature detection means 34 detects the temperature of the hand of the user X.

Next, the detected free-form surface data and temperature data are transmitted from the communication unit 76 to the booth 12B.
Next, in the booth 12B, the free curved surface data and the temperature data received by the communication unit 76 are sent to the arithmetic processing unit 50, and the free curved surface restoration unit 60 causes the free curved surface data in the booth 12B based on the free curved surface data. The free curved surface is restored through the curved surface forming unit 18, and the temperature of the user X's hand is reproduced by the temperature adjusting means 36.
Thereby, the user Y in the booth 12B visually recognizes the movement of the hand of the user X of the booth 12A through the free curved surface restoration unit 60 in the booth 12B, and visually recognizes the trace that the user X has pressed against the user. The user Y also touches the trace to perceive the user X's tactile sensation and warmth, and simulates the state where the user X and the user Y are touching each other with their hands. It becomes possible.

With the control as described above, when the user Y touches the position of the hand of the user X displayed on the display panel 14 in the booth 12B, the hands touch each other in a state where the lines of sight are aligned. It becomes possible to perform such a pseudo-experience, and it is thereby possible for the user X and the user Y who are located apart from each other to communicate with each other while feeling the existence of the other party.

Although the embodiments of the present invention have been described in detail above, various modifications and variations can be made by those skilled in the art without departing from the scope of the present invention. For example, in this embodiment, although the case where there were two communication booths 12 was demonstrated, it is not limited to it and more may be sufficient. In the present embodiment, an example in which the free curved surface forming unit and the free curved surface restoring unit are separately provided on the display panel 14 has been described. However, the present invention is not limited thereto. Unless communication is performed, the free curved surface forming unit and the free curved surface restoring unit are used together, and the free curved surface information from one user is reproduced through the free curved surface forming unit of the other user, and the other restored through the free curved surface forming unit. One user may touch the movement of the user's hand and the trace of the palm.

1 is a schematic diagram of a virtual communication system according to an embodiment of the present invention. It is a fragmentary figure which shows the back part of the display panel of the communication booth which concerns on embodiment of this invention in detail. It is a fragmentary figure which shows the state by which the free curved surface formation part of the communication booth which concerns on embodiment of this invention was provided in the display panel. It is a functional lineblock diagram of a virtual communication system concerning an embodiment of the present invention.

Explanation of symbols

User X, Y
Virtual communication system 10
Communication booth 12
Perspective display panel 14
First guide 15
Second guide 17
Elongated member 24
End face 26
Support member 28
Through hole 30
Temperature detection means 34
Temperature adjusting means 36
Arithmetic processing unit 50
Display control unit 52
Guide drive unit 54
Imaging processing unit 58
Free curved surface restoration unit 60
Gaze direction detector 62
Position detector 64
Free curved surface detector 66
Storage unit 74
Communication part 76

Claims (10)

A virtual communication system through perception,
Have at least two communication booths separated from each other;
Each communication booth
A display panel arranged so as to be able to face a user in the booth, the transparent display panel being visible to the user from behind the display panel;
An imaging means installed on the opposite side of the display panel from the user in the booth and capable of imaging the user from behind the display panel;
Transmitting means for transmitting the image of the user imaged by the imaging means to the other booth;
Receiving means for receiving an image of the user from the other booth,
The perspective display panel is provided with a free-form surface forming part that forms a free-form surface when pressed by a user,
Furthermore, free curved surface detection means for detecting a free curved surface formed in the free curved surface forming unit,
Transmitting means for transmitting the free curved surface information detected by the free curved surface detecting means to the other booth;
Receiving means for receiving free-form surface information from the other booth;
Virtual communication comprising: free-form surface restoration means for restoring a free-form surface in association with the image of the other booth user displayed on the display panel based on the free-form surface information received by the receiving means system. Each of the communication booths has a circular cross section,
Furthermore, face front position detection means for detecting the front position of the face of the user located at the center of the communication booth,
Transmitting means for transmitting the front position information of the user's face detected by the face front position detecting means to the other booth;
Receiving means for receiving front face position information of the user's face from the other booth;
A guide that guides the imaging means and has a circular cross section, and is arranged concentrically with the circular cross section of the communication booth;
Control means for moving the imaging means of the other booth along the guide of the other booth based on the face front position information detected by the face front position detection means of one booth,
Accordingly, by controlling the imaging position of the other user displayed for the user according to the movement of the face of the one user, the line of sight of both users is always in a state of matching. Item 8. The virtual communication system according to item 1. The virtual communication system according to claim 2, wherein the face front position detection unit includes a gaze direction detection unit that detects a gaze direction of the user in the booth and a position detection unit that detects a position of the user's eyes. The guide includes a first guide for guiding the imaging unit along a circular arc surface of the guide;
The virtual communication system according to claim 2 , further comprising a second guide for guiding the first guide in a vertical direction. The virtual communication system according to claim 1 , wherein the display panel is a liquid crystal panel . The free-form surface forming part has a plurality of elongated member bundles whose side surfaces are held in close contact with each other and can be independently moved in the longitudinal direction,
Each of the plurality of elongated members has the same length, and is formed to form a flat surface at both ends in a state of being bundled with each other,
Accordingly, by applying a pressing force to one end face of the plurality of elongated member bundles, a free curved surface corresponding to the pressing force is formed on the other end face of the plurality of elongated member bundles. Virtual communication system. The virtual communication system according to claim 6, wherein the free curved surface detection means detects a displacement amount of the other end of each of the plurality of elongated members. The perspective display panel is provided with a through hole at a position corresponding to the height of the user's hand,
The through-holes are formed through a support member that bundles and supports the plurality of elongated members from the periphery so that one end surface of the plurality of elongated member bundles is flush with the transparent display panel surface facing the user. The virtual communication system according to claim 7, which is fitted inside. A temperature detecting means is provided on one end face of the plurality of elongated member bundles,
Transmitting means for transmitting temperature information of the user's hand detected by the temperature detecting means to the other booth;
Receiving means for receiving temperature information of the user's hand transmitted from the other booth;
9. The virtual communication system according to claim 8, wherein a temperature adjusting means for reproducing the temperature of the user's hand is provided at the other end of the plurality of elongated member bundles based on the received temperature information of the user's hand. The virtual communication system according to claim 9, wherein the temperature detection means has a heat-sensitive seal shape provided so as to cover one end surfaces of the plurality of elongated member bundles.

Images