JP4571816B2 - Dummy head and highly realistic communication device using the same - Google Patents

Description

  The present invention relates to a dummy head and a highly realistic communication device using the same, and in particular, displays a facial expression of a remote conversation person on a dummy head having a face on a display screen formed in a curved shape of an image display device. In particular, the present invention relates to a dummy head that realizes a remote conversation technology with a high sense of auditory and visual presence, and a highly realistic communication device using the same.

In recent years, an elemental technology called tele-existence that conveys visual and auditory realism to remote places has attracted attention, and the realization of high-realistic remote conversation technology that applies them is desired (non- (See Patent Document 1).
As a technology that realizes remote conversation with a high sense of presence, a dummy head with the same acoustic transmission characteristics as the conversation person's head is used to remotely convey the sense of reality of the sound field in which the conversation person is talking. There has been established a technique for transmitting voice data recorded on a ground using a microphone installed on the ground to a remote conversation partner (see Non-Patent Document 2).

Here, since the dummy head has a shape that simulates the shape of a human head, when considering both the visual and auditory sense of reality in a remote location, the dummy head can be used to share the conversation partner in the remote location. It is effective to use a dummy head to communicate the expressions of conversations.
However, it is difficult to transmit human facial expressions with a dummy head. For example, there is a movement to develop a facial robot that simulates changes in facial expression of the human face, but there is a limit to the performance of the actuator that can be used, and the high degree of freedom and richness of the human face It has not yet been possible to reproduce a complex expression (see Non-Patent Document 3). In addition, even when considering the expression of conversational expressions as an image, the image is generally displayed on a flat display device and is not compatible with a dummy head. .
Hideaki Oyama, Naoki Tsunemoto, Taro Maeda, 舘 Susumu, “A Method for Tele-Gigence into Virtual Environments”, Journal of the Robotics Society of Japan, Vol. 12, No. 8, pp. 97-104, 1994 Yuki Tojima, Nao Uematsu, Tatsuya Hirahara, “Dummy Head that Follows Head Movement”, Proceedings of the Autumn Meeting of the Acoustical Society of Japan, 439-440, 2002 Hiroshi Kobayashi, Mitsuro Ichikawa, Dai Senda, Taichi Shiiba, “Development of Face Robot IV, Renovation of Control Points and Improvement of Mechanism”, Proceedings of 20th Anniversary Academic Lecture Meeting, 2C14, 2002

  The present invention solves the above-mentioned problems by displaying the facial expression of a remote conversation person on a dummy head having a face on a display screen formed in a curved shape of an image display device. A dummy head having high presence and a high presence communication device using the same are provided.

It has an image display device that displays a moving image in which the face of a conversation partner at a remote place is displayed on a display screen formed in a curved shape, and has the same sound transmission as the head of the remote conversation partner It consists of a dummy head part with characteristics and a dummy head drive part that drives the posture and position of the dummy head part to the same state as the posture and position of the remote conversation partner, and the surface screen is the length of the chin by the chin joint The dummy head that constitutes the transmission / reception unit in the highly realistic communication is divided not at the mouth but at the middle part of the chin so that it can be changed.
And it has an image display device for displaying a moving image in which a face is constituted by a display screen formed in a curved surface shape and photographing a facial expression of a remote conversation partner, and is the same as the head of the remote conversation partner An image of a dummy head having sound transmission characteristics and a camera that photographs the facial expression of the remote conversation partner to be displayed on the display screen of the dummy head unit on the head or other part of the remote conversation partner The voice of the conversation partner's facial expression shooting unit that transmits the signal to the dummy head unit, and the voice signal of the talker recorded by the microphone installed in the ear part of the dummy head unit to the headphones worn by the remote conversation partner It consists of a signal transmission unit and a dummy head drive unit that drives the posture and position of the dummy head unit to the same state as the posture and position of the remote conversation partner. Rather than the mouth portion so as to be able to change your length, and the dummy head device which constitutes the transceiver unit in the high realistic communications are separated by an intermediate portion of the chin.

Further, to constitute a high realistic communication apparatus constituting the dummy head device in combination pair.

  This invention has a display screen of an image display device formed in a curved surface shape, and uses a dummy head in which the acoustic transmission characteristics of the conversation person's head are matched, thereby providing both visual and auditory realism. Realized remote conversation technology.

The best mode for carrying out the invention will be specifically described with reference to FIGS. 1 and 2. FIG. FIG. 1 is a diagram for explaining the whole embodiment, and FIG. 2 is a diagram for explaining a configuration for driving a mouth.
In FIG. 1, a conversation person A and a conversation person B are remote from each other, and in front of each of the conversation persons A and B, a dummy head 4B simulating the head shape of the conversation partner and A dummy head 4A is installed, and both conversation parties talk while looking at the dummy head 4B or dummy head 4A of the conversation partner. Hereinafter, the embodiment of the present invention will be described from the viewpoint of the conversation person A.

  First, sound recording and transmission will be described. In FIG. 1, the voice uttered by the conversation person A is recorded by the microphone 5B attached to the ear part of the dummy head 4B simulating the head shape of the conversation person B installed facing the person. The voice signal recorded by the microphone 5B is transmitted to the remote talker B in real time, and the talker B can hear the voice produced by the talker A through the headphones 2B with an acoustic presence. it can. This is because the dummy head 4B simulates the head shape of the conversation person B and has the same acoustic transmission characteristics as the conversation person B's head.

Next, image capturing and transmission will be described. In FIG. 1, the expression of the conversation person A is photographed by the conversation partner expression photographing camera 1A fixed to the conversation person A's head via the camera fixing jig 3, and the image signal of the photographed expression is in real time. Then, the data is transmitted to the location of the remote conversation person B. That is, the image is displayed on the display screen of the image display device 6 described later, which is mounted on the face 6A of the dummy head 4A installed facing the conversation person B.
By fixing the head to the conversation partner expression photographing camera 1 A conversation party A through the camera fixing jig 3, can be reliably capturing facial expressions as seen from the front of the conversation's A. As the image display device 6, an organic electroluminescence image display device formed by an organic electroluminescence element capable of forming a display screen in a curved surface shape is used (Keiji Koji, “All about organic EL”, Nihon Jitsugyo Shuppan, 2003). The conversation person B knows the expression of the conversation partner A by displaying a moving image obtained by photographing the expression of the conversation partner at the remote place on the image display device 6 having a face formed on the display screen formed in a curved surface shape. be able to.

Incidentally, conversation fixing position of the opponent expression photographing camera 1A, the conversation party to a place other than the head A, conversation partner expression photographing camera 1 dummy head plus the processing on the image signal captured by the A 4 A There may be a method of displaying by the image display device 6. In the above photographing, since the conversation partner facial expression photographing camera 1 and the conversation person's face are fixed at a constant distance, there arises a problem that the acquired image is photographed with different magnifications in the central portion and the peripheral portion. By the way, it is possible to calculate which point in the image captured by the conversation partner facial expression photographing camera 1 corresponds to which point on the face as long as there is face shape data. By displaying an image of a corresponding point in the captured image at a corresponding point on the dummy head face at the time of display, this process can be simplified and this problem can be solved.

By the way, the display screen formed in the above curved shape forms the face 6A itself of the dummy head 4A. Referring also to FIG. 2, as an example, the eye 91 is formed in a shape with the eyelids closed, but the closed eye line is not explicitly formed. This is because the image displayed in accordance with the opening and closing of the conversation person's eyes and the eyes 91 formed here are added together so that they can be seen without any failure. The same applies to the mouth 92, and a configuration that can change the length of the jaw according to the movement of the jaw can be provided. Reference numeral 93 denotes a chin joint. By rotating this, the length of the chin attached to the chin joint 93 is changed and adjusted. It is preferable to divide the jaw around the middle of the jaw, not at the mouth 92 portion. The human face is deformed depending on the facial expression, and there is a possibility that the peripheral part may be distorted at this time, but these can be solved by displaying an image of the lower half of the chin on the mouth part. .

  Here, a configuration for supporting the dummy head will be described. In FIG. 1, a dummy head driving device 7A is employed that rotates the posture of the dummy head 4A with three degrees of freedom and translates the position with three degrees of freedom. Transmitting data related to the posture and position of the talker A measured by the talker posture and position measuring device 8A to the remote dummy head drive device 7A and controlling the posture and position of the dummy head 4A based on this data. Thus, the conversation person A can talk while taking a favorite posture. Similarly, the conversation person B can also talk while taking a favorite posture. It is necessary to synchronize the transmission of the data signal related to the posture and position of the conversation person A and the transmission of the image signal measured by the conversation person attitude and position measuring device 8A. It is conceivable to record the time of shooting and the time of exercise, transmit them together with signals to a remote location, and synchronize on the playback side.

  As described above, according to the present invention, the display screen of the image display device is formed into a curved surface shape by using a material having high flexibility such as organic EL, and the face 6A of the dummy head portion is configured by this display screen. By displaying the expression of the remote talker on this, the talkers in the remote place can talk with high sense of reality both auditorily and visually.

The figure explaining an Example. The figure explaining the structure which drives an opening | mouth.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Camera for a conversation partner facial expression photography 2 Headphone 3 Camera fixing jig 4 Dummy head 5 Microphone 6 Image display device 7 Dummy head drive device 8 Conversation posture and position measurement device

Claims (5)

It has an image display device that displays a moving image in which the face of a conversation partner at a remote place is displayed on a display screen formed in a curved shape, and has the same sound transmission as the head of the remote conversation partner A dummy head having characteristics;
A dummy head drive unit that drives the posture and position of the dummy head unit to the same state as the posture and position of a remote conversation partner,
The display screen is divided at the middle portion of the jaw, not at the mouth so that the length of the jaw can be changed by the jaw joint, and constitutes a transmission / reception unit in high presence communication Dummy head. The dummy head according to claim 1, wherein
A dummy head, wherein the image display device is an organic electroluminescence image display device. It has an image display device that displays a moving image in which the face of a conversation partner at a remote place is displayed on a display screen formed in a curved shape, and has the same sound transmission as the head of the remote conversation partner A dummy head having characteristics;
A camera that captures the facial expression of the remote conversation partner to be displayed on the display screen of the dummy head unit is installed on the head or other part of the remote conversation partner, and the captured image signal is transmitted to the dummy head unit. Conversation partner facial expression shooting part,
A voice signal transmission unit for transmitting a voice signal of a conversation person recorded by a microphone installed in an ear part of the dummy head part to headphones worn by a remote conversation partner;
A dummy head drive unit that drives the posture and position of the dummy head unit to the same state as the posture and position of a remote conversation partner,
The display screen is divided at the middle portion of the jaw, not at the mouth so that the length of the jaw can be changed by the jaw joint, and constitutes a transmission / reception unit in high presence communication Dummy head device. The dummy head according to claim 3,
A dummy head device, wherein the image display device is an organic electroluminescence image display device.   A highly realistic communication device configured by combining a pair of dummy head devices according to claim 3.

Images