JP4431746B2 - Indirect motion transmission apparatus and method - Google Patents

Description

  The present invention relates to an apparatus and a method for transmitting a motion of a head or the like by an indirect means instead of a live image.

  Now that it has become easier to talk over long periods of time without worrying about telephone charges, such as IP telephones using Internet lines, etc., the use of telephone time has increased more than before. A person who uses such a telephone often sees the telephone and its surroundings, but the telephone remains the same no matter how much it is viewed. If you have been on the phone for a long time, you tend to feel like you are talking to the other party and talking to it.

  This phenomenon seems to be caused by the tendency to anthropomorphize the part that is likely to be a face in an object that we usually see often. In particular, the tendency is strong if it exists at the line of sight, a circular shape that is likely to be seen, or a shape that is nearly symmetrical. For example, when anthropomorphizing a PC, you will often see an example where the eyes and nose are drawn on the PC monitor. Actually, the computer itself is equipped with a computing device and storage device, but there are few examples of drawing the eyes, nose and mouth on the PC body. Similarly, if the eyes and nose are drawn on the telephone, it is not a handset but a telephone body with a dial and buttons.

  In recent years, it has been advancing to talk with close friends in the form of a video phone using a live camera. However, when faced for a long time, it may cause mental fatigue compared to voice-only dialogue. There may also be problems with face-to-face conversations and clothes after removing makeup. Therefore, there are many cases where it is desirable to transmit only the head movement without directly showing the face or the body itself.

  When people interact naturally, head movements such as whispering and tilting occur naturally. If this head movement can be transmitted to the other party in real time with a slight extension without replacing the existing telephone material, it is possible to easily increase the realism of the dialogue.

  On the other hand, there is no denying the possibility that someone with malicious intent peeks at the information that is being transmitted by the device that relays the communication. Also, many people have resistance against placing a TV camera in the room. Even when the power is turned off, there is a fear that the video is still on and this video may be missing.

  However, conventionally, there has been no technique for transmitting the movement of the head or the like that occurs when a person has a natural conversation with an indirect means instead of a live image of the body.

  Therefore, the present invention provides an apparatus and method for transmitting the motion of an object, such as the motion of the head that occurs when a person has a natural conversation with each other, using indirect means instead of the live image. This is the issue.

In order to solve the above-described problems, the motion indirect transmission device of the present invention has the following configuration.
That is, an apparatus that indirectly transmits an action of an object by reproducing the action of the object as an action of an agent, and each amount of variation in three independent directions with respect to the action of the target object. In each direction, the operation information receiving means for receiving the operation information including the information to be shown, the operation information component dividing means for dividing the received operation information into components in each direction and obtaining each variation amount information for each direction, It is characterized by comprising an operation reproduction means for driving and changing the placement unit on which the agent is placed by an amount corresponding to the fluctuation amount information.

  Here, the three independent orientations are an orientation that rotates in a substantially horizontal direction, a front-rear rotation orientation that rotates perpendicularly to the substantially horizontal rotation orientation, and a left-right rotation that rotates perpendicularly to both the substantially horizontal rotation orientation and the front-rear rotation orientation. You may make it contribute to the operation | movement by a joint as a rotation direction.

  The motion reproducing means may be provided with an angle adjusting unit that supports the mounting unit and adjusts the support angle, and may contribute to a simple configuration.

  The angle adjustment unit includes a horizontal rotation shaft that rotates in a substantially horizontal rotation direction, a yaw adjustment unit that includes a drive member that adjusts the rotation angle (yaw angle), a front and rear rotation shaft that rotates in the front-rear rotation direction, A pitch adjusting unit having a drive member that adjusts the rotation angle (pitch angle), a roll adjusting unit having a left and right rotation shaft that rotates in the left-right rotation direction, and a drive member that adjusts the rotation angle (roll angle); A thing provided with is suitable.

  The target object may be a person who is having a conversation with the telephone, and the movement of the object may be a movement of the head that occurs when a person and a person have a natural conversation, thereby contributing to an improvement in the realism of the telephone.

  The receiver is placed on the coupler, the receiver's speaker faces the coupler's microphone, and the receiver's microphone faces the coupler's speaker, and the coupler's microphone is connected to the speaker provided on the mounting portion. At the same time, the speaker of the coupler may be connected to a microphone provided in the placement unit to contribute to the improvement of the sense of presence on the audio side.

  As an agent, a imitation imitating the shape of a human head is provided, and the head imitation is provided with a hollow pipe line from the bottom facing the speaker of the placement unit to the mouth along the airway equivalent part. Are preferred.

  You may comprise an agent with the display apparatus which can change a display content dynamically.

  Then, the partner area output from the local situation search unit based on the partner agent specific information output from the partner information detection unit that detects information about the target object and the partner region information output from the partner information detection unit The agent image formed by combining the situation information with the agent combining means is displayed on the display device, so that it is possible to exchange information closely.

  Of the operation information including information indicating each variation amount in three independent directions, the variation amount in at least one direction is expressed as display contents displayed on the display device, thereby contributing to simplification of the configuration. Also good.

  A communication unit that enables simultaneous communication between a large number of operation indirect transmission devices may be provided for the convenience of a conference system with a large number of people.

  The motion indirect transmission method of the present invention is a method for indirectly transmitting the motion of an object as a target by reproducing the motion of the object as a motion of an agent. With regard to the motion, the motion information including information indicating each variation amount in three independent directions is received, and the motion information component dividing unit divides the received motion information into each direction, and each variation amount for each direction The information is obtained, and the placement unit on which the agent is placed is driven and changed in an amount corresponding to the change amount information in each direction by the action reproducing means.

According to the present invention, an agent such as a telephone body or a person imitating a person's head operates in response to an action of a conversation partner's head or the like that occurs when a person and a person have a natural conversation. Therefore, the realism of dialogue is improved.
At this time, since the other party's movement is transmitted as movement information by an indirect means instead of a live image, privacy is also protected.

Embodiments of the present invention will be described below with reference to the drawings.
There are various ways to reproduce the movement of the other speaker's movement information when communicating with the other party's speaker. It is also possible to play back the motion. In addition, if the telephone body moves in response to the body movement of the other party's head or the like, there is an advantage that the realism of the dialogue is also increased.
The conventional telephone has no moving parts. In the first place, a telephone is a communication device for exchanging voice information with each other, and it is impossible to exchange operation information such as heads with each other unless a dedicated device is used.

The present invention is basically based on the fact that yaw, pitch, and roll information related to the head movement of one of the speakers is transmitted to the other speaker and reproduced in substantially real time. The reproducing apparatus is called an agent table in this embodiment.
The partner speaker regards the object on the agent table as the speaker agent.
It is desirable for both parties to have dialogue to use this device, and even if one speaker is unilaterally standing, the whispering etc. can be visually observed, and the understanding situation of the story can be easily known. Can do.

  Note that the present invention relates to an agent table, and the user can arbitrarily change what is placed on the agent table. In addition to placing a communication device such as a telephone, it is of course possible to place a head model that imitates the face of the other party, a photo of the face, and an object that has nothing to do with the face.

FIG. 1 is an explanatory diagram showing an outline of an agent table on which a telephone body is mounted as an agent.
The agent base has a substantially dish-shaped placement portion (21) for placing the agent (10), and an angle adjustment portion (22) that supports the placement portion (21) and adjusts the support angle. And a pedestal portion (23) for supporting and fixing the angle adjusting portion (22).

  The angle adjuster (22) is a yaw adjuster (31) that has a horizontal rotation shaft that stands on the pedestal (23) and rotates in a substantially horizontal direction, and a drive member that adjusts the rotation angle (yaw angle). A pitch adjusting unit (32) having a front and rear rotating shaft that is connected to the yaw adjusting unit (31) and rotates perpendicularly to the horizontal rotating shaft, and a drive member that adjusts the rotation angle (pitch angle); And a roll adjusting section (33) having a left and right rotating shaft that is connected to the pitch adjusting section (32) and rotates perpendicularly to the front and rear rotating shaft, and a drive member that adjusts the rotation angle (roll angle).

FIG. 2 is an explanatory diagram showing the flow of operation information.
The motion information related to the movement of the speaker's head includes the direction and amount of movement, the motion speed, acceleration, and the like. In particular, regarding the moving direction, it is information in which three components of yaw, pitch, and roll are mixed, and it consists of information by electrical, magnetic, or mechanical means.
The combined information of the yaw, pitch, and roll (solid arrow in FIG. 1) is divided for each yaw, pitch, and roll by the operation information splitter (40) (dashed line in FIG. 1). That is, in this embodiment, one input in which three information is mixed is divided into three outputs as the twist angle information of each of yaw, pitch, and roll.

  Information on the three twist angles of yaw, pitch, and roll is output as an analog signal such as voltage (high voltage: large twist angle, low voltage: small twist angle, etc.) or a digital signal such as a pulse, respectively. The signals are input to the respective torsion decoders provided in the yaw adjusting unit (31), the pitch adjusting unit (32), and the roll adjusting unit (33).

FIG. 3 is an explanatory diagram showing adjustment means in each of the yaw adjustment unit (31), the pitch adjustment unit (32), and the roll adjustment unit (33).
Each yaw / pitch / roll twist angle information divided and output is first input to the controller in each torsion decoder. The controller obtains the torsion angle information and the angle information currently indicated by the placement part (21) of the agent table via a sensor such as a potentiometer, and the agent is moved to the desired posture from the difference between the two information. In order to move the table, a signal such as a voltage or a pulse for operating a motor or the like is generated.

The signal generated by the controller is transmitted to driving means such as a motor capable of controlling the rotation angle, and the driving means operates according to the signal.
The power generated by the driving means such as a motor is directly or via power transmission means such as a gear box, hydraulic pressure, pneumatic pressure, water pressure, wire, sprocket and chain, etc. (33), the yaw / pitch / roll twist angle is adjusted to move the agent table to a desired position.

The agent platform moves by this power, but the current position at that time is recognized by various sensors including a rotary encoder, and the information is fed back to the controller.
As a result of the output from the controller, too little or too much movement due to some reason is recognized by the controller from the information from this sensor, and in order to complete the processing inside each torsion decoder so as to be in the correct position, the driving means is used. Generate a signal for operation again.
Note that the controller constantly adjusts the position given from the motion information splitter (40) in real time in consideration of both inputs of the motion information splitter (40) and a sensor such as a potentiometer.

  As the driving means, a stepping motor or the like that adjusts the rotation angle by a pulse can be used.

As a sensor for monitoring the change of the rotation angle, a rotary encoder for measuring the angular displacement, a resolver for converting the rotation angle displacement into an electric quantity using the structure of a two-phase AC motor, and the like can be used. These are preferably arranged at positions where the rotations of the adjusting sections (31), (32), and (33) can be directly measured.
Further, a gyro sensor or the like for examining the rotational angular displacement using the gyro principle can be used.

Examples of the agent (10) include a communication device such as a telephone body (11) that is normally used as shown in the figure, and a telephone receiver or an intercom can be used for a call with the other party's speaker.
An open-type microphone and speaker are useful for natural dialogue with the other party.
FIG. 4 is an explanatory diagram showing an outline of an open type agent stand device.
When the handset (12) is placed on the coupler (50), the speaker (13) faces the microphone (51) provided on the coupler (50) and the microphone (14) is provided on the coupler (50). It arrange | positions facing the made speaker (61).

The microphone (51) of the coupler (50) is connected to the speaker terminal (53) provided on the placement unit (21) of the agent table via the amplifying device (52) with a variable device, and the speaker terminal (53 ) Is connected to a speaker (54) with a terminal.
The speaker (61) of the coupler (50) is connected to the microphone terminal (63) provided on the placement unit (21) of the agent table via the amplifying device (62) with a variable device, and the microphone terminal (63 ) Is connected to a microphone (64) with a terminal.
Thus, a more natural dialogue can be performed toward the agent (10) without holding the handset.

The agent (10) may be an article imitating a human head, not such a communication device.
FIG. 5 is an explanatory diagram showing an outline of an agent stand device that imitates the head.
As the agent (10), a human head imitation (15) is placed on the placement unit (21).
The head imitation (15) is provided with a pipe line (16) to the mouth along the portion corresponding to the airway. The pipe line (16) may be constituted by passing a hose through the hollow head imitation (15), or may be constituted by making a hole in the solid head imitation (15).
In addition, the part corresponding to the mouth is made with a hole or a material that allows easy passage of sound.

When this head imitation (15) is applied to the open-type agent stand device of the above embodiment, for example, a sound can be heard from the right to the left with respect to the yaw change of the face, and according to the action of the agent (10). Because the sound is synchronized and transformed, the realism of the dialogue is further increased.
In order to reproduce the natural left / right swing of the sound generation point with respect to the yaw change without using this configuration, generally two or more speakers are required. On the other hand, according to the present invention, it is easily reproduced by one speaker.

FIG. 6 is an explanatory view showing another embodiment of the agent platform device imitating the head.
The difference from the embodiment is that the speaker (55) and the microphone (65) are provided inside the agent (10).
That is, the speaker (55) installed in the mouth-corresponding portion is connected to the speaker terminal (53) provided in the placement part (21) of the agent table via the cable (17), and corresponds to the forehead or ear. The microphone (65) installed in the part is connected to the microphone terminal (63) provided on the placement part (21) of the agent table via the cable (18).
According to this method, since the shape of the agent (10) is flexible, it does not stick to the shape of the agent (10). For example, it looks like a stuffed animal such as a cat or a non-vase. It may be a living organism.

FIG. 7 is an explanatory view showing another embodiment of the agent stand apparatus using a monitor, and FIG. 8 is an explanatory view illustrating the flow of processing in the apparatus.
A display device (19) that can dynamically change the display content, such as a monitor, instead of the user placing the agent (10) having a specific shape or a face photograph of the other party by himself / herself. ) Is also possible.
Since the display device (19) is only placed on the placement unit (21) of the agent table, the yaw / pitch / roll operation is performed according to the operation of the agent table.

The partner information detection means obtains information about the partner when sending and receiving communications. For example, the other party's speaker can be specified to some extent from a service that displays a telephone number when receiving communication from the other party, or in the case of an IP phone, the other party's IP address or other specific information. It is also possible to specify who the other party is based on the voice and movement characteristics of the other party.
In this partner information detection means, the user can also input various information of the partner as a database during standby (non-communication). For example, name, telephone number, IP address, regional information, type of partner's agent image, and the like.

This partner information detecting means selects information unique to the partner agent, such as an appropriate agent image and symbol of the partner area, from these inputs, and sends it to the agent composition means.
Depending on the speaker who is communicating, for example, if the conversation partner is a friend, an image pre-assigned to a friend is used, and if it is a stranger, an appropriate image “appropriate from the information of the other party” is used. Here, “appropriate from the other party's information” means that, for example, if you use IP address or landline number information, the other party's image that reflects the information on the area where the other party lives ( Person, typical American agent in the United States, etc.), and the gender and age are estimated from the voice quality itself, and the typical partner suitable for the gender and age is selected from the many agent images prepared in advance. Select an image. Of course, it does not have to be humanoid.

  Here, even if the pattern of the agent image is defined by the user, for example, it reads the change in characteristics such as cold and wake up compared to the voice pattern registered by the user, and Can be reflected in the agent image as a facial expression.

At the same time, the partner information detection means outputs the partner's area information to the area status search means.
In order to know the situation of the environment surrounding the current communication partner from the local information of the other party, the local information search means checks the current weather and time according to the other party's whereabouts in real time, for example via the Internet line. The weather and time information is sent to the agent composition means.

  The agent compositing means uses information specific to the partner agent such as “agent image, local symbol” sent from the partner information detecting means, and information on the partner area such as “weather / time” sent from the local situation search means. The situation information is synthesized and output to the agent display means as a final agent image.

FIG. 9 is an explanatory diagram illustrating information specific to the partner agent and status information of the partner region.
FIG. 9 (a) shows wake-up as an agent image pattern, FIG. 9 (b) shows a shrine as a symbol of the partner area, and FIG. 9 (c) shows a rainy day as the situation of the partner area. Is shown.

FIG. 10 is an explanatory diagram showing another embodiment of an apparatus having a monitor as an agent.
The difference from the embodiment is that the display device (19) does not always operate.
Similarly to the above-described embodiment, the display device (19) may be placed on the placement unit (21) of the agent table and operated by yaw, pitch, or roll. By fixing, at least one or two directions of the yaw, pitch, and roll may be fixed, and all directions may be fixed.
FIG. 10 shows an example in which the display device (19) is fixed without any yaw, pitch, or roll. In this case, the information on the yaw / pitch / roll operation is expressed as the yaw / pitch / roll operation of the display content of the display device (19). Even if the display device (19) is completely fixed, the display content includes information specific to the agent and status information of the partner area.

  The communication as described above is not limited to a one-to-one conversation, but can be applied to a conference system with a large number of people if a plurality of apparatuses according to the present invention are prepared so that they can communicate with each other.

  According to the motion indirect transmission device and method of the present invention, the motion of an object, such as the motion of the head that occurs when a person and a person have a natural conversation, can be transmitted by indirect means instead of the live image. Therefore, it is possible to improve the realism of communication such as telephone conversation while maintaining confidentiality.

Explanatory drawing showing the outline of the agent stand with the telephone body as an agent Explanatory diagram showing the flow of operation information Explanatory drawing which shows the adjustment means in each of a yaw adjustment part, a pitch adjustment part, and a roll adjustment part Explanatory diagram showing the outline of the open agent platform Explanatory drawing showing the outline of the agent stand device imitating the head Explanatory drawing which shows another Example of the agent stand apparatus imitating a head Explanatory drawing which shows another Example of the agent stand apparatus using a monitor Explanatory drawing illustrating the flow of processing in the apparatus of the embodiment Explanatory diagram illustrating information specific to partner agent and status information of partner region Explanatory drawing which shows another Example of the apparatus which has a monitor as an agent

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Agent 11 Telephone main body 12 Handset 13 Speaker 14 Microphone 15 Head imitation 16 Pipe line 17, 18 Cable 19 Display apparatus 21 Mounting part 22 Angle adjustment part 23 Base part 31 Yaw adjustment part 32 Pitch adjustment part 33 Roll adjustment part 40 Operation information splitter 50 Coupler 51 Microphone 52 Amplifier with variable device 53 Speaker terminal 54 Speaker with terminal 55 Speaker 61 Speaker 62 Amplifier with variable device 63 Microphone terminal 64 Microphone with terminal 65 Microphone

Claims (12)

A device for indirectly transmitting an action of an object by reproducing the action of the object as an action of an agent.
An agent stand for placing a desired agent appropriately selected;
With respect to the motion of the target object, motion information receiving means for receiving motion information including information indicating the amount of variation in each of three independent directions of yaw angle, pitch angle, and roll angle ;
The received motion information is divided into components in each direction, and the motion information component dividing means for obtaining each variation information for each direction,
An operation indirect transmission device comprising: an operation reproduction unit that drives and varies an agent table by an amount corresponding to variation amount information in each direction. Three independent directions
The azimuth rotating in a substantially horizontal direction, the front-rear rotation azimuth rotating perpendicular to the substantially horizontal rotation azimuth, and the left-right rotation azimuth rotating vertically in both the substantially horizontal rotation azimuth and the front-rear rotation azimuth. Motion indirect transmission device. The motion reproduction means
The motion indirect transmission device according to claim 1, further comprising an angle adjustment unit that supports the agent table and adjusts a support angle thereof. The angle adjuster
A yaw adjustment unit having a horizontal rotation shaft that rotates in a substantially horizontal rotation direction, and a drive member that adjusts the rotation angle (yaw angle);
A pitch adjusting unit having a front-rear rotating shaft that rotates in the front-rear rotating direction and a drive member that adjusts the rotation angle (pitch angle);
The motion indirect transmission device according to claim 3, further comprising: a roll adjusting unit having a left and right rotation shaft that rotates in a left and right rotation direction and a drive member that adjusts the rotation angle (roll angle). The target object is the person who is talking on the phone,
The motion indirect transmission device according to claim 1, wherein the motion of the object is a head movement that occurs when a person and a person have a natural conversation. The handset is placed on the coupler, the handset speaker faces the coupler microphone, and the handset microphone faces the coupler speaker,
6. The operation indirect transmission device according to claim 1, wherein the coupler microphone is connected to a speaker provided on the agent table , and the coupler speaker is connected to a microphone provided on the agent table . An agent imitates the human head shape,
The motion indirect transmission device according to claim 6, wherein the head imitation is provided with a hollow pipe line from a bottom portion facing the speaker of the agent table to a mouth along a portion corresponding to the airway. The motion indirect transmission device according to claim 1, wherein the agent is a display device capable of dynamically changing display contents. The situation of the partner area output from the local situation search means based on the information specific to the partner agent output from the partner information detection means for detecting information on the target object and the information of the partner area output from the partner information detection means The motion indirect transmission device according to claim 8, wherein the display device displays an agent image formed by combining the information with the agent combining means. The operation indirect according to claim 8 or 9, wherein among the operation information including information indicating each amount of variation in three independent directions, the amount of variation in at least one direction is expressed as display content displayed on a display device. Transmission device. The operation indirect transmission device according to any one of claims 1 to 10, wherein a communication means that enables simultaneous communication with each other is provided between a large number of operation indirect transmission devices, and is used in a conference system with a large number of people. It is a method of indirectly transmitting the motion of the object as an agent motion by targeting the object performing the motion,
In a configuration including an agent table for placing a desired agent selected as appropriate,
The motion information receiving means receives motion information including information indicating each variation amount in three independent directions of the yaw angle, the pitch angle, and the roll angle with respect to the motion of the target object.
By the motion information component dividing means, the received motion information is divided into components in each direction, and each variation information for each direction is obtained,
An indirect motion transmission method characterized in that the motion reproduction means drives and varies the agent table by an amount corresponding to the variation amount information in each direction.

Images