JP2020087003A - Character display method - Google Patents

Abstract

To provide character display means which uses a planar screen and which is suitable for causing sensation of being gazed by a character to a person who observes the character.SOLUTION: There is provided a method for displaying a character Ch in a planar screen Sc toward a target person Tg existing on a front side of the screen Sc. The character Ch is displayed so that a visual line of the character Ch continuously comes out of a front direction of the screen Sc by a first trigger including an actuation condition that a drift angle p of polar coordinates of the target person using a display position of the character Ch as an origin is within an angle range for generating a Mona Lisa effect. The visual line of the character Ch is smoothly changed by animation so that the visual line of the same character Ch turns to the front direction of the screen Sc by a second trigger including an actuation condition that display of the character Ch at least by the first trigger continues for predetermined time.SELECTED DRAWING: Figure 4

Description

The present invention relates to a display method.

Patent Document 1 discloses a method of displaying a character on a flat screen. With this display method, the character's line of sight is always directed to the front, even if the character is facing sideways. As a result, it is possible to display a screen that speaks to the user even if the character is facing sideways (paragraph [0033]).

JP 2012-050791 A

ACM Transactions on Interactive Intelligent Systems, Vol. 1, No. 2, Article 11, Pub. date: January 2012., Taming Mona Lisa: Communicating Gaze Faithfully in 2D and 3D Facial Projections, SAMER AL MOUBAYED, JENS EDLUND, and JONAS BESKOW , KTH Royal Institute of Technology Kana Misawa, 2 others, “LiveMask: Evaluation of communication in telepresence system using 3D face shape display”, [online], March 15, 2012, IPSJ Interaction 2012 (IPSJ Interaction 2012), [Heisei Era] Search on November 8, 2018], Internet <URL: http://www.interaction-ipsj.org/archives/paper2012/data/Interaction2012/oral/data/pdf/12INT006.pdf>

In the above method, the computer always directs the eyes of the character on the screen to the front. Therefore, the user can get a feeling that the character speaks to the user. However, this method does not allow the user to satisfactorily obtain the feeling that the character on the screen is gazing at himself. Or the user is not happy with the feeling that he or she is in eye contact with the on-screen character. This tendency is remarkable as compared with the case where the target person receives a line of sight from another real person.

The above-mentioned phenomenon is generalized as a psychological natural law beyond the scope of subjectivity and assumptions of the target person. An object of the present invention is to provide a character display unit using a flat screen, which is suitable for inducing a person who observes the character to feel the character's gaze.

[1] A method of displaying a character in the screen toward a target person existing on the front side of a flat screen,
By the first trigger including as an operating condition that the polar coordinate declination p of the subject whose origin is the display position of the character is in the angular range that produces the monaural effect, the character's line of sight is continuously changed from the front of the screen. Display the character so that it will come off,
The second trigger including at least the display of the character by the first trigger continued for a predetermined time as an operating condition smoothly changes the line of sight of the character by animation so that the line of sight of the same character faces the front direction of the screen. ,
Method.
[2] The first trigger further includes, as a condition, that the distance d between the display position of the character and the target person has changed from being larger than a predetermined value R1 to being smaller than a predetermined value R1.
The method according to [1].
[3] The second trigger includes, as a condition, at least the distance d after the lapse of the predetermined time is smaller than a predetermined value R2 smaller than the predetermined value R1.
The method described in [2].
[4] A lapse of time from the actuation of the first trigger until the distance d becomes smaller than a predetermined value R2 smaller than a predetermined value R1 is regarded as the display of the character by the first trigger continued for a predetermined time,
The method described in [3].
[5] After the operation of the second trigger, even if the polar angle p of the subject changes, the character's line of sight and the direction of the character's face continue to be fixed in the direction of the front of the screen.
The method described in [3].
[6] After the operation of the second trigger, while fixing the line of sight of the character in the direction of the front of the screen, the direction of the character's face is smoothly animated by following the change in the polar coordinate declination p of the subject. move,
The method described in [3].
[7] When the character is displayed by the first trigger, the character's line of sight is smoothly moved by an animation so that the line of sight of the character, which was facing the front of the screen before that, deviates from the front of the screen.
The method according to [1].
[8] While the character is displayed by the first trigger so that the character's line of sight deviates from the direction of the front of the screen, the character's line of sight continues to be animated.
The method according to [7].
[9] When the character's line of sight continues to move, the character's line of sight passes in the front direction of the screen, but does not stare in the front direction of the screen.
The method according to [8].
[10] The line of sight is changed by the second trigger even if the polar coordinate declination p of the subject is within the angular range that produces the monaural effect after the lapse of the predetermined time.
The method according to [1].
[11] Even if the subject is present in front of the display position after the lapse of the predetermined time, the line of sight is changed by the second trigger,
The method according to [1].
[12] Even if the line of sight of the character changes due to the second trigger, the direction of the character's face does not change,
The method according to [1].
[13] The polar coordinates are circular coordinates when a plane parallel to the display direction of the screen is viewed in plan.
The method according to [1].
[14] The screen is configured by attaching a half mirror to a display area of a self-luminous display,
The display area of the display is observed from the subject through the half mirror,
The method according to [1].
[15] A controller that receives a detection signal from a sensor and sends a video signal to a display,
The display has a flat screen, and receives the video signal to display a character at a predetermined display position in the screen based on the video signal.
The sensor sends, as a detection signal, the presence of a subject on the front side of the screen to the controller, wherein the sensor is a polar coordinate of the subject whose origin is the display position of the character. In the case where the angle p is in the angular range that produces the Mona Lisa effect, the presence of the subject can be detected,
The controller is
When it receives the detection signal, it sends a video signal to continuously display the character in the line of sight from the front of the screen at the display position, and the character in the line of sight continues to be displayed for a predetermined time. After that, the same character sends a video signal for displaying an animation in which the line of sight moves smoothly toward the front of the screen at the display position,
controller.
[16] Even if the subject is present in front of the display position, the subject is sent to the controller as the detection signal.
The controller sends a video signal that displays the character that is out of the line of sight when the detection signal is received even when the subject is present in front of the display position,
The controller according to [15].
[17] A program for operating a computer as the controller according to [15].
[18] Transmitting the program according to [17] to the computer via a network and installing the program on the computer.
[19] A server storing the program according to [17], which transmits the program to the computer via a network.

According to the present invention, it is possible to provide a character display means using a flat screen, which is suitable for causing a person who observes the character to feel the gaze of the character.

Top view of observer and character 1. Plan view of the observer and character 2. Flow chart of display method. Character image 1. Character image 2. Transition diagram of the position of the target person and the line of sight of the character. Transition diagram of the character's line of sight. Transition diagram of the position of the target person. Movement of the subject Figure 1. Movement of the subject Figure 2. Connection diagram of the controller and other devices. Observation view of the screen. Top view of the target person and the character. Logical configuration diagram of the controller.

<Mona Lisa effect>

This embodiment is used under the condition where the mona Lisa effect occurs. The general concept of the Mona Lisa effect is described in [1]. FIG. 1 shows the positional relationship when the character Ch and the observers Ob1 and Ob2 are viewed in a plan view. The character Ch is originally displayed flat on the screen Sc. The representation of the character Ch in the figure is virtual. The character Ch is looking toward the front of the screen Sc. The observer Ob1 is ahead of his line of sight. The observer Ob2 is also on the front side of the screen Sc. However, the observer Ob2 is at a position displaced to the left when viewed from the character Ch.

In FIG. 1, the observer Ob1 has a feeling that he or she is gazing at the character Ch. This is naturally understood because the observer Ob1 is ahead of the line of sight of the character Ch. Even if the character Ch is a real person, the observer Ob1 feels that the character Ch is gazing at the character Ch.

On the other hand, in FIG. 1, the observer Ob2 is not in the line of sight of the character Ch. However, the observer Ob2 also feels as if he or she is gazing at the character Ch. This is the so-called Mona Lisa effect. If the character Ch is a real person, the observer Ob2 cannot have a feeling that the character Ch is gazing at the character Ch. This is because the line of sight is completely deviated from the direction toward the observer Ob2. That is, the mona Lisa effect is a phenomenon peculiar to a character displayed on a flat screen.

In FIG. 1, it is not known whether the observer Ob1 is potentially having the mona Lisa effect. In the present specification, the embodiment will be described on the assumption that the observer Ob1 is in a range where the monaural effect is not generated. Further, the embodiment will be described on the assumption that the observer Ob2 is in a range where the monaural effect occurs. All of these are for convenience of technical explanation. In some aspects, the description herein may not accurately represent a psychological natural law.

FIG. 2 also shows the positional relationship when the character Ch and the observers Ob1 and Ob2 are viewed in a plan view. The character Ch is looking toward the left side of the character Ch. For this reason, the observer Ob1 feels that the character Ch is looking away from himself. Alternatively, the observer Ob1 has a feeling that the observer Ob2 is gazing at the character Ch. From the viewpoint of the observer Ob1, it seems that the observer Ob2 is ahead of the line of sight of the character Ch, which is naturally understood.

On the other hand, in FIG. 2, the observer Ob2 also feels as if the character Ch is looking away from himself. From the viewpoint of the observer Ob2, it is felt that the line of sight of the character Ch is facing to a position further to the left than himself as seen from the character Ch. This is one aspect of the Mona Lisa effect.

<Character display method>

The flowchart of FIG. 3 shows the character display method of this embodiment. 4 and 5 are plan views of the target person Tg and the character Ch. In this display method, the character Ch is displayed on the flat screen Sc. At this time, the character Ch is displayed toward the target person Tg existing on the front side of the screen Sc. Hereinafter, description will be given with reference to these figures in a timely manner. The subject Tg may or may not be observing the character Ch. Therefore, in this embodiment, a person who is a potential observer is expressed as a target person Tg. One aspect of the method of the present embodiment is to perform automated display under the assumption that the subject Tg is observing the character Ch.

As shown in FIGS. 4 and 5, the character Ch is displayed in the screen Sc while the eyes of the character Ch are directed toward the target person Tg. An illustration of a bust of a girl is shown as an example of the character Ch.

4 and 5, by changing the depiction of the pupil of the character Ch, the direction of the line of sight can be displayed in various ways. It is considered that the depiction of the eyes requires at least depiction of the white eye and iris. Medically, the pupil is the pupil surrounded by the iris. However, since the present embodiment discusses how to draw, the iris surrounded by white eyes is called a "pupil".

As shown in FIG. 3, in step S31, it is determined whether the operating condition of the first trigger is satisfied. When the first trigger is activated, the line of sight of the character is removed from the front in step S32 to display the character. The operation condition of the first trigger is that at least the subject Tg has a mona Lisa effect. The first trigger may have other additional operating conditions.

As shown in FIG. 4, whether or not the monaural effect is produced can be considered in a horizontal coordinate system (circular coordinate system) having the display position of the character Ch as the origin and extending in the horizontal direction. The polar coordinate system is a polar coordinate system centered on the display position of the character Ch when viewed in a plane parallel to the display direction of the screen Sc.

In FIG. 4, the center of the virtual existence position of the character Ch set behind the screen Sc is the origin of the polar coordinate system. The origin of the polar coordinate system may be the midline of the character Ch displayed on the screen Sc.

As shown in FIG. 4, the direction directly in front of the screen Sc is 0 degree. The direction of the subject Tg is defined as the argument p. If p=0, the target person Tg is located in front of the display position of the character Ch. In the figure, the subject Tg is located on the right side of the origin. The target person Tg may be located on the left side of the origin. The left side in the figure can be considered as an area of p<0. However, the positive/negative of p is not taken into consideration in this specification. p is considered as an absolute value.

Further, as shown in FIG. 4, the distance between the target person Tg and the origin is d. The distance d corresponds to the radius vector of the polar coordinate system. The positional relationship among the character Ch, the screen Sc, and the subject Tg can be specified by the argument p and the distance d. If the declination angle p is within a predetermined angle range, the monaural effect for the subject Tg is produced.

As shown in FIG. 4, the predetermined angle range is Mn1<p<Mn2. The angle Mn1 is greater than 0 degrees. The angle Mn1 may be experimentally obtained. In this embodiment, the angle Mn1 is 10 degrees (Non-Patent Document 2). When p<Mn1, there is no influence of the mona Lisa effect on whether or not the line of sight from the character Ch is felt. The angle Mn2 can be expanded up to 90 degrees.

When the condition is satisfied and the first trigger is activated, the character Ch is displayed such that the line of sight of the character Ch deviates from the front direction of the screen Sc as shown in FIG. The state where the character Ch's line of sight is off is continuous.

In FIG. 4, the direction of the line of sight of the character Ch can be changed by drawing the pupil. Since the character Ch is an illustration, the direction of the line of sight of the character Ch depends on the subjectivity of the observer. Therefore, for example, as shown in FIG. 2, the observer Ob1 in the front may observe the character Ch and evaluate the direction of the line of sight of the character drawn in advance. The evaluation may be performed statistically.

In FIG. 4, the direction of the character's line of sight is represented by the argument g. When g=0, the line of sight is parallel to the front direction of the screen Sc. The fact that the line of sight of the character Ch has deviated from the front direction of the screen Sc (Averting) is represented by g>Av. The angle Av is greater than zero. The angle Av may be experimentally acquired. If the argument g is greater than the angle Av, it can be said that the line of sight of the character Ch has deviated. In the figure, the line-of-sight declination g is smaller than the subject's Tg declination p. The deflection angle g may be larger than the deflection angle p. The deflection angle g may match the deflection angle p.

Next, as shown in FIG. 3, it is determined in step S33 whether or not the operating condition of the second trigger is satisfied. Monitor the passage of time for judgment. When the second trigger is activated, the line of sight of the character moves to the front in step S34. The operation condition of the second trigger is that the character display (step S32) by the first trigger continues for a predetermined time. Here, the predetermined time is preferably several seconds at the longest. In setting the predetermined time, the time required for natural communication between living human beings is considered. The second trigger may have other additional operating conditions. As will be described later, the determination of the second trigger does not have to be accompanied by monitoring the passage of time.

When the condition is satisfied and the second trigger is activated, the line of sight of the character Ch moves as shown in FIG. The character Ch shown in FIG. 5 is the same character as the character Ch shown in FIG. Remove the line of sight and change the line of sight to the front direction with the same character.

As a result of the movement of the line of sight, the character Ch is displayed so that the line of sight of the character Ch faces the front of the screen Sc as shown in FIG. When moving the line of sight, move smoothly with animation. In the change from FIG. 4 to FIG. 5, the line of sight makes a transition from the diagonal left direction for the character Ch to the front direction.

As shown in FIG. 4, the change in the line of sight of the character Ch due to the second trigger does not follow the change in the direction of the face of the character Ch. In one aspect, even if the line of sight of the character Ch changes due to the second trigger, the face direction of the character Ch does not change. In the figure, the face of the character Ch remains facing the front direction of the screen Sc while the line of sight changes. In another aspect, the face of the character Ch may face a direction other than the front.

In FIG. 5, the state in which the line of sight of the character Ch faces the front is continuous. The character Ch gazes at the front. In the figure, Ob2 is located in the range (Mn1<p<Mn2) where the mona Lisa effect occurs. Due to the mona Lisa effect, the subject Tg feels that the character Ch is looking at himself. When the target person Tg is gazing at the character Ch, the target person Tg strongly feels that the character Ch is gazing himself. Furthermore, the target person Tg feels that his eyes match the character Ch.

<Characteristics of characters used>

4 and 5, the character Ch may blink. In FIGS. 4 and 5, the character Ch may represent facial expressions and emotions. For example, a facial expression may not be displayed when the first trigger is activated (FIG. 4), and a smile may be made while the line of sight is directed to the front when the second trigger is activated (FIG. 5). When the display method is used to provide entertainment, it is also effective to present a facial expression that surprises or gazes at the observer.

In Figures 4 and 5, the iris may be represented by a darker color than the white eye. An expression method of adding a highlight to an iris filled with a dark color is included in one aspect of this embodiment. White eyes are not limited to pure white. The white eye can be thought of as the area around the iris in the eye. An iris may be made brighter than a white eye in the expression method of reversing the light and shade. There is a shadow tone as an expression method for reversing the shading.

4 and 5, the character Ch has a face having at least pupils. The character Ch may be a face image, a bust image, an upper body image, or a full body image. The character Ch may be a sitting image or a standing image. If the face of the character Ch faces the front side, the pose is not limited. In the figure, the character Ch faces the front. As an expression, as long as the line of sight can be directed to the front side of the screen Sc, the orientation of the face is not limited.

4 and 5, the character Ch is the naked eye. The character Ch may wear glasses or goggles. The type of glasses is not limited. However, it is not appropriate to wear dark sunglasses because the direction of the line of sight cannot be seen from the outside. Thick goggles are also not suitable.

In FIGS. 4 and 5, one of the eyes of the character Ch may be hidden by the hair or the eyepatch. It is also effective to make the character Ch look into the front from behind.

In FIGS. 4 and 5, when the orientation of the pupil is changed, the orientation of the pupil may be different in both eyes. It is possible to express according to the expression technique. Such an expression method is considered to be useful when the face does not face the front. It suffices that the target person Tg can specify the direction of the line of sight of the character Ch without any discomfort. Apart from this, expressing that the character Ch has a squint is also included in one aspect of the present embodiment.

In FIGS. 4 and 5, the character Ch may be an actual image of a person. The character Ch may be a more deformed illustration. The character Ch may be made by 3DCG modeling. The character Ch may use humans and animals other than humans as motifs. The character Ch may be a robot or a fantasy creature. The character Ch is preferably anthropomorphic to the extent that the direction of the line of sight of the character can be seen by the subject. For example, since insects look around with compound eyes, they do not have a line of sight in the first place. If the character Ch has an insect illustration, it is effective to replace the eyes in the illustration with comical eyes.

<Consideration of height direction>

The polar coordinate system centered on the display position of the character Ch shown in FIGS. 4 and 5 is circular coordinates when the plane parallel to the display direction of the screen Sc is viewed in plan view. In one aspect, the height direction can also be considered for the Mona Lisa effect. When the character Ch removes the line of sight based on the first trigger, the line of sight may be removed in the height direction. The direction of the line of sight may be defined in a cylindrical coordinate system or a spherical coordinate system. Furthermore, when the character Ch directs his or her line of sight to the front based on the second trigger, it is preferable that the line of sight be directed horizontally. For example, even if the subject is a short child, the character Ch does not have to look downward when the second trigger is activated.

<Gaze before actuation of the first trigger>

In one aspect, the state before actuation of the first trigger may be considered. FIG. 6 shows the transition between the line of sight of the character Ch and the position of the observer. First, the change in the line of sight of the character Ch will be described. In the original state (Original) in the upper part of the figure, the line of sight of the character Ch faces the front of the screen Sc.

Next, as shown in the middle part of FIG. 6, a state in which the line of sight of the character Ch is deviated from the front of the screen Sc by the first trigger is displayed. At this time, the line of sight of the character Ch is smoothly moved by animation. In one aspect, during this time, the face of the character Ch is still facing the front direction of the screen Sc. In another aspect, the face of the character Ch may face a direction other than the front.

In the middle part of FIG. 6, the line of sight transitions from the front direction, which is the original state for the character Ch, to the diagonal left direction. Even when the subject Tg is gazing at the character Ch, the subject Tg feels that the character Ch is not gazing himself. However, the subject Tg recognizes that the character Ch is looking at something ahead of the character. For the subject Tg, the direction is on the right side of the subject Tg.

Further, as shown in the lower part of FIG. 6, the state in which the line of sight of the character Ch faces the front of the screen Sc by the second trigger is displayed. This operation is as described above. Move the character's line of sight smoothly with animation. In one aspect, the face of the character Ch remains facing the front of the screen Sc during this period. In another aspect, the face of the character Ch may face a direction other than the front.

In the lower part of FIG. 6, the line of sight transitions from the diagonal left direction for the character Ch to the front direction. When the target person Tg is gazing at the character Ch, the target person Tg strongly feels that the character Ch is gazing himself. Furthermore, the target person Tg feels that his eyes match the character Ch.

In another aspect, the sight line of the character Ch may be directed to a position other than the front of the screen Sc before the first trigger is activated. That is, in one aspect, in the starting state, the line of sight of the character Ch may be facing either the front of the screen Sc or the other than the front. However, the first trigger causes the line of sight to be directed to a position other than the front. Furthermore, the line of sight turns to the front by the second trigger.

<Approach of observer>

In FIG. 6, the position of the observer has further changed. In the example shown in the figure, the subject Tg is approaching the screen Sc. Specifically, he is approaching the character Ch. In the original state (Original) shown in the upper part of the figure, the distance d is larger than the predetermined value R1.

As shown in the middle part of FIG. 6, the subject Tg approaches the character Ch. In the figure, the distance d between the display position of the character Ch and the target person Tg is smaller than the predetermined value R1. In one aspect, the distance d smaller than the predetermined value R1 may be an additional operating condition of the first trigger. In another aspect, the additional operating condition of the first trigger may be that the distance d, which was larger than the predetermined value R1 in the original state, becomes smaller than the predetermined value R1. In other words, the first trigger may be activated by detecting that the subject Tg is approaching.

As shown in the lower part of FIG. 6, the target person Tg further approaches the character Ch. In the figure, the distance d between the display position of the character Ch and the target person Tg is smaller than the predetermined value R2 after the lapse of a predetermined display time until the second trigger operation. The predetermined value R2 is smaller than the predetermined value R1.

As shown in the lower part of FIG. 6, in one aspect, the distance d being smaller than the predetermined value R2 may be an additional operating condition of the second trigger. In another aspect, the additional operation condition of the second trigger may be that the distance d, which was larger than the predetermined value R2 in the original state, becomes smaller than the predetermined value R2. In one aspect, after the lapse of a predetermined display time, the second trigger is activated by detecting that the subject Tg is further approaching.

There are various modes for considering the approach of the subject Tg. For example, in FIG. 6, the time during which the distance d transits from R1 to R2 may be set as the predetermined display time that is the operating condition of the second trigger. In this case, monitoring the elapsed time from the actuation of the first trigger to the actuation of the second trigger is included in the monitoring of the distance d. That is, the second trigger operating condition is that the distance d is less than R2 while at least the display of the character Ch after the first trigger operating is continued without using the direct operating condition regarding the elapsed time. Good. In other words, it may be considered that the time has elapsed when the distance d has changed, and the second trigger may be activated. For example, if the distance d becomes smaller than the predetermined value R2 before the predetermined display time elapses, the predetermined display time may elapse before the predetermined display time actually elapses. In another aspect, even if the distance d becomes smaller than the predetermined value R2 before the predetermined display time elapses, the second trigger may not be activated until the predetermined display time actually elapses. In another aspect, it is also conceivable that the distance d is smaller than the predetermined value R2 at the time of actuation of the first trigger. In this case, the lapse of a predetermined display time may be set as the operation condition of the second trigger regardless of the distance d. In another aspect, the approach of the observer need not be considered at all.

<Expression when removing the line of sight with the first trigger>

In one aspect, the state after actuation of the first trigger may be considered. FIG. 7 shows the transition of the line of sight of the character Ch. As shown in the upper part of FIG. 7, the character Ch that is off the line of sight is displayed by the first trigger. It takes a certain amount of time until the second trigger is activated. The line of sight of the character Ch may not be fixed during this period (Interval).

In the middle part of FIG. 7, the line of sight changes from the diagonal left direction for the character Ch to the diagonal right front direction. Even after displaying the character Ch whose eyes are off line, the eyes of the character Ch may be kept moving. At this time, the line of sight of the character Ch is smoothly moved by animation. After a predetermined time has passed, as shown in the lower part of FIG. 7, the line of sight of the character Ch is turned to the front by the second trigger. This operation is as described above.

When continuing to move the line of sight of the character Ch in the middle part of FIG. 7, the line of sight of the character Ch may pass in the front direction of the screen Sc. However, the line of sight should not stare in the direction of the front of the screen. The sight line of the character Ch may be set so as not to pass in the direction in front of the screen Sc.

By controlling the line of sight as described above, it is possible to maintain the state in which the monaural effect is not generated until the operation of the second trigger.

<The position where the Mona Lisa effect occurs and the second trigger>

4 to 5, the target person Tg remains in the same place. During this period, the Mona Lisa effect always affects the subject Tg. By removing the line of sight from the direction in which the target person Tg is present, a feeling that the character Ch is gazing at the target person Tg is generated. This is due to the Mona Lisa effect.

The mode shown in FIG. 5 is characterized in that the line of sight is changed by the second trigger even if the polar coordinate declination p of the subject Tg is within the angular range that produces the monaural effect after a predetermined display time. .. The subject Tg being in a position where the monaural effect is occurring may be an additional operating condition of the second trigger. The subject Tg remaining in the same place as when the first trigger is activated may be an additional activation condition for the second trigger.

As shown in FIG. 8, the position of the subject Tg may change in another aspect. The subject Tg may move to a range that is not affected by the Mona Lisa effect while the character Ch is out of sight. As an example, in the middle of the figure, the target person Tg is moving to the front of the character Ch.

As shown in the lower part of FIG. 8, after a predetermined display time has elapsed, a state in which the line of sight of the character Ch faces the front of the screen Sc by the second trigger is displayed. Even if the target person Tg exists in front of the display position of the character Ch, the line of sight is changed by the second trigger. The operation condition of the second trigger may not include the subject Tg being in the position where the monaural effect is occurring. The fact that the subject Tg is not in the position where the monaural effect is occurring may be an additional operating condition of the second trigger.

<Display for migrants>

9 and 10 show the positions P0 to P5 of the target person moving across the front of the screen Sc. For convenience of designing the display method, the origin of the polar coordinate system is set to the midline of the character Ch displayed on the screen Sc. The argument p changes as the position changes. The declination p in the figure represents the declination at the position P4. It is assumed that the position P3 is in front of the character Ch.

In FIGS. 9 and 10, the line of sight is removed by the first trigger at the position P0, and the line of sight is changed by the second trigger at the position P1. Furthermore, it is assumed that the target person passes in front of the character Ch in the order of positions P2, P3, P4, and P5.

In FIG. 9, an image of the character Ch is displayed on the screen Sc with the direction of sight and the direction of the face fixed in the front direction. This state is after activation of the second trigger. If the mover observes the character Ch, the mover feels that he or she is gazing at the character Ch. This sensation occurs at any position of P1 to P5 by the moving person. That is, it is a sensation that occurs regardless of the standing position of the mover.

In FIG. 9, the face of the character Ch feels as if it is not moving when viewed from the subject. Moreover, the face of the character Ch has not changed its direction in fact. For this reason, it is difficult for the subject to have the feeling that the character Ch is tracking the subject with his or her line of sight. The Mona Lisa effect creates the feeling of being watched. However, even the feeling of being tracked by the line of sight does not effectively occur.

The character Ch is displayed on the screen Sc so that the face direction of the character Ch changes as shown in FIG. The direction of the line of sight of the character Ch is fixed to the front, and the face of the character is followed by the moving person. The direction of the face of the character Ch is linked to the positions P1 to P5 of the mover. The direction of the face of the character Ch is smoothly changed by animation so as to follow the change in the polar angle p of the target person.

In FIG. 10, the moving person can be made to feel as if the character Ch is tracking the moving person with his or her line of sight. Alternatively, it is possible to make the moving person feel such a feeling stronger than the method shown in FIG.

<Connection between controller and other devices>

Hereinafter, a device for carrying out the display method of the present embodiment will be mainly described. The device is an example and is not the only mechanical means for carrying out the display method. FIG. 11 shows the controller 40 and each device connected to them. A display device 41 is connected to the controller 40. This connection may be via a network or via suitable video signal wiring. This connection may be wired or wireless.

As shown in FIG. 11, the display device 41 includes a display 42 and a half mirror 43. The display 42 is, for example, a liquid crystal display or an organic EL display. The display 42 has no irregularities and is flat. Here, the minute irregularities that cannot be visually recognized are not considered. Flatness means that there is no large unevenness to the extent that it affects the basic shaping of the displayed image. The half mirror 43 is installed on the display area 44 of the display 42. The half mirror 43 may be attached to the display area 44 of the display 42.

The display area 44 shown in FIG. 11 presents an image to the subjects 45a and 45b. The image is displayed through the half mirror 43. For convenience, the front surface of the half mirror 43, that is, the left surface in the figure is the screen Sc. The half mirror 43 is one size larger than the display area 44.

In FIG. 11, a camera 39 corresponding to a sensor is connected to the controller 40. This connection may be via a network or via suitable video signal wiring. This connection may be wired or wireless. The camera 39 may be a camera that captures an image with a CMOS (Complementary metal-oxide-semiconductor) sensor. The camera 39 may be replaced with another optical sensor such as an infrared sensor.

In FIG. 11, the controller 40 receives a detection signal from the camera 39. Further, the controller 40 sends a video signal to the display 42. The display 42 receives a video signal from the controller 40. The display 42 displays the character Ch at a predetermined display position in the display area 44 based on the video signal.

In FIG. 11, the camera 39 photographs at least one of the subjects 45a and 45b. The camera 39 sends information indicating that at least one of the subjects 45a and 45b exists on the front side of the screen Sc to the controller 40 as a detection signal. The detection signal may be a video signal generated by the camera 39.

In FIG. 11, the camera 39 may continuously shoot images, continuously generate video signals, and continuously send video signals to the controller 40 regardless of whether the subject is within the shooting range. The camera 39 may shoot, generate video signals, and send video signals when requested by the controller 40.

FIG. 12 shows the screen Sc of the display device 41. The character Ch is displayed in the display area 44. The background around the character Ch displayed in the display area 44 is dark. Therefore, on the screen Sc, the function of the half mirror 43 causes a real background due to the reflection of the half mirror to appear around the character Ch displayed by the light from the display passing through the half mirror. The real background and the character Ch are superposed on the screen Sc.

In FIG. 12, the screen Sc is flat without any unevenness. No attempt has been made to make the face and eyes of the character Ch three-dimensional by providing irregularities in the vicinity of the face of the character Ch displayed on the screen Sc.

Return to FIG. A camera 39 is further connected to the controller 40 as a sensor. The camera 39 is arranged behind the half mirror 43, on the right side in the figure. The camera 39 photographs the subjects 45a and 45b. As shown in FIG. 12, the camera 39 cannot be visually recognized by the subject. This is because the camera 39 is blocked from the field of view by the half mirror 43.

Note that the device configuration for forming the screen Sc shown in FIGS. 11 and 12 is not limited to the above. The screen Sc does not have small irregularities. However, the screen Sc may be uniformly and gently curved. When displaying a character Ch having a face width of about 20 cm, the radius of curvature of the screen Sc is preferably larger than 10 cm, and the screen Sc is preferably a flat surface. The screen Sc may be formed by a projection screen instead of the display device including the half mirror. Such a screen may be a front type screen using reflection of light or a rear type screen using transmission of light. The half mirror 43 may be omitted. That is, the display device 41 may include only the display 42. The display 42 is, for example, a liquid crystal display or an organic EL display.

The operation of the camera will be described with reference to FIG. The figure shows the subjects 45a and 45b and the character Ch in a plan view. The figure further shows a target person 45c, which will be described later. The character Ch is facing the direction Fc of the face toward the target person 45a. The character Ch directs the line-of-sight direction Gz1 or Gz2 toward the front side of the screen Sc. In the figure, the line-of-sight direction Gz1 is a direction removed from the front direction of the screen Sc. In the figure, the line-of-sight direction Gz2 is the front direction of the screen Sc.

In FIG. 13, a camera 39 is a sensor capable of detecting the presence of the subject person when at least one of the subject persons 45a and 45b is in a range where the monaural effect is produced. The range in which the monaural effect is generated means that the polar angle p of at least one of the subjects 45a and 45b, whose origin is the display position of the character Ch, is located in the angular range in which the monaural effect is generated. The angle range is as described above with reference to FIGS. 4 and 5. The argument p in the figure represents the argument of polar coordinates of the target person 45a.

<System and first trigger>

FIG. 14 describes the structure of the system that constitutes the controller 40. This system may be composed of a computer and a program for operating the computer as a controller. The program is a kind of video generation software. The program does not need to be composed of one program. You may transmit some or all of the program to a computer from the server which memorize|stores some or all of the program via a network. For example, the character may be further installed with a program on a computer. The program may be stored in a non-volatile memory or a disk medium. The computer does not have to be composed of a single computer. A part of the system that operates as a controller may be arranged on the cloud network.

The video generation software of the controller 40 includes components 51 to 54. Components 51-54 are each software components in one aspect. Although hardware such as a signal converter suitable for this may be required for exchanging signals between the controller 40 and the outside, it is omitted in the drawing.

The processing relating to the first trigger in FIG. 14 is as follows. The controller 40 converts the detection signal received from the camera 39. It is the component 51 that converts the detection signal. The component 51 generates information for identifying the position of the subject based on the converted detection signal. The component 51 stores the position of the subject. The subject may not be detected. More than one subject may be detected at the same time. The component 51 may have a motion capture function. According to the example of FIG. 13, the component 51 generates information that specifies the argument p of the target person 45a.

In FIG. 14, the controller 40 generates background information. It is the component 52 that generates the background information. The component 52 stores the virtual background object and its position in advance. The component 52 may acquire the virtual background object and its position in advance via a network (not shown). According to the example of FIG. 13, virtual background objects B01 to B09 and their positions are stored in advance. The position of the virtual background object provides, so to speak, a typical value of the deviation angle between the direction of the line of sight of the character and the direction of the face. The position of the virtual background object B05 provides the direction of the front of the screen Sc, so to speak.

In FIG. 14, the controller 40 selects a gaze target. It is the component 53 that selects the gaze target. The component 53 refers to the position of the target person of the component 51. The component 53 refers to the position of the virtual background object of the component 52. According to the example of FIG. 13, the component 53 refers to the positions of the virtual background objects B01 to B09 and the position of the target person 45a. The component 53 may further refer to the location of the target person 45b.

In FIG. 14, the component 53 determines whether the position of the target person 45a produces the monaural effect. If the position produces the monaural effect, then the component 53 selects a position of the virtual background object that is not in the front direction as a gaze target. The component 53 stores the selected gaze target. According to the example of FIG. 13, the component 53 selects a virtual background object other than the virtual background object B05 as a gaze target. In the figure, the virtual background object B07 is selected as a gaze target. The component 53 may select the position of the target person 45b as a gaze target instead of the virtual background object B07. The component 53 may also store a record of the actuation of the first trigger along with a record of the time of day.

In FIG. 14, the controller 40 selects a face-oriented target. It is the component 53 that selects the face target. According to the example of FIG. 13, the position of the target person 45a may be the target for the face, and the virtual background objects B01 to B09 may be the target for the face. The face target does not have to match the gaze target. The component 53 stores the selected face target.

In FIG. 14, the controller 40 sets the direction of the line of sight. It is the component 54 that sets the direction of the line of sight. The component 54 refers to the gaze target selected by the component 53. The component 54 stores the set line-of-sight direction. According to the example of FIG. 13, the line-of-sight direction Gz1 is stored.

In FIG. 14, the controller 40 sets the direction of the face. It is the component 54 that sets the orientation of the face. The component 54 refers to the face-oriented target selected by the component 53. The component 54 stores the set face direction. According to the example of FIG. 13, the face direction Fc is stored.

In FIG. 14, the component 55 refers to the direction of the line of sight and the direction of the face of the component 54. The component 55 contains the CG model of the character. The component 55 may previously acquire the CG model of the character via a network (not shown). The component 55 refers to the character's CG model. The CG model of the character can set the direction of the line of sight and the direction of the face at least by targeting the virtual background object. Further, it is possible to display an animation in which the direction of the line of sight and the direction of the face change by targeting between these virtual background objects.

In FIG. 14, the component 55 draws based on the direction of the line of sight, the direction of the face, and the CG model of the character. A video signal is generated based on the drawing. According to the example of FIG. 13, the video signal continuously displays the character Ch at a predetermined display position on the screen Sc. The video signal includes an animation in which the character Ch moves the line of sight smoothly and removes the line of sight from the front direction of the screen Sc. The character Ch changes the line of sight in the direction Gz1. Note that the line of sight is not accurately drawn in FIG. This video signal includes that the character Ch has his/her face directed in the direction Fc.

In FIG. 14, the controller 40 sends a video signal to the display device 41. It is the component 55 that sends the video signal. The display device 41 animation-displays the character whose line of sight moves smoothly, and then continues to display the character whose line of sight is off for a predetermined time. After that, the second trigger is activated as follows.

<System and second trigger>

The processing relating to the second trigger in FIG. 14 is as follows. The component 51 converts the detection signal received from the camera 39 again. The component 51 generates information for identifying the position of the subject based on the converted detection signal. The component 51 stores the position of the subject. The subject may not be detected. More than one subject may be detected at the same time.

In FIG. 14, the component 53 refers to the position of the target person of the component 51. If the target person of the component 51 cannot refer to it, the subsequent operation may not be performed. For example, the subject having caused the result of actuating the first trigger moves to a position outside the range detectable by the camera.

In FIG. 14, the component 53 further refers to the activation time of the first trigger. The component 53 determines whether or not a predetermined display time has elapsed since the activation of the first trigger. If the predetermined display time has elapsed from the operation of the first trigger, the component 53 next selects the gaze target again. The component 53 refers to the virtual background object of the component 52. According to the example of FIG. 13, the component 53 refers to the virtual background object B05 that represents the front direction of the screen Sc. According to the example of FIG. 13, the virtual background object B05 is selected as a gaze target. The component 53 stores the selected virtual background object B05.

In FIG. 14, the component 54 sets the direction of the line of sight. It is the component 54 that sets the direction of the line of sight. The component 54 refers to the selected virtual background object B05. The component 54 stores the set line-of-sight direction. According to the example of FIG. 13, the line-of-sight direction Gz2 is stored.

In FIG. 14, the component 55 refers to the direction of the line of sight of the component 54. The component 55 refers to the CG model of the same character displayed by the first trigger. The component 55 draws based on the direction of the line of sight and the CG model of the character. A video signal is generated based on the drawing. According to the example of FIG. 13, this video signal is such that the line of sight of the character Ch, which is the same as the character Ch displayed by the first trigger, faces the front of the screen Sc. At this time, the video signal includes an animation in which the character Ch smoothly moves his line of sight. The component 55 sends the video signal to the display device 41.

In FIG. 14, the component 55 sends the video signal to the display device 41. The display device 41 animation-displays a character whose sight line is smoothly moved from the direction Gz1 to the direction Gz2, and then continues to display the character whose sight line is directed to the front. The Mona Lisa effect gives the subject a sense of being gazed at by the character.

<Controller versatility>

The controller 40 shown in FIG. 13 is suitable for executing the above-mentioned display method. The controller 40 and the device connected thereto are not limited to those having only the function of executing the above-described display method. The controller 40 may be capable of causing the character to take an action of removing the line of sight even when the subject is present at a position where the monaural effect is not generated. The presentation of the character Ch for such a subject may be performed using the system shown in FIG.

As shown in FIG. 13, the target person 45c exists in front of the display position of the character Ch. The Mona Lisa effect does not occur in the subject 45c. The camera 39 takes an image of the entire area in front of the screen Sc including the target person 45c. The camera 39 sends a detection signal including the presence of the target person 45c to the controller 40.

13, even if the target person 45c is present in front of the display position of the character Ch, the controller 40, when receiving the detection signal thereof, causes the display device to display a video signal for displaying the character Ch that is out of the line of sight. send. The character Ch whose line of sight is directed in the direction Gz1 is displayed on the screen Sc. Further, after a lapse of a predetermined display time, an animation in which the character Ch directs his or her line of sight in the direction Gz2 is displayed on the screen Sc. The target person 45c feels that the character Ch is gazing at the target person 45c.

<Omission of background information generator and virtual background object>

In the embodiments shown in FIGS. 11 to 14, the selection of the gaze target and the setting of the direction of the line of sight are separated for the convenience of mounting. In another aspect, it is possible to cause the controller 40 to carry out the display method only by setting the direction of the line of sight. The component 52 that generates background information is unnecessary. Further, the virtual background objects B01 to B09 are unnecessary. The component 53 for selecting a gaze target is unnecessary.

First, as shown in FIG. 13, the camera 39 serving as a sensor detects the positions of the subjects 45a and 45b. Next, it is determined whether or not the target person is present at the position where the first trigger is activated within the range of the detected target person. If the target person is at the position to activate the first trigger, the direction other than the front is set as the direction of the line of sight of the character Ch. The direction of the face of the character Ch may be set to other than the front. You don't have to.

Next, it is determined whether or not the target person is present at the position where the second trigger is activated within the range of the detected target person. If the target person is at the position to activate the second trigger, the front direction Gz2 is set as the direction of the line of sight of the character Ch.

Further, as shown in FIG. 10, the orientation of the face of the character Ch may be changed. The face direction sets the direction of the subject who was in the position to activate the second trigger. The component 54 directly references the location of the subject of the component 51. The direction of the face of the character Ch is set to the direction of the subject. This process is continued until another cause occurs in which the face of the character Ch should be set in a direction other than the target person. The other cause is, for example, another subject actuating the second trigger.

The present invention is not limited to the above-mentioned embodiments, but can be modified as appropriate without departing from the spirit of the present invention. For example,

39 camera 40 controller 41 display device 42 display 43 half mirror 44 display area 45a-c target person 51-55 component Av angle B01-B09 virtual background object Ch character d distance Fc face direction g line-of-sight angle Gz1 line-of-sight direction Gz2 Direction of line of sight Mn1 angle Mn2 angle Ob1 observer Ob2 observer p declination P0-P5 position R1 predetermined value R2 predetermined value S31-S34 step Sc screen Tg target person

Claims (19)

A method of displaying a character in the screen toward the target person existing on the front side of the flat screen,
With the first trigger that includes as an operating condition that the polar coordinate declination p of the subject whose origin is the display position of the character is within an angle range that produces the mona Lisa effect, the character's line of sight continuously from the front direction of the screen Display the character so that it will come off,
The second trigger including at least the display of the character by the first trigger for a predetermined time as an operating condition smoothly changes the line of sight of the same character by animation so that the line of sight of the same character faces the front of the screen. ,
Method. The first trigger further includes, as a condition, that the distance d between the display position of the character and the subject changes from a state larger than a predetermined value R1 to a state smaller than a predetermined value R1.
The method of claim 1. The second trigger includes at least the condition that the distance d after the predetermined time has elapsed is smaller than a predetermined value R2 that is smaller than a predetermined value R1.
The method of claim 2. The lapse of time from the actuation of the first trigger until the distance d becomes smaller than the predetermined value R2 smaller than the predetermined value R1 is regarded as the display of the character by the first trigger continued for a predetermined time,
The method of claim 3. After the operation of the second trigger, even if the polar angle p of the subject changes, the gaze of the character and the direction of the face of the character continue to be fixed in the direction of the front of the screen.
The method of claim 3. After the actuation of the second trigger, the character's line of sight is fixed in the direction of the front of the screen, and the direction of the character's face is smoothly moved by animation so as to follow the change in the polar coordinate declination p of the subject.
The method of claim 3. When the character is displayed by the first trigger, the character's line of sight is smoothly moved by animation so that the line of sight of the character that was facing the front of the screen before that deviates from the front of the screen.
The method of claim 1. While the character is displayed by the first trigger so that the character's line of sight deviates from the direction of the front of the screen, the character's line of sight continues to be animated.
The method of claim 7. While continuing to move the character's line of sight, the character's line of sight passes in the direction of the front of the screen, but does not stare in the direction of the front of the screen.
The method of claim 8. Even if the polar angle declination p of the subject is within the angular range that produces the monaural effect after the predetermined time, the line of sight is changed by the second trigger.
The method of claim 1. Even if the subject is present in front of the display position after the predetermined time has elapsed, the line of sight is changed by the second trigger,
The method of claim 1. Even if the sight line of the character changes due to the second trigger, the direction of the character's face does not change,
The method of claim 1. The polar coordinates are circular coordinates when a plane parallel to the display direction of the screen is viewed in plan,
The method of claim 1. The screen is configured by attaching a half mirror to the display area of a self-luminous display,
The display area of the display is observed from the subject through the half mirror,
The method of claim 1. A controller that receives the detection signal from the sensor and sends the video signal to the display,
The display has a flat screen, and receives the video signal and displays a character at a predetermined display position in the screen based on the video signal.
The sensor sends, as a detection signal, the presence of a subject on the front side of the screen to the controller, wherein the sensor is a polar coordinate of the subject whose origin is the display position of the character. In the case where the angle p is in the angular range that produces the Mona Lisa effect, the presence of the subject can be detected,
The controller is
When it receives the detection signal, it sends a video signal to continuously display the character in the line of sight from the front of the screen at the display position, and the character in the line of sight continues to be displayed for a predetermined time. After that, the same character sends a video signal for displaying an animation in which the line of sight moves smoothly toward the front of the screen at the display position,
controller. Even if the target person is present in front of the display position, this is sent to the controller as the detection signal,
The controller sends a video signal that displays a character that is out of the line of sight when receiving the detection signal, even when the subject is present in front of the display position,
The controller according to claim 15. A program for operating a computer as the controller according to claim 15. Transmitting the program according to claim 17 to the computer via a network and installing the program on the computer. A server storing the program according to claim 17, which transmits the program to the computer via a network.

Images