JP7128473B2 - Character display method - Google Patents

Description

Article 30, Paragraph 2 of the Patent Act is applied Published in the Entertainment Computing Symposium 2018 Proceedings on September 6, 2018

The present invention relates to a display method.

Patent Literature 1 discloses a method of displaying a character on a flat screen. With this display method, the character's line of sight is always directed to the front, even if the character is looking sideways. As a result, it becomes possible to display a screen that speaks to the user even when the character is facing sideways (paragraph [0033]).

JP 2012-050791 A

ACM Transactions on Interactive Intelligent Systems, Vol. 1, No. 2, Article 11, Pub. date: January 2012., Taming Mona Lisa: Communicating Gaze Faithfully in 2D and 3D Facial Projections, SAMER AL MOUBAYED, JENS EDLUND, and JONAS BESKOW , KTH Royal Institute of Technology Kana Misawa, 2 others, “LiveMask: Evaluation of communication in telepresence system using 3D face shape display”, [online], March 15, 2012, Information Processing Society of Japan Interaction 2012 (IPSJ Interaction 2012), [Heisei Retrieved November 8, 2018], Internet <URL: http://www.interaction-ipsj.org/archives/paper2012/data/Interaction2012/oral/data/pdf/12INT006.pdf>

In the above method, the computer always directs the line of sight of the character on the screen to the front. Therefore, the user can get the feeling that the character is talking to the user. However, this method does not give the user a satisfactory feeling that he or she is being watched by the character on the screen. Alternatively, the user cannot get a satisfactory feeling of making eye contact with the character on the screen. This tendency is remarkable when compared with the case where the target person is gazed at by another real person.

The phenomenon described above can be generalized as a psychological law of nature beyond the scope of the subject's subjectivity and assumptions. SUMMARY OF THE INVENTION An object of the present invention is to provide a character display means using a flat screen, which is suitable for causing the observer of the character to feel that the character is being watched.

[1] A method of displaying a character on a flat screen toward a target person present on the front side of the screen, comprising:
The line of sight of the character is continuously directed from the front of the screen by a first trigger including, as an operating condition, that the declination p of the polar coordinates of the subject with the display position of the character as the origin is within the angle range that produces the Mona Lisa effect. Display the character to come off,
A character's line of sight is smoothly changed by animation so that the same character's line of sight faces the front of the screen by a second trigger including as an operating condition that at least the display of the character by the first trigger has continued for a predetermined period of time. ,
Method.
[2] The first trigger further includes as a condition that the distance d between the display position of the character and the target person changes from a state larger than a predetermined value R1 to a state smaller than a predetermined value R1.
The method according to [1].
[3] The second trigger includes as a condition that at least the distance d after the elapse of the predetermined time is smaller than a predetermined value R2 which is smaller than a predetermined value R1.
The method according to [2].
[4] The passage of time from actuation of the first trigger until the distance d becomes smaller than a predetermined value R2, which is smaller than a predetermined value R1, is regarded as the display of the character by the first trigger continuing for a predetermined time.
The method according to [3].
[5] After the actuation of the second trigger, the character's line of sight and the direction of the character's face continue to be fixed in the front direction of the screen even if the declination angle p of the polar coordinates of the subject changes.
The method according to [3].
[6] After the activation of the second trigger, while fixing the line of sight of the character in the direction of the front of the screen, the direction of the character's face is smoothly animated to follow the change in the declination angle p of the polar coordinates of the target person. move,
The method according to [3].
[7] When the character is displayed by the first trigger, the character's line of sight, which was previously directed toward the front of the screen, moves smoothly by animation so that the line of sight of the character moves away from the front of the screen;
The method according to [1].
[8] While the first trigger displays the character so that the character's line of sight deviates from the direction of the front of the screen, the character's line of sight continues to move with animation;
The method according to [7].
[9] When the character's line of sight continues to move, the character's line of sight passes in the direction in front of the screen, but the character does not stare in the direction in front of the screen;
The method according to [8].
[10] changing the line of sight by the second trigger even if the declination p of the polar coordinates of the subject person is within the angle range causing the Mona Lisa effect after the predetermined time has elapsed;
The method according to [1].
[11] changing the line of sight by the second trigger even if the target person is in front of the display position after the predetermined time has elapsed;
The method according to [1].
[12] Even if the line of sight of the character changes due to the second trigger, the direction of the character's face does not change.
The method according to [1].
[13] The polar coordinates are circular coordinates when a plane parallel to the display direction of the screen is planarly viewed.
The method according to [1].
[14] The screen is configured by attaching a half mirror to the display area of a self-luminous display,
The display area of the display is observed from the subject through the half mirror,
The method according to [1].
[15] A controller that receives a detection signal from a sensor and sends a video signal to a display,
The display has a flat screen, receives the video signal, and displays the character at a predetermined display position on the screen based on the video signal,
The sensor sends to the controller a detection signal indicating the presence of the target person in front of the screen, wherein the sensor detects the deviation of the target person's polar coordinates with the display position of the character as an origin. When the angle p is in the angle range that produces the Mona Lisa effect, it is possible to detect the presence of the subject,
The controller is
When the detection signal is received, a video signal is sent to continuously display the character whose line of sight is away from the front of the screen at the display position, and further, the display of the character whose line of sight is removed continues for a predetermined period of time. After that, sending a video signal that displays an animation in which the same character smoothly moves the line of sight toward the front of the screen at the display position;
controller.
[16] Even if the target person exists in front of the display position, this is sent to the controller as the detection signal,
The controller sends a video signal to display the character who is out of line of sight when receiving the detection signal even if the target person is in front of the display position.
The controller according to [15].
[17] A program for operating a computer as the controller according to [15].
[18] Sending the program described in [17] to the computer via a network and installing the program in the computer.
[19] A server that stores the program described in [17] and that transmits the program to the computer via a network.

According to the present invention, it is possible to provide a character display means using a flat screen, which is suitable for causing the observer of the character to feel that the character is watching him or her.

Plan view 1 of observer and character. Plan view 2 of observer and character. Flowchart of the display method. Image 1 of the character. Character image 2. A transition diagram between the position of the target person and the line of sight of the character. A transition diagram of a character's line of sight. A transition diagram of a subject's position. Figure 1 of the movement of the subject. Figure 2 of the movement of the subject. Connection diagram of the controller and other devices. Observation diagram of the screen. Top view of target person and character. Logical configuration diagram of the controller.

<Mona Lisa effect>

This embodiment is utilized under conditions where the Mona Lisa effect occurs. The general concept of the Mona Lisa effect is explained in Non-Patent Document 1. FIG. 1 shows the positional relationship of the character Ch and the observers Ob1 and Ob2 in plan view. The character Ch is originally displayed two-dimensionally in the screen Sc. The representation of the character Ch in the figure is virtual. The character Ch faces the front of the screen Sc. The observer Ob1 is ahead of the line of sight. The observer Ob2 is also on the front side of the screen Sc. However, the observer Ob2 is at a position shifted to the left side of the character Ch.

In FIG. 1, the observer Ob1 feels that he/she is being watched by the character Ch. This is naturally understood because the observer Ob1 is in front of the line of sight of the character Ch. Even if the character Ch is a real person, the observer Ob1 feels that the character Ch is watching.

On the other hand, in FIG. 1, the observer Ob2 is not in the line of sight of the character Ch. However, the observer Ob2 also feels that he himself is being watched by the character Ch. This is the so-called Mona Lisa effect. If the character Ch is a real person, the observer Ob2 cannot feel that the character Ch is watching. This is because the line of sight is completely deviated from the direction toward the observer Ob2. That is, the Mona Lisa effect is a phenomenon peculiar to characters displayed on a flat screen.

In FIG. 1, it is unknown whether the observer Ob1 is potentially experiencing the Mona Lisa effect. In this specification, the embodiment will be described on the premise that the observer Ob1 is in a range where the Mona Lisa effect does not occur. Also, the embodiment will be described on the premise that the observer Ob2 is in the range where the Mona Lisa effect occurs. All of these are for convenience of technical explanation. The descriptions herein may not accurately represent psychological laws of nature in some aspects.

FIG. 2 also shows the positional relationship between the character Ch and the observers Ob1 and Ob2 in a plan view. The character Ch has his line of sight directed to the left side of the character Ch. Therefore, the observer Ob1 feels that the character Ch is looking away from himself. Alternatively, the observer Ob1 feels that the observer Ob2 is being watched by the character Ch. From the observer Ob1, it seems that the observer Ob2 is in front of the line of sight of the character Ch, so this is naturally understood.

On the other hand, in FIG. 2, the observer Ob2 also feels that the character Ch is looking away from him. The observer Ob2 feels that the line of sight of the character Ch is directed to the left side of the character Ch. This is one aspect of the Mona Lisa effect.

<Character display method>

The flow chart of FIG. 3 shows the character display method of this embodiment. 4 and 5 are plan views of the subject Tg and the character Ch. In this display method, a character Ch is displayed in a planar screen Sc. At this time, the character Ch is displayed toward the target person Tg present on the near side of the screen Sc. The following description will be made with reference to these figures as appropriate. The subject Tg may or may not observe the character Ch. Therefore, in this embodiment, a person who is a potential observer is expressed as a target person Tg. One aspect of the method of the present embodiment is the automated presentation under the assumption that the subject Tg is observing the character Ch.

As shown in FIGS. 4 and 5, the character Ch is displayed on the screen Sc while directing the eyes of the character Ch toward the target person Tg. An illustration of a girl's bust is shown as an example of the character Ch.

In FIGS. 4 and 5, by changing the depiction of the eyes of character Ch, the direction of the line of sight can be displayed in various ways. It is considered necessary to depict at least the white of the eye and the iris to depict the pupil. In medical terms, the pupil is the pupil surrounded by the iris. However, since this embodiment discusses the rendering method, the iris surrounded by the white of the eye will be called a "pupil".

As shown in FIG. 3, in step S31, it is determined whether or not the operating condition of the first trigger is satisfied. When the first trigger is activated, the line of sight of the character is removed from the front and the character is displayed in step S32. The first trigger has an operating condition that the Mona Lisa effect occurs at least in the subject Tg. The first trigger may have other additional activation conditions.

As shown in FIG. 4, whether or not the Mona Lisa effect is generated can be considered within a polar coordinate system (circular coordinate system) extending horizontally with the display position of the character Ch as the origin. This polar coordinate system is a polar coordinate system centered on the display position of the character Ch when viewed from above in a plane parallel to the display direction of the screen Sc.

In FIG. 4, the center of the virtual existence position of the character Ch set behind the screen Sc is the origin of the polar coordinate system. The origin of the polar coordinate system may be the midline of the character Ch displayed on the screen Sc.

As shown in FIG. 4, the direction directly in front of the screen Sc is assumed to be 0 degrees. Let p be the direction of the target person Tg. If p=0, the target person Tg is positioned in front of the display position of the character Ch. In the drawing, the target person Tg is located on the right side of the origin. The target person Tg may be located on the left side of the origin. The left side of the figure can also be considered as an area where p<0. However, this specification does not consider whether p is positive or negative. p is considered as an absolute value.

Furthermore, as shown in FIG. 4, let d be the distance between the target person Tg and the origin. The distance d corresponds to the radius of the polar coordinate system. The positional relationship between the character Ch, the screen Sc, and the subject Tg can be identified by the declination p and the distance d. If the declination p is within a predetermined angle range, the Mona Lisa effect is produced for the subject Tg.

As shown in FIG. 4, the predetermined angular range is assumed to be Mn1<p<Mn2. The angle Mn1 is greater than 0 degrees. The angle Mn1 may be obtained experimentally. In this embodiment, the angle Mn1 is 10 degrees (Non-Patent Document 2). In the case of p<Mn1, the Mona Lisa effect has no effect on whether or not the line of sight from the character Ch is felt. The angle Mn2 can be expanded up to 90 degrees.

When the condition is satisfied and the first trigger is actuated, the character Ch is displayed such that the line of sight of the character Ch deviates from the front direction of the screen Sc as shown in FIG. The state in which the character Ch is out of line of sight continues.

In FIG. 4, the direction of the line of sight of the character Ch can be changed by drawing the pupil. Since the character Ch is an illustration, the direction of the line of sight of the character Ch depends on the subjectivity of the observer. Therefore, for example, as shown in FIG. 2, the front observer Ob1 may observe the character Ch to evaluate the line-of-sight direction of the previously drawn character. Evaluation may be done statistically.

In FIG. 4, the direction of the line of sight of the character is represented by the declination g. When g=0, the line of sight is parallel to the front direction of the screen Sc. Averting of the line of sight of the character Ch from the front direction of the screen Sc is represented by g>Av. Angle Av is greater than zero. Angle Av may be obtained experimentally. If the angle of declination g is greater than or equal to the angle Av, it can be said that the line of sight of the character Ch is off. In the figure, the declination angle g of the line of sight is smaller than the declination angle p of the subject Tg. The argument g may be greater than the argument p. Argument g may coincide with argument p.

Next, as shown in FIG. 3, in step S33, it is determined whether or not the operating condition of the second trigger is satisfied. Monitor the passage of time for judgment. When the second trigger is activated, the line of sight of the character moves forward in step S34. The operation condition of the second trigger is that the display of the character by the first trigger (step S32) continues for a predetermined period of time. Here, it is preferable that the predetermined time is several seconds at the longest. When setting the predetermined time, the time required for natural communication between real people is taken into consideration. The secondary trigger may have other additional activation conditions. As will be described later, the determination of the second trigger need not involve monitoring the passage of time.

When the condition is satisfied and the second trigger is activated, the line of sight of the character Ch moves as shown in FIG. Character Ch shown in FIG. 5 is the same character as character Ch shown in FIG. Remove the line of sight and change the line of sight to the front direction with the same character.

As a result of the movement of the line of sight, the character Ch is displayed such that the line of sight of the character Ch faces the front of the screen Sc as shown in FIG. When moving the line of sight, it moves smoothly with animation. In the change from FIG. 4 to FIG. 5, the line of sight transitions from the oblique left direction for the character Ch to the front direction.

As shown in FIG. 4, the change in the line of sight of the character Ch caused by the second trigger does not follow the change in the direction of the face of the character Ch. In one aspect, even if the line of sight of the character Ch changes due to the second trigger, the direction of the face of the character Ch does not change. In the drawing, the face of the character Ch remains facing the front of the screen Sc while the line of sight is transitioning. In other aspects, the face of the character Ch may face in a direction other than the front.

In FIG. 5, the state in which the line of sight of the character Ch faces the front continues. Character Ch gazes straight ahead. In the figure, Ob2 is located in the range where the Mona Lisa effect occurs (Mn1<p<Mn2). Due to the Mona Lisa effect, the subject Tg feels as if the character Ch is looking at himself. When the subject Tg is gazing at the character Ch, the subject Tg strongly feels that the character Ch is gazing at him/herself. Furthermore, the target person Tg feels that the eyes have met with the character Ch.

<Features of characters used>

In Figures 4 and 5, the character Ch may blink. In FIGS. 4 and 5, character Ch may express facial expressions and emotions. For example, when the first trigger is actuated (FIG. 4), the facial expression may not be shown, and when the second trigger is actuated (FIG. 5), the eyes may be directed to the front and a smile may appear. When the display method is used to provide entertainment, it is also effective to show a facial expression that surprises or glares at the observer.

In Figures 4 and 5, the iris may be represented in a darker color than the white of the eye. An expression method of adding a highlight to an iris filled with a dark color is included in one aspect of the present embodiment. The whites of the eyes are not limited to pure white. The white of the eye may be thought of as the region in the eye surrounding the iris. The iris may be made brighter than the white of the eye in an expression technique that inverts the gradation. As an expression technique for inverting gradation, there is a shadow play style.

In FIGS. 4 and 5, character Ch has a face with at least pupils. The character Ch may be a face image, a bust image, an upper body image, or a full body image. The character Ch may be a seated image or a standing image. As long as the face of the character Ch faces the front, the pose is not restricted. In the drawing, the character Ch faces the front. As long as the line of sight can be directed to the front side of the screen Sc as an expression, there is no restriction on the orientation of the face.

In Figures 4 and 5, the character Ch is unaided. Character Ch may wear glasses or goggles. The type of spectacles is not limited. However, it is not appropriate to wear dark sunglasses because the direction of the line of sight cannot be seen from the outside. Dark goggles are also not suitable.

In FIGS. 4 and 5, one of the pupils of the character Ch may be hidden by hair or an eyepatch. It is also effective to express that the character Ch is looking straight ahead from the shadows.

In FIGS. 4 and 5, when the directions of the pupils are changed, the directions of the pupils of both eyes may be different. It is possible to express according to the expression technique. Such an expression method is considered useful when the face is not facing the front. It is sufficient if the target person Tg can specify the line-of-sight direction of the character Ch without discomfort. Aside from this, expressing that the character Ch has a squint is also included in one aspect of the present embodiment.

In FIGS. 4 and 5, the character Ch may be a real image of a person. The character Ch may be a more deformed illustration. The character Ch may be created by 3DCG modeling. The character Ch may have human or non-human animal motifs. Character Ch may be a robot or a fantasy creature. It is preferable that the character Ch be anthropomorphized to the extent that the direction of the line of sight of the target person can be understood. For example, since insects see their surroundings with compound eyes, they do not have a line of sight in the first place. If the character Ch is an illustration of an insect, it is effective to replace the eyes in the illustration with comical eyes.

<Consider the height direction>

The polar coordinate system centered on the display position of the character Ch shown in FIGS. 4 and 5 is circular coordinates when a plane parallel to the display direction of the screen Sc is viewed from above. In one aspect, the height direction can also be considered for the Mona Lisa effect. When the character Ch looks away from the line of sight based on the first trigger, the line of sight may be removed in the height direction. The viewing direction may be defined in a cylindrical or spherical coordinate system. Furthermore, when the character Ch turns his line of sight forward based on the second trigger, it is preferable to turn his line of sight horizontally. For example, even if the target person is a short child, the line of sight of the character Ch does not have to be directed downward when the second trigger is activated.

<Gaze before activation of the first trigger>

In one aspect, the condition prior to actuation of the first trigger may be considered. FIG. 6 shows the transition between the line of sight of the character Ch and the position of the observer. First, the explanation will focus on the change in the line of sight of the character Ch. In the original state (Original) in the upper part of the figure, the line of sight of the character Ch faces the front of the screen Sc.

Next, as shown in the middle part of FIG. 6, a state is displayed in which the line of sight of the character Ch is off the front of the screen Sc by the first trigger. At this time, the line of sight of the character Ch is smoothly moved by animation. In one aspect, during this time, the face of the character Ch also remains facing the front of the screen Sc. In other aspects, the face of the character Ch may face in a direction other than the front.

In the middle part of FIG. 6, the line of sight transitions from the frontal direction, which corresponds to the original state for the character Ch, to the oblique direction to the left. Even when the target person Tg is gazing at the character Ch, the target person Tg feels that he or she is not being watched by the character Ch. However, the subject Tg recognizes that the character Ch is looking at something in front of the character. For the subject Tg, the direction is to the right of the subject Tg.

Furthermore, as shown in the lower part of FIG. 6, a state in which the line of sight of the character Ch faces the front of the screen Sc is displayed by the second trigger. This operation is as described above. The line of sight of the character Ch is smoothly moved by animation. In one aspect, during this time, the face of the character Ch remains facing the front of the screen Sc. In other aspects, the face of the character Ch may face in a direction other than the front.

In the lower part of FIG. 6, the line of sight transitions from the oblique left direction for the character Ch to the front direction. When the subject Tg is gazing at the character Ch, the subject Tg strongly feels that the character Ch is gazing at him/herself. Furthermore, the target person Tg feels that the eyes have met with the character Ch.

In another aspect, the line of sight of the character Ch may be directed in a direction other than the front of the screen Sc before the first trigger is activated. That is, in one aspect, in the starting state, the line of sight of the character Ch may be directed either toward the front of the screen Sc or other than the front. However, the line of sight is turned to other than the front by the first trigger. Furthermore, the line of sight is directed to the front by the second trigger.

<Approaching Observer>

In FIG. 6, the observer's position is also changed. In the illustrated example, the target person Tg is approaching the screen Sc. Specifically, the character Ch is approaching. In the original state (Original) shown in the upper part of the figure, the distance d is greater than the predetermined value R1.

As shown in the middle part of FIG. 6, the target person Tg approaches the character Ch. In the figure, the distance d between the display position of the character Ch and the target person Tg is smaller than the predetermined value R1. In one aspect, the distance d being less than a predetermined value R1 may be an additional operating condition for the first trigger. In another aspect, the additional actuation condition of the first trigger may be that the distance d, which was larger than the predetermined value R1 in the original state, becomes smaller than the predetermined value R1. In other words, the first trigger may be activated by detecting that the target person Tg is approaching.

As shown in the lower part of FIG. 6, the target person Tg approaches the character Ch. In the figure, the distance d between the display position of the character Ch and the target person Tg is smaller than a predetermined value R2 after a predetermined display time has passed until the second trigger is activated. Note that the predetermined value R2 is smaller than the predetermined value R1.

In one aspect, as shown in the lower part of FIG. 6, the distance d being smaller than a predetermined value R2 may be an additional operating condition for the second trigger. In another aspect, the additional actuation condition of the second trigger may be that the distance d, which was greater than the predetermined value R2 in the original state, becomes smaller than the predetermined value R2. In one aspect, after a predetermined display time has elapsed, the second trigger is activated by detecting that the target person Tg has come closer.

There are various aspects of considering the proximity of the subject Tg. For example, in FIG. 6, the time during which the distance d transitions from R1 to R2 may be the predetermined display time corresponding to the operating condition of the second trigger. In this case, the monitoring of the elapsed time from actuation of the first trigger to actuation of the second trigger is included in monitoring the distance d. That is, the second trigger operating condition is that the distance d has fallen below R2 at least while the character Ch continues to be displayed after the first trigger is operated, without using a direct operating condition related to the passage of time. good too. In other words, the change in the distance d may be regarded as the passage of time, and the second trigger may be activated. For example, if the distance d becomes smaller than the predetermined value R2 before the predetermined display time elapses, it may be assumed that the predetermined display time has elapsed before the predetermined display time actually elapses. In another aspect, even if the distance d becomes smaller than the predetermined value R2 before the predetermined display time elapses, the second trigger may not be activated until the predetermined display time actually elapses. Alternatively, it is conceivable that the distance d is less than the predetermined value R2 at the moment of actuation of the first trigger. In this case, the lapse of a predetermined display time may be set as the operating condition of the second trigger regardless of the distance d. In other embodiments, observer proximity may not be considered at all.

<Expression when line of sight is removed by the first trigger>

In one aspect, conditions after actuation of the first trigger may be considered. FIG. 7 shows the transition of the line of sight of the character Ch. As shown in the upper part of FIG. 7, the first trigger causes the character Ch, who is out of line of sight, to be displayed. Here, a certain amount of time elapses until the second trigger is activated. It is not necessary to fix the line of sight of the character Ch during this interval (Interval).

In the middle part of FIG. 7, the line of sight transitions from the oblique left direction for the character Ch to the oblique right front direction. The line of sight of the character Ch may continue to be moved even after the character Ch, which is out of line of sight, is once displayed. At this time, the line of sight of the character Ch is smoothly moved by animation. After a predetermined period of time has elapsed, as shown in the lower part of FIG. 7, the line of sight of the character Ch is turned to the front by the second trigger. This operation is as described above.

When continuing to move the line of sight of the character Ch in the middle part of FIG. 7, the line of sight of the character Ch may pass in the front direction of the screen Sc. However, make sure that the line of sight does not stare at the front of the screen. The line of sight of the character Ch may not pass through the front of the screen Sc at all.

By controlling the line of sight as described above, it is possible to maintain a state in which the Mona Lisa effect does not occur until the actuation of the second trigger.

<Position where the Mona Lisa effect occurs and the second trigger>

From FIG. 4 to FIG. 5, the subject Tg remains in the same place. During this period, the Mona Lisa effect always affects the subject's Tg. By moving the line of sight away from the direction in which the target person Tg is present, the user feels that the target person Tg is being watched by the character Ch. This is due to the Mona Lisa effect.

The mode shown in FIG. 5 is characterized in that the line of sight is changed by the second trigger even if the declination angle p of the polar coordinates of the target person Tg is within the angle range that causes the Mona Lisa effect after the predetermined display time has elapsed. . An additional activation condition for the second trigger may be that the subject Tg is in a position where the Mona Lisa effect is occurring. An additional operating condition for the second trigger may be that the subject Tg remains in the same place as when the first trigger was operated.

As shown in FIG. 8, the position of the subject Tg may transition in another aspect. While the character Ch is out of line of sight, the target person Tg may move to a range free from the influence of the Mona Lisa effect. As an example, in the middle part of the drawing, the target person Tg is moving in front of the character Ch.

As shown in the lower part of FIG. 8, after a predetermined display time elapses, a state in which the line of sight of the character Ch faces the front of the screen Sc is displayed by the second trigger. Even if the target person Tg exists in front of the display position of the character Ch, the line of sight is changed by the second trigger. The conditions for activating the second trigger may not include that the subject Tg is in a position where the Mona Lisa effect is occurring. An additional activation condition for the second trigger may be that the subject Tg is not in a position where the Mona Lisa effect is occurring.

<Indication for moving person>

9 and 10 show the positions P0 to P5 of the subject moving across the front of the screen Sc. For convenience in designing the display method, the origin of the polar coordinate system is set at the midline of the character Ch displayed on the screen Sc. The argument p changes as the position changes. The deflection angle p in the figure represents the deflection angle at the position P4. Assume that the position P3 is in front of the character Ch.

In FIGS. 9 and 10, at position P0, the line of sight is removed by the first trigger, and at position P1, the line of sight is changed by the second trigger. Further assume that the subject passes in front of the character Ch at positions P2, P3, P4 and P5 in that order.

In FIG. 9, an image of a character Ch is displayed on the screen Sc with both the line of sight direction and the face direction fixed to the front. This state is after actuation of the second trigger. If the moving person observes the character Ch, the moving person will feel that he or she is being watched by the character Ch. This sensation occurs wherever the traveler is at P1-P5. In other words, it is a sensation that arises regardless of the standing position of the moving person.

In FIG. 9, the subject feels that the face of the character Ch is not moving. Also, the face of the character Ch does not actually change its orientation. Therefore, it is difficult for the target person to feel that the character Ch is tracking the target person with the line of sight. The Mona Lisa effect creates a feeling of being watched. However, even the feeling of being tracked by the line of sight is not effectively generated.

The character Ch is displayed on the screen Sc such that the orientation of the face of the character Ch changes as shown in FIG. While the direction of the line of sight of the character Ch is fixed to the front, the direction of the face follows the moving person. The direction of the character Ch's face is linked to the positions P1 to P5 of the moving person. The direction of the face of the character Ch is smoothly changed by animation so as to follow the change in the declination angle p of the polar coordinates of the subject.

In FIG. 10, the moving person can be made to feel as if the character Ch is tracking the moving person with his line of sight. Alternatively, the moving person can be made to feel such a sensation more strongly than the method shown in FIG.

<Connection between the controller and other devices>

An apparatus for implementing the display method of this embodiment will be mainly described below. The device is exemplary and not the only mechanical means for carrying out the display method. FIG. 11 shows the controller 40 and each device connected thereto. A display device 41 is connected to the controller 40 . This connection may be via a network or via suitable video signal wiring. This connection can be wired or wireless.

As shown in FIG. 11, the display device 41 has a display 42 and a half mirror 43 . The display 42 is, for example, a liquid crystal display or an organic EL display. The display 42 is flat without irregularities. Here, fine unevenness that cannot be visually recognized is not taken into consideration. Flatness means not having large unevenness to the extent that it affects the basic shape of the display image. A half mirror 43 is installed on the display area 44 of the display 42 . The half mirror 43 may be pasted on the display area 44 of the display 42 .

The display area 44 shown in FIG. 11 presents images to the subjects 45a and 45b. An image is displayed through the half mirror 43 . For the sake of convenience, the surface of the front side of the half mirror 43, which is the left side in the drawing, is referred to as the screen Sc. The half mirror 43 is one size larger than the display area 44 .

In FIG. 11, a camera 39 corresponding to a sensor is connected to the controller 40 . This connection may be via a network or via suitable video signal wiring. This connection can be wired or wireless. The camera 39 may be a CMOS (Complementary metal-oxide-semiconductor) sensor. Camera 39 may be replaced by other optical sensors, such as infrared sensors.

In FIG. 11, controller 40 receives a detection signal from camera 39 . Further, controller 40 sends a video signal to display 42 . A display 42 receives a video signal from the controller 40 . The display 42 displays the character Ch at a predetermined display position in the display area 44 based on the video signal.

In FIG. 11, the camera 39 photographs at least one of the subjects 45a and 45b. The camera 39 sends information indicating that at least one of the subjects 45a and 45b exists on the front side of the screen Sc to the controller 40 as a detection signal. The detection signal may be a video signal generated by camera 39 .

In FIG. 11, the camera 39 may continuously take pictures, continuously generate video signals, and continuously send video signals to the controller 40 whether or not there is a subject within its picture-taking range. Camera 39 may take pictures, generate video signals, and send video signals when requested by controller 40 .

FIG. 12 shows the screen Sc of the display device 41. As shown in FIG. A character Ch is displayed in the display area 44 . The background around the character Ch displayed in the display area 44 is dark. Therefore, on the screen Sc, the half mirror 43 works to allow the light from the display to pass through the half mirror, so that the actual background reflected by the half mirror appears around the character Ch displayed. A real background and a character Ch are superimposed on the screen Sc.

In FIG. 12, the screen Sc is flat without unevenness. There is no idea of three-dimensionalizing the face and eyes of the character Ch by providing special unevenness in the vicinity of the face of the character Ch displayed on the screen Sc.

Return to FIG. A camera 39 is further connected to the controller 40 as a sensor. The camera 39 is arranged behind the half mirror 43, on the right side in the figure. Camera 39 photographs subjects 45a and 45b. As shown in FIG. 12, the subject cannot visually recognize the camera 39 . This is because the camera 39 is blocked from view by the half mirror 43 .

Note that the device configuration for forming the screen Sc shown in FIGS. 11 and 12 is not limited to the above. The screen Sc does not have small irregularities. However, the screen Sc may be uniformly gently curved. When displaying a character Ch whose face width is about 20 cm, the radius of curvature of the screen Sc is preferably larger than 10 cm, and the screen Sc is preferably flat. The screen Sc may be formed by a projection type screen instead of the display device provided with a half mirror. Such a screen may be a front type screen using light reflection or a rear type screen using light transmission. The half mirror 43 may be omitted. That is, the display device 41 may have only the display 42 . The display 42 is, for example, a liquid crystal display or an organic EL display.

The function of the camera will be described with reference to FIG. The drawing shows target persons 45a and 45b and a character Ch in plan view. Also shown in the figure is a subject 45c, who will be described later. The character Ch faces the target person 45a in the direction Fc of the face. The character Ch faces the direction of the line of sight Gz1 or Gz2 toward the front side of the screen Sc. In the figure, the line of sight direction Gz1 is the direction away from the front direction of the screen Sc. In the drawing, the line of sight direction Gz2 is the front direction of the screen Sc.

In FIG. 13, the camera 39 is a sensor capable of detecting the existence of the subject when at least one of the subjects 45a and 45b is within the range where the Mona Lisa effect occurs. The range that produces the Mona Lisa effect means that the declination angle p of the polar coordinates of at least one of the subjects 45a and 45b, with the display position of the character Ch as the origin, is positioned within the angular range that produces the Mona Lisa effect. Such angular ranges are as described above with reference to FIGS. A declination p in the drawing represents the declination of the polar coordinates of the subject 45a.

<System and first trigger>

FIG. 14 describes the structure of the system that constitutes the controller 40. As shown in FIG. This system may consist of a computer and a program for operating the computer as a controller. A program is a kind of image generation software. The program does not have to consist of one program. Part or all of the program may be transmitted to the computer from a server storing part or all of the program via a network. For example, a program may be installed on the character and on the computer. The program may be stored in non-volatile memory or disk media. The computer does not have to consist of one unit. A part of the system acting as a controller may be placed on the cloud network.

The image generation software of the controller 40 comprises components 51-54. Components 51-54 are each software components in one aspect. Although hardware such as a signal converter suitable for this may be required for exchanging signals between the controller 40 and the outside, it is omitted in the figure.

The processing related to the first trigger in FIG. 14 is as follows. The controller 40 converts detection signals received from the camera 39 . It is the component 51 that transforms the sensed signal. Component 51 generates information for identifying the location of the subject based on the converted sensing signals. Component 51 contains the location of the subject. Subjects may not be detected. More than one subject may be detected at the same time. Component 51 may have motion capture capabilities. Following the example of FIG. 13, the component 51 generates information specifying the declination p of the subject 45a.

In FIG. 14, controller 40 generates background information. It is the component 52 that generates the background information. The component 52 pre-stores virtual background objects and their positions. The component 52 may acquire the virtual background object and its position in advance via a network (not shown). Following the example of FIG. 13, the virtual background objects B01 to B09 and their positions are stored in advance. The position of the virtual background object provides, so to speak, a representative value of the declination between the direction of the line of sight and the direction of the face of the character. The position of the virtual background object B05 provides, so to speak, the front direction of the screen Sc.

In FIG. 14, the controller 40 selects a gaze target. It is the component 53 that selects the gaze target. Component 53 references the location of the subject of component 51 . Component 53 references the position of the virtual background object of component 52 . Following the example of FIG. 13, the component 53 refers to the positions of the virtual background objects B01 to B09 and the position of the subject 45a. Component 53 may also refer to the location of subject 45b.

In FIG. 14, component 53 determines whether the position of subject 45a produces the Mona Lisa effect. If the position produces the Mona Lisa effect, then the component 53 selects the position of the virtual background object that is not in the front direction as the gaze target. Component 53 contains the selected gaze target. Following the example of FIG. 13, the component 53 selects a virtual background object other than the virtual background object B05 as a gaze target. In the drawing, the virtual background object B07 is selected as the gaze target. The component 53 may select the position of the target person 45b as the gaze target instead of the virtual background object B07. Component 53 may also contain a record of the actuation of the first trigger along with a time record.

In FIG. 14, the controller 40 selects a facing target. It is the component 53 that selects the facing target. Following the example of FIG. 13, the position of the target person 45a may be set as the face-oriented target, and the virtual background objects B01 to B09 may be set as the face-oriented target. The facing target does not have to match the gaze target. Component 53 contains the selected facing target.

In FIG. 14, the controller 40 sets the line-of-sight direction. It is the component 54 that sets the direction of the line of sight. Component 54 references the gaze selected by component 53 . The component 54 stores the set gaze direction. Following the example of FIG. 13, the line-of-sight direction Gz1 is accommodated.

In FIG. 14, the controller 40 sets the orientation of the face. It is component 54 that sets the orientation of the face. Component 54 references the facing target selected by component 53 . The component 54 contains the orientation of the face that has been set. Following the example of FIG. 13, the direction Fc of the face is accommodated.

In FIG. 14, component 55 refers to the gaze direction and face direction of component 54 . A component 55 contains a CG model of the character. The component 55 may acquire in advance the CG model of the character via a network (not shown). Component 55 refers to the CG model of the character. The CG model of the character can set the direction of the line of sight and the direction of the face, targeting at least the virtual background object. In addition, it is possible to display an animation in which the direction of the line of sight and the direction of the face change by targeting between these virtual background objects.

In FIG. 14, the component 55 draws based on the direction of the line of sight, the direction of the face, and the CG model of the character. A video signal is generated based on the drawing. Following the example of FIG. 13, the video signal continuously displays the character Ch at a predetermined display position in the screen Sc. This video signal includes an animation in which the character Ch moves its line of sight smoothly while moving its line of sight away from the front of the screen Sc. The character Ch transitions the line of sight to the direction Gz1. Note that the line of sight is not drawn accurately in FIG. This video signal includes the character Ch turning its face in the direction Fc.

In FIG. 14, controller 40 sends a video signal to display device 41 . It is the component 55 that sends the video signal. The display device 41 displays the animation of the character whose line of sight is smoothly moved, and then continues the display of the character whose line of sight is removed for a predetermined period of time. The second trigger is then activated as follows.

<System and second trigger>

The processing related to the second trigger in FIG. 14 is as follows. The component 51 converts the detection signal received from the camera 39 again. Component 51 generates information for identifying the location of the subject based on the converted sensing signals. Component 51 contains the location of the subject. Subjects may not be detected. More than one subject may be detected at the same time.

In FIG. 14, component 53 references the location of the subject in component 51 . If the target person of the component 51 cannot refer to it, the subsequent operations need not be performed. For example, the subject that resulted in the actuation of the first trigger moves out of the detectable range of the camera.

In FIG. 14 component 53 also references the activation time of the first trigger. Component 53 determines whether a predetermined display time has elapsed since activation of the first trigger. If the predetermined display time has elapsed since the actuation of the first trigger, then the component 53 again selects the gaze target. Component 53 references the virtual background object of component 52 . Following the example of FIG. 13, the component 53 refers to the virtual background object B05 representing the front direction of the screen Sc. Following the example of FIG. 13, the virtual background object B05 is selected as the gaze target. The component 53 houses the selected virtual background object B05.

In FIG. 14, component 54 sets the direction of the line of sight. It is the component 54 that sets the direction of the line of sight. Component 54 references the selected virtual background object B05. The component 54 stores the set gaze direction. Following the example of FIG. 13, the line-of-sight direction Gz2 is accommodated.

In FIG. 14, component 55 refers to the viewing direction of component 54 . The component 55 refers to the CG model of the same character as the character displayed by the first trigger. The component 55 draws based on the line-of-sight direction and the CG model of the character. A video signal is generated based on the drawing. According to the example of FIG. 13, this video signal is such that the line of sight of the same character Ch as the character Ch displayed by the first trigger is directed to the front of the screen Sc. At this time, the video signal includes an animation in which the character Ch smoothly moves the line of sight. The component 55 sends video signals to the display device 41 .

In FIG. 14, component 55 sends a video signal to display device 41 . After animation-displaying the character whose line of sight smoothly moves from the direction Gz1 to the direction Gz2, the display device 41 continues to display the character whose line of sight is directed to the front. The Mona Lisa effect creates the feeling that the subject is being watched by the character.

<Versatility of controller>

The controller 40 shown in FIG. 13 is suitable for carrying out the display method described above. The controller 40 and the devices connected thereto are not limited to those having only the function of executing the display method described above. The controller 40 may be one that can make the character look away even if the target person is in a position where the Mona Lisa effect does not occur. Such presentation of the character Ch for the subject may be executed using the system shown in FIG.

As shown in FIG. 13, the target person 45c exists in front of the display position of the character Ch. The subject 45c does not have the Mona Lisa effect. The camera 39 captures the entire front of the screen Sc including the subject 45c. Camera 39 sends a detection signal to controller 40 including the presence of subject 45c.

In FIG. 13, even if the target person 45c is present in front of the display position of the character Ch, the controller 40, upon receiving the detection signal, sends a video signal to the display device to display the character Ch, which is out of line of sight. send. A character Ch whose line of sight is directed in the direction Gz1 is displayed on the screen Sc. Furthermore, after a predetermined display time has elapsed, an animation in which the character Ch turns his line of sight in the direction Gz2 is displayed on the screen Sc. The target person 45c feels that the character Ch is watching the target person 45c.

<Regarding background information generation and omission of virtual background objects>

The separation of gaze target selection and gaze direction setting in the embodiments shown in FIGS. 11-14 is for convenience of implementation. In another aspect, it is also possible to cause the controller 40 to implement the display method only by setting the direction of the line of sight. A component 52 that generates background information is not required. Also, the virtual background objects B01 to B09 are unnecessary. Component 53 for selecting a gaze target is unnecessary.

First, as shown in FIG. 13, the camera 39 serving as a sensor detects the positions of the subjects 45a and 45b. Next, it is determined whether or not the target person is in the position where the first trigger is operated within the range of the detected target person. If the target person is at the position where the first trigger is to be actuated, a direction other than the front is set as the line-of-sight direction of the character Ch. The direction of the face of the character Ch may be set to other than the front. You don't have to do it again.

Next, it is determined whether or not the target person is in the position where the second trigger is operated within the range of the detected target person. If the target person is at the position to activate the second trigger, the front direction Gz2 is set as the line-of-sight direction of the character Ch.

Also, as shown in FIG. 10, the orientation of the face of the character Ch may be changed. The direction of the face sets the direction of the subject who was in the position to activate the second trigger. Component 54 directly references the location of the subject in component 51 . The direction of the face of the character Ch is set to the direction of the target person. This process is continued until another cause for setting the face of the character Ch in a direction other than the direction of the subject occurs. Other causes include, for example, activation of the second trigger by another subject.

It should be noted that the present invention is not limited to the above embodiments, and can be modified as appropriate without departing from the scope of the invention. for example,

39 Camera 40 Controller 41 Display device 42 Display 43 Half mirror 44 Display area 45a-c Subject 51-55 Component Av Angle B01-B09 Virtual background object Ch Character d Distance Fc Direction of face g Deviation angle of line of sight Gz1 Direction of line of sight Gz2 Direction of line of sight Mn1 Angle Mn2 Angle Ob1 Observer Ob2 Observer p Deviation angle P0-P5 Position R1 Predetermined value R2 Predetermined value S31-S34 Step Sc Screen Tg Subject

Claims (19)

A method of displaying a character on a flat screen toward a target person present on the front side of the screen, comprising:
The line of sight of the character is continuously directed from the front of the screen by a first trigger including, as an operating condition, that the declination p of the polar coordinates of the subject with the display position of the character as the origin is within the angle range that produces the Mona Lisa effect. Display the character to come off,
The line of sight of the character is smoothly changed by animation so that the line of sight of the character faces the front of the screen by a second trigger including as an operating condition that at least the display of the character by the first trigger has continued for a predetermined period of time.
Method. The first trigger further includes as a condition that the distance d between the display position of the character and the target person changes from a state larger than a predetermined value R1 to a state smaller than a predetermined value R1.
The method of claim 1. The second trigger includes as a condition that at least the distance d after the elapse of the predetermined time is smaller than a predetermined value R2 which is smaller than a predetermined value R1,
3. The method of claim 2. The passage of time from actuation of the first trigger until the distance d becomes smaller than a predetermined value R2, which is smaller than a predetermined value R1, is regarded as the display of the character by the first trigger continuing for a predetermined time.
4. The method of claim 3. After the actuation of the second trigger, even if the declination p of the polar coordinates of the subject changes, the character's line of sight and the direction of the character's face continue to be fixed in the front direction of the screen.
4. The method of claim 3. After the actuation of the second trigger, the direction of the character's face is smoothly moved by animation so as to follow the change in the declination angle p of the polar coordinates of the target person, while fixing the line of sight of the character in the direction of the front of the screen.
4. The method of claim 3. When the character is displayed by the first trigger, the line of sight of the character, which was previously directed toward the front of the screen, is smoothly moved by animation so that the line of sight of the character deviates from the front of the screen.
The method of claim 1. While the first trigger causes the character's line of sight to move away from the front of the screen to display the character, the character's line of sight continues to move with animation.
8. The method of claim 7. When the character's line of sight continues to move, the character's line of sight passes in the direction in front of the screen, but is prevented from staring in the direction in front of the screen.
9. The method of claim 8. changing the line of sight by the second trigger even if the declination angle p of the polar coordinates of the subject person is within the angle range causing the Mona Lisa effect after the predetermined time has elapsed;
The method of claim 1. Changing the line of sight by the second trigger even if the target person is in front of the display position after the predetermined time has elapsed;
The method of claim 1. Even if the line of sight of the character changes due to the second trigger, the direction of the character's face does not change.
The method of claim 1. The polar coordinates are circular coordinates when a plane parallel to the display direction of the screen is viewed in plan,
The method of claim 1. The screen is configured by attaching a half mirror to the display area of a self-luminous display,
The display area of the display is observed from the subject through the half mirror,
The method of claim 1. A controller that receives a detection signal from a sensor and sends a video signal to a display,
The display has a flat screen, receives the video signal, and displays the character at a predetermined display position on the screen based on the video signal,
The sensor sends to the controller a detection signal indicating the presence of the target person in front of the screen, wherein the sensor detects the deviation of the target person's polar coordinates with the display position of the character as an origin. When the angle p is in the angle range that produces the Mona Lisa effect, it is possible to detect the presence of the subject,
The controller is
When the detection signal is received, a video signal is sent to continuously display the character whose line of sight is away from the front of the screen at the display position, and further, the display of the character whose line of sight is removed continues for a predetermined period of time. after that, sending a video signal to display at the display position an animation in which the character smoothly moves the line of sight toward the front of the screen;
controller. Even if the target person exists in front of the display position, this is sent to the controller as the detection signal,
The controller sends a video signal to display the character who is out of line of sight when receiving the detection signal even if the target person is in front of the display position.
16. The controller of claim 15. A program for operating a computer as the controller according to claim 15. The program according to claim 17 is transmitted to the computer via a network and installed in the computer to make the computer operate as the controller,
How . 18. A server storing the program of claim 17, the server transmitting the program to the computer via a network.

Images