JP7325757B2 - INTERACTABLE OBJECT DISPLAY PROGRAM AND INTERACTABLE OBJECT DISPLAY DEVICE - Google Patents

Description

The present invention relates to an interactive object display program and an interactive object display device.

2. Description of the Related Art Conventionally, it is known to display an object in a virtual space, such as a game character, on the screen of a portable information processing device such as a smart phone, tablet, or mobile phone. However, simply displaying a character in the virtual space on the screen does not give the user a sense of reality. Therefore, the present inventor reflects the interactive character in the virtual space displayed on the screen of the information processing device to the real world by reflecting it on a half mirror, and fuses the object in the real world with the character in the virtual space. We thought that an interactive object display device was necessary.

In addition, if a three-dimensional humanoid character that stands vertically in the virtual space appears to stand vertically when projected onto the real world, the user will easily feel that the character exists in reality. Furthermore, by adjusting the photographing direction of the three-dimensional character in the virtual space according to the line-of-sight direction in which the user views the virtual image, the user can be made to feel that he or she is looking at the character in the real world.

Utility Model Registration No. 3202689

However, according to the inventor's study, for example, when the user places the interactive object display device on the desk and looks at the virtual image through the half mirror, the user stands up, changes the height of the chair, etc. As a result, the line of sight of the user viewing the virtual image may change. When the user's line of sight changes and the virtual image becomes difficult to see, the user adjusts the positional relationship between the information processing device and the half mirror to make the virtual image easier to see.

However, if the information processing device and the half mirror are not properly moved, the character, which had been standing upright until then, will be displayed tilted, or the line-of-sight direction and the shooting direction will be misaligned, resulting in a loss of reality. It turned out that there is such a problem. In particular, when projecting an interactive three-dimensional character, the user strongly desires the realism of having a dialogue with the character, so it is required to reduce the visual discomfort.

Moreover, if it is difficult to adjust the orientation of the character, the original purpose of enjoying interaction with the character in the virtual space is lost. In addition, if a child or someone unfamiliar with the principle adjusts the position, the character will be viewed at an unnatural angle as it is viewed from an unnatural shooting direction.

By the way, there is a virtual image observation stand that displays a planar image displayed on a portable terminal in a lying or standing state in real space by reflecting a planar image displayed on a mobile terminal on a concave mirror, as in Patent Document 1. However, in the example of the virtual image observation stand disclosed in Patent Document 1, the positional relationship between the mobile terminal and the concave mirror is fixed by the mobile terminal fixing section, and the range in which the user views the virtual image is limited.

Further, as another example of Patent Document 1, changing the angle of a mobile terminal or a concave mirror is disclosed. In other words, the focus is on intentionally tilting a planar virtual image, as opposed to maintaining the angle. Moreover, in Patent Literature 1, it is necessary to manually adjust the positions of the mobile terminal and the concave mirror.

The present invention automatically determines an appropriate line-of-sight angle for viewing a virtual image of a three-dimensional interactive object in a virtual space according to the positions of an information processing device and a half-mirror with a simple configuration. It is an object of the present invention to provide an interactive object display program and an interactive object display device capable of displaying .

A first aspect is an interactive object display program executed in an interactive object display device for displaying a three-dimensional interactive object in a virtual space in a real space for interaction with a user. A device includes an information processing device and a support base capable of detachably supporting the information processing device, and the information processing device can interact with a posture detection unit and a display unit including a screen for displaying an image. a processor capable of executing an object display program; a half mirror having a reflecting surface capable of at least partially reflecting a screen; and an adjustment mechanism capable of adjusting an angle defined between the screen of the information processing device mounted on the mounting portion and the reflecting surface of the half mirror, wherein the first angle and the second angle are provided. The adjustment mechanism is configured such that the angle of and changes based on a first correspondence, and the first angle is between a virtual measurement reference plane in real space and a first plane extending along the screen. A second angle is the angle between the measurement reference plane and a second plane extending along the reflective surface, and a first correspondence is that the first angle is θA radians. At some point, the second angle θB radians becomes (−π/4+θA/2) radians, and the orientation detection unit is capable of detecting angle information corresponding to the first angle, and interactive object display program causes the information processing device to perform an interactive object display process when executed by the processor, and the interactive object display process determines the line-of-sight direction and measurement reference plane suitable for viewing the virtual image of the image formed by the reflective surface based on the angle information and the second correspondence relationship, and capturing an image of the interactable object from a shooting direction tilted by the viewing angle with respect to the shooting reference plane in the virtual space and displaying the created image on a screen, wherein a second correspondence is defined based on the first correspondence to represent a relationship between the first angle and the line-of-sight angle It is an interactive object display program.

According to the first aspect, the adjustment mechanism is configured such that the first angle of the screen and the second angle of the reflective surface are changed based on the first correspondence, and the second correspondence is , is defined based on the first correspondence to represent the relationship between the first angle and the line-of-sight angle. Then, the line-of-sight angle is determined based on the angle information corresponding to the first angle and the second correspondence. That is, even if the second angle of the reflecting surface is not measured when determining the line-of-sight angle, the positional relationship between the screen and the reflecting surface is indirectly reflected by the second correspondence, and furthermore, the virtual image plane is Reflects that it does not rotate. As a result, an appropriate line-of-sight angle for viewing a virtual image of a three-dimensional interactive object in virtual space can be automatically determined with a simple configuration according to the positions of the information processing device and the half mirror.

Further, according to this aspect, since the line-of-sight angle is determined based on the angle information, unlike the case where the angle information is not used, it is possible to provide the user with an image that reflects the orientation of the information processing apparatus in the real space. Further, according to this aspect, since the amount of change in the first angle: the amount of change in the second angle=2:1, the virtual image plane on which the virtual image of the screen is positioned in the physical space remains orthogonal to the measurement reference plane. maintained. Therefore, the user viewing the virtual image at a line-of-sight angle with respect to the measurement reference plane can accurately experience the effects of different line-of-sight angles without feeling the tilt of the virtual image plane, regardless of the positions of the mobile information terminal and the half mirror. can.

A second aspect is the interactive object display program according to the first aspect, wherein the second correspondence is that the line-of-sight angle with respect to the imaging reference plane is equal to the first angle.

According to the second aspect, since the line-of-sight angle is equal to the first angle, the direction parallel to the first plane along the screen is the optimum angle for viewing the virtual image. As a result, the user's motion to optimally view the virtual image is likely to be induced, and the user can be naturally guided to a position where it is difficult to view the screen directly.

A third aspect is that the second correspondence relationship is that the line-of-sight angle φ radians with respect to the imaging reference plane is a value obtained by adding the correction value C radians to the first angle θA radians, and the correction value C radians is 0 The interactive object display program of the first aspect greater than radians and less than θB radians.

According to the third aspect, the line-of-sight angle is a value obtained by adding the correction value C to the first angle. is the optimum angle for As a result, the user's motion to optimally view the virtual image is likely to be induced, and the user can be naturally guided to a position where the screen cannot be viewed directly.

In a fourth aspect, the support base includes a support-side connection portion for transmitting electrical signals, the information processing device includes a device-side connection portion for transmitting electrical signals, and the information processing device is mounted on the mounting portion, the supporting side connecting portion is arranged at a position where the supporting side connecting portion is connected to the device side connecting portion, and when the interactive object display program is executed by the processor, the starting The information processing device is caused to execute processing, and the start processing includes detecting whether or not the support-side connection portion is connected to the device-side connection portion, and detecting whether or not the support-side connection portion is connected to the device-side connection portion. activating an interactive object display process in response to a change from a connected state to a connected state in which the support-side connection unit is connected to the device-side connection unit. is an interactive object display program according to any one of the aspects of

According to the fourth aspect, when the information processing device is mounted on the mounting portion, the supporting side connecting portion is connected to the device side connecting portion and the interactive object display process is started. A display interactive object process can be initiated.

In a fifth aspect, the interactable object display device includes an external wireless power transmission device capable of wirelessly transmitting power, the information processing device includes an internal wireless power transmission device capable of wirelessly receiving power, and information processing is performed. The external wireless power transmission device is arranged at a position where power can be transmitted between the external wireless power transmission device and the internal wireless power transmission device when the device is mounted on the mounting part, and the interactive object display program is installed. , when executed by the processor, causes the information processing device to execute start-up processing, and the start-up processing detects whether power can be transmitted between the external wireless power transmission device and the internal wireless power transmission device. and from a non-transmission state in which power transmission is impossible between the external wireless power transmission device and the internal wireless power transmission device, power can be transmitted between the external wireless power transmission device and the internal wireless power transmission device. and activating an interactive object display process in response to a change in transmission state.

According to the fifth aspect, when the information processing device is mounted on the mounting portion, power can be transmitted between the external wireless power transmission device and the internal wireless power transmission device, and as a result, the interactive object display process is started. Therefore, the user can start interactive object display processing with a simple operation.

In a sixth aspect, the information processing device includes a voice input unit capable of inputting voice from the outside, and the interactable object display processing includes changing an image based on the voice. is an interactive object display program according to any one of the aspects of

According to the sixth aspect, since the image changes based on the voice, it is possible to diversify the display.

A seventh aspect is an interactive object display device for displaying a three-dimensional interactive object in a virtual space in a real space for interaction with a user, wherein the interactive object display device comprises an information processing device, information a support base capable of detachably supporting a processing device, wherein the information processing device includes an orientation detection section, a display section including a screen for displaying an image, and a processor capable of executing an interactive object display program. , wherein the support base comprises a half mirror having a reflective surface capable of at least partially reflecting the screen, a mounting portion capable of detachably mounting the information processing device with the screen facing the reflective surface, and a mounting portion. an adjustment mechanism capable of adjusting an angle defined between the screen of the information processing device mounted in the device and the reflecting surface of the half mirror, wherein the first angle and the second angle have a first correspondence relationship wherein the first angle is the angle between a virtual measurement reference plane in real space and the first plane extending along the screen, and the second is the angle between the measurement reference plane and a second plane extending along the reflective surface, and the first correspondence is that when the first angle is θA radians, the second angle θB radians is (−π/4+θA/2) radians, the orientation detection unit can detect angle information corresponding to the first angle, and the interactive object display program is executed by the processor, The information processing apparatus is caused to execute interactive object display processing, and the interactive object display processing converts the line-of-sight angle between the line-of-sight direction suitable for viewing the virtual image of the image formed by the reflecting surface and the measurement reference plane into an angle making a decision based on the information and the second correspondence; creating an image of the interactive object captured in a virtual space from a shooting direction tilted by the line-of-sight angle with respect to the shooting reference plane; and creating the created image. on a screen, wherein a second correspondence is defined based on the first correspondence to represent the relationship between the first angle and the line-of-sight angle is.

According to the present invention, an appropriate line-of-sight angle for viewing a virtual image of a three-dimensional interactive object in a virtual space is automatically determined with a simple configuration according to the positions of the information processing device and the half-mirror. image can be displayed.

Fig. 2 is a left side view of an embodiment interactive object display in a first state; Figure 2 is a front view of the interactive object display of Figure 1 in a first state; 2 is a plan view of the support base of FIG. 1 in a first state; FIG. Figure 2 is a left side view of the interactive object display of Figure 1 in a second state; 2 is a schematic block diagram illustrating an electrical configuration of the information processing device of FIG. 1; FIG. FIG. 4 is a correspondence diagram for explaining the relationship between the physical space and the virtual space in the first state; FIG. 11 is a correspondence diagram for explaining the relationship between the physical space and the virtual space in the second state; FIG. 2 is a flowchart of interactive object display processing executed by the information processing apparatus of FIG. 1; FIG. 11 is a correspondence diagram for explaining the relationship between the physical space and the virtual space in the first state of the modified example;

(Overall configuration)
FIG. 1 is a left side view of an embodiment interactive object display 100 in a first state. FIG. 2 is a front view of the interactive object display device 100 in the first state. As illustrated in FIG. 2, the interactive object display device 100 displays a three-dimensional interactive object 105 in virtual space in real space.

The virtual space is, for example, created by arithmetic processing to resemble reality, or created by arithmetic processing as a world different from reality. Interactable objects 105 exist in a virtual space and include, but are not limited to, characters resembling real humans or other living things, objects resembling real robots or other machines, spherical objects, real Objects that do not exist in the world can be mentioned. In this embodiment, the interactable object 105 is a three-dimensional human-like character that can interact with the user in response to real-world sounds. Interaction means that some kind of response processing is performed in response to input from the user. The response process is, for example, a natural language conversation.

First, as shown in FIG. 1, the interactive object display device 100 includes an information processing device 101 and a support base 102 that detachably supports the information processing device 101 . Examples of the information processing device 101 include, but are not limited to, smart phones, tablet computers, and mobile phones. The information processing apparatus 101 of the present embodiment is a thin, rectangular parallelepiped smart phone. For example, a usage example is conceivable in which a small portable information processing device 101 such as a smartphone is easily mounted on the support base 102 when the user returns home. For the purpose of allowing the user to easily use the interactive object display device 100 at home, the information processing device 101 is desirably of a size that can be held with one hand, for example, like a smart phone.

The information processing apparatus 101 includes a display unit 110 including a screen 111 for displaying images. The display unit 110 is, for example, a liquid crystal display or an organic EL display. The screen 111 is planar, and is horizontally mounted on the support base 102 while facing downward in the first state. The information processing device 101 has a device-side connector 112, which is a USB-C terminal for transmitting electrical signals, at its rear end. Examples of the device-side connection unit 112 include a USB-C terminal, other USB standard terminals, LIGHTNING (registered trademark) terminals, and other electrical terminals. The device-side connection portion 112 may be any connection member that can electrically recognize the connection relationship with the external device. The electrical configuration of the information processing device 101 will be described later in detail.

FIG. 3 is a plan view of the support base 102 in the first state. FIG. 4 is a left side view of the interactive object display device 100 in the second state. Interactable object display 100 can assume a variety of successive states, including the first state of FIG. 1 and the second state of FIG. 4, as described below. First, the configuration of the interactive object display device 100 in the first state will be described with reference to FIGS. 1 to 3. FIG. Directions are used to describe relative positions and do not limit directions of actual use.

As shown in FIG. 2, the configuration of the support base 102 is generally bilaterally symmetrical, and the configuration on the left side will be mainly described first. Components arranged symmetrically to the left and right are given the same number. As shown in FIG. 1 , the support base 102 includes a horizontal fixing portion 120 that extends linearly in the front-rear direction at the lowest portion of the support base 102 and a vertical fixing portion 120 extending vertically from the vicinity of the rear end of the horizontal fixing portion 120 . and a vertical fixing portion 121 linearly extending upward. The horizontal fixing portion 120 includes a plurality of stopper portions 122 protruding upward between the front end of the horizontal fixing portion 120 and the vertical fixing portion 121 .

The support base 102 has an adjustment mechanism 123 near the upper end of the vertical fixing portion 121 . As shown in FIG. 2, the adjusting mechanism 123 is located on the right side of the vertical fixing portion 121 and has a first gear 124, a second gear 125, and a third gear 126 in order from above. Gear teeth are omitted except in FIG. As shown in FIG. 1, the first gear 124, the second gear 125, and the third gear 126 are mounted on the vertical fixing portion 121 so as to be rotatable around a rotation shaft extending in the left-right direction. .

As shown in FIG. 1, the second gear 125 meshes with the rotation of the first gear 124 and the rotation of the third gear 126 so as to mutually transmit the rotation. The number of teeth of the first gear 124 : the number of teeth of the third gear 126 = 1:2. In another example, the rotation of the first gear 124 and the third gear 126 may be directly meshed. There may be multiple gears between the first gear 124 and the third gear 126 . The arrangement of the gears of the adjustment mechanism 123 is not limited to that shown. The configuration of the adjustment mechanism 123 is not limited to using gears.

The support base 102 has a mounting portion 130 near the upper end of the vertical fixing portion 121 . The mounting portion 130 includes a mounting beam 131 linearly extending in the front-rear direction. A first gear 124 is positioned between the vicinity of the rear end of the mounting beam 131 and the vertical fixing portion 121 . As shown in FIG. 2 , the mounting beam 131 is fixed to the right side of the first gear 124 and rotatably supported by the vertical fixing portion 121 . Support platform 102 also includes mirror beam 127 . A third gear 126 is positioned between the vicinity of the rear end of the mirror beam 127 and the vertical fixing portion 121 . The mirror beam 127 is fixed to the right side of the third gear 126 and rotatably supported by the vertical fixing portion 121 .

As shown in FIG. 1, the support platform 102 has a stop bar 128 to the left of the mounting beam 131 . The upper end of the stop bar 128 and the vicinity of the longitudinal center of the mounting beam 131 are rotatably connected to each other. The lower end of the stop bar 128 is configured to be engaged with the stop portion 122 of the horizontal fixing portion 120 with the lower end of the stop bar 128 positioned in front of the upper end of the stop bar 128 . That is, by stopping the movement of the stopping bar 128 by the stopping portion 122, the mounting portion 130 is fixed at the position shown in FIG. 1 and the position shown in FIG. 4, for example. Mounting portion 130 may be secured by other configurations.

As another example, there may be more or less than three stops 122 . The method for fixing the mounting portion 130 is not limited to the method of locking the stop bar 128 with the stop portion 122 . For example, the lateral fixing part 120 may not have the protrusion-shaped stop part 122, and may be fixed at any position by various means such as locking with other uneven shapes, fixing with screws, clamping with clips, etc. . The angle at which the mounting portion 130 is fixed may be discrete or continuous. For example, by using a non-slip material such as rubber on either the lower end of the stop bar 128 or the upper surface of the horizontal fixing part 120, the stop bar 128 can be continuously moved to any position by friction or other mechanism. It may be fixable, resulting in the mount 130 being fixable at any continuous angle.

As shown in FIG. 2, the support base 102 is bilaterally symmetrical with the left lateral fixed part 120, the vertical fixed part 121, the adjustment mechanism 123, the mounting beam 131, and the mirror beam 127 (collectively referred to as the left member). It comprises a right lateral fixed part 120, a vertical fixed part 121, an adjustment mechanism 123, a mounting beam 131 and a mirror beam 127 (collectively referred to as a right member) arranged. Both the left member and the right member are made of a hard material such as hard resin.

As shown in FIG. 2, the mounting portion 130 includes a plate-like cushioning material 132 fixed on two mounting beams 131 on the left and right sides. As shown in FIG. 3, the rectangular parallelepiped cushioning material 132 has an opening 135 penetrating vertically. As shown in FIG. 1, the top surface of the cushioning material 132 is horizontal. When the information processing apparatus 101 is placed on the upper surface of the cushioning material 132 , all or part of the screen 111 can be viewed from below through the opening 135 . The cushioning material 132 is made of a material that is softer than the hard material described above, for example, a material that is soft enough to prevent the screen 111 from shaking and that does not damage the screen 111, such as polystyrene or urethane. The cushioning material 132 may be made of other materials.

As shown in FIG. 2, the mounting portion 130 has two side walls 133 and a rear wall 134 extending upward from the top surface of the cushioning material 132 . The side walls 133 extend in the front-rear direction, an opening 135 is positioned between the side walls 133 in the left-right direction, and the side walls 133 fix the information processing apparatus 101 in the left-right direction. A rear wall 134 positioned behind the opening 135 extends in the left-right direction to prevent the information processing apparatus 101 from shifting rearward. As shown in FIG. 2, the mounting portion 130 is positioned between the two left and right first gears 124 .

As shown in FIG. 2, the support base 102 has a plate-like half mirror 136 fixed on the two left and right mirror beams 127 . As shown in FIG. 1, the half mirror 136 has a reflecting surface 137 whose normal direction is the direction between the upward direction and the forward direction. The normal direction of the screen 111 mounted on the mounting portion 130 is downward. Reflective surface 137 is capable of at least partially reflecting light from screen 111, at least when viewed from the front. Also, the half mirror 136 partially transmits light.

A portion near the screen 111 and likely to be reflected in the half mirror 136, such as the cushioning material 132 and the mounting beam 131, is preferably matte black. By adopting a matte black color, even if an image other than the image of the screen 111 is reflected on the half mirror 136, it is difficult for the user to visually recognize it. Therefore, the user can easily concentrate on the image projected on the real world.

Support base 102 includes a connecting cable 140 fixed at one end to rear wall 134 . The connection cable 140 has a support-side connection portion 141 that penetrates the rear wall 134 and protrudes forward. The support-side connection portion 141 is a member that transmits electrical signals and is made according to a standard corresponding to the device-side connection portion 112 . The support-side connection portion 141 is not limited to the USB-C terminal. In this embodiment, the convex support-side connection portion 141 can be inserted into the concave device-side connection portion 112 .

As shown in FIG. 1 , the upper surface of the cushioning material 132 of the mounting portion 130 can detachably mount the information processing apparatus 101 with the screen 111 facing the reflecting surface 137 through the opening 135 . The support side connection portion 141 is arranged at a position where the information processing device 101 is connected to the device side connection portion 112 when the information processing device 101 is mounted on the upper surface of the cushioning material 132 of the mounting portion 130 . The support-side connecting portion 141 functions to position the information processing device 101 .

(Information processing device)
FIG. 5 is a schematic block diagram for explaining the electrical configuration of the information processing device 101. As shown in FIG. The information processing apparatus 101 includes a communication device 150, a storage device 151, a processor 152, an attitude detection unit 153, an audio input unit 154, an audio output unit 155, an interface 156, a display input unit 157, a storage battery 159, and an internal wireless power transmission device 162. and the device-side connection portion 112 described above.

The communication device 150 is configured to be able to communicate with the external network 103 and can communicate with various external computers 104 connected to the network 103 . The external computer 104 is, for example, a server that provides a program download site, a search site, and the like. Communication between processor 152 and the outside world may be performed via device-side connection 112 or other mechanisms.

As a communication method of the communication device 150, for example, wired communication means (for example, optical fiber, telephone line, power line), wireless communication means (for example, wireless LAN, Bluetooth (registered trademark), mobile communication line, infrared communication, short-range wireless communication), but are not limited to these. Examples of the network 103 include various communication paths such as the Internet, an intranet, a local area network (LAN), a wide area network (WAN), a wired communication network, a wireless communication network, a television communication network, a satellite communication network, and acoustic communication. but not limited to these.

The storage device 151 stores an interactive object display program 160 and is configured to be able to store necessary information as needed. Types of storage device 151 include, but are not limited to, read-only memory, read-write memory, volatile memory, non-volatile memory, tangible non-transitory storage media, and combinations thereof.

Processor 152 reads interactive object display program 160 from storage device 151 and executes it. When the interactive object display program 160 is executed by the processor 152, the processor 152 functions as an interactive object display unit 161, and the interactive object display unit 161 causes the information processing apparatus 101 to perform startup processing and interactive object display processing, which will be described later. to run. Interactive object display program 160 may be downloaded from external computer 104 .

The orientation detection unit 153 can detect the tilt of the information processing device 101, that is, angle information corresponding to a first angle θA (FIG. 6) described later. Examples of the orientation detection unit 153 include a gyro and an acceleration sensor. The voice input unit 154 has a function of inputting voice from the outside, and is, for example, a microphone. Input sounds include, for example, human voices, sounds made by objects, and other sounds. The audio output unit 155 has a function of outputting audio to the outside, and is, for example, a speaker. Examples of sounds to be output include recorded human voices, synthesized voices resembling human voices, and other synthesized voices.

The interface 156 mediates electrical signals between the processor 152 and the device-side connector 112 . For example, it is possible to electrically detect whether or not the support-side connection portion 141 of the support-side cable is connected to the device-side connection portion 112 described above. The interface 156 processes electric signals between the device-side connection unit 112 and the support-side connection unit 141, for example, so that communication according to the USB-C standard is performed. As a function of the interface 156, for example, a configuration for charging the storage battery 159 with power input from the device-side connection unit 112 may be provided. The interface 156 may function as the audio input unit 154 or the audio output unit 155 by connecting an external audio input device such as a microphone or an audio output device such as a speaker to the device-side connection unit 112 .

Display input unit 157 is a touch panel, and includes display unit 110 described above for displaying an image, and input unit 158 that receives input from the outside by touch operation. The display unit 110 has a screen 111 which is a liquid crystal panel, and displays an image on the screen 111 according to instructions from the processor 152 . Note that the input is not limited to the touch operation on the input unit 158, and may be input by other means such as a keyboard, mouse, voice, trackpad, and button. The screen 111 is not limited to a liquid crystal panel, and may be another display device such as an organic EL display.

The interactive object display device 100 further comprises an external wireless power transmission device 163 capable of wirelessly transmitting power. The external wireless power transmission device 163 is arranged at a position where power can be transmitted between the external wireless power transmission device 163 and the internal wireless power transmission device 162 when the information processing device 101 is mounted on the mounting section 130 . . For example, the external wireless power transmission device 163 may be mounted, for example, on the support base 102 or may be installed separately from the support base 102 . Internal wireless power transmission device 162 receives power wirelessly transmitted from external wireless power transmission device 163 and charges storage battery 159 . Also, the internal wireless power transmission device 162 may transmit the electrical energy charged in the storage battery 159 to the outside. Power transmission between the internal wireless power transmission device 162 and the external wireless power transmission device 163 is performed based on, for example, the Qi standard or other standards.

(motion)
With the above configuration, the adjusting mechanism 123 shown in FIG. 1 can adjust the angle defined between the screen 111 of the information processing device 101 mounted on the mounting portion 130 and the reflecting surface 137 of the half mirror 136 . FIG. 6 is a correspondence diagram for explaining the relationship between the first state (FIG. 1) in the real space and the line-of-sight angle of the interactive object 105 in the virtual space in the first state (FIG. 1). FIG. 7 is a correspondence diagram for explaining the relationship between the second state (FIG. 4) in the real space and the line-of-sight angle of the interactive object 105 in the virtual space in the second state (FIG. 4). First, the operation in the physical space of the interactive object display device 100 of FIG. 1 will be described. In the description of this specification, the unit of angle is radian.

As shown in FIG. 6, it is assumed that a virtual measurement reference plane 170, a first plane 171, and a second plane 172 exist in the physical space. Measurement reference plane 170 is horizontal and extends infinitely. The first plane 171 extends infinitely along the screen 111 ( FIG. 1 ) of the information processing device 101 . The second plane 172 extends infinitely along the reflective surface 137 (FIG. 1) of the half mirror 136 (FIG. 1). The measurement reference plane 170 , the first plane 171 and the second plane 172 intersect along a virtual common intersection line 173 . That is, the first plane 171 and the second plane 172 assume various positions by rotating the measurement reference plane 170 about the line of intersection 173 according to the movement of the screen 111 and the reflective surface 137 .

A first angle θA is formed between the measurement reference plane 170 and the first plane 171 . A second angle θ B is formed between measurement reference plane 170 and second plane 172 . In this description, regarding the angle with respect to the measurement reference plane 170, the clockwise rotation along the plane orthogonal to the measurement reference plane 170 is positive. The adjustment mechanism 123 (FIG. 1) is configured such that the first angle .theta.A and the second angle .theta.B change based on the first correspondence. The first correspondence is that when the first angle is θA, the second angle θB is (−π/4+θA/2). This relationship is realized by the number of teeth of the first gear 124 (FIG. 1):the number of teeth of the third gear 126 (FIG. 1)=1:2. The adjusting mechanism 123 is desirably configured so that the first angle θA can be changed in at least a part of the range of −π/2≦θA<π/2. <π/2 can be changed.

The viewing direction 174 is the direction in which the user views the virtual image through the half mirror 136 (FIG. 1). If the screen 111 (FIG. 1) is directly in the field of view of the user, the user will not feel the reality of the virtual image. It is desirable to In this example, it is assumed that the viewing direction 174 is always parallel to the first plane 171 along a plane perpendicular to the measurement reference plane 170 . The reflected light path 178 shows the path that the image in the first plane 171 reflects off the second plane 172 and travels parallel to the viewing direction 174 toward the user's eyes.

As shown in FIG. 6, in the first state (FIG. 1), the first plane 171 is horizontal, that is, parallel to the measurement reference plane 170, so the first angle .theta.A=0. At the same time, the second angle θB=-π/4. The image displayed on screen 111 (FIG. 1) appears to lie in virtual image plane 175 . The first correspondence causes virtual image plane 175 to appear orthogonal to measurement reference plane 170 along intersection line 173 . That is, if the measurement reference plane 170 is horizontal, the virtual image will be perpendicular to the ground. FIG. 2 shows an interactive object 105, which is a character, as an example of an image viewed from the front.

As shown in FIG. 7, in the second state (FIG. 4), the first angle θA between the first plane 171 and the measurement reference plane 170 is θA=π/6. At the same time, the second angle .theta.B=(-.pi./4+.theta.A/2)=-.pi./6. When viewed from a viewing direction 174 parallel to the first plane 171 , the image appears to lie in a virtual image plane 175 . If the measurement reference plane 170 is horizontal, the virtual image remains perpendicular to the ground. According to the first correspondence, the amount of change in the second angle θB from the first state is half the amount of change in the first angle θA. In other words, the angle between the second plane 172 and the virtual image plane 175 is equal to the angle between the first plane 171 and the second plane 172, so the first angle θA and the second angle θB, the virtual image plane 175 is maintained vertical regardless of the first angle θA.

In other words, when the horizontal fixing portion 120 and the vertical fixing portion 121 of FIG. , the measurement reference plane 170 and the virtual image plane 175 are automatically determined. Note that, according to the configuration of FIG. 1, the line of intersection 173 is slightly translated in accordance with the first angle θA. However, since measurement reference plane 170 is only slightly translated up and down and virtual image plane 175 is only slightly translated back and forth in accordance with the movement of intersection line 173, first angle θA does not change. However, the virtual image plane 175 does not tilt. Adjustment mechanism 123 may be configured to fix intersection line 173 when first plane 171 is moved.

(start processing)
First, start-up processing that the interactive object display unit 161 shown in FIG. 5 causes the information processing apparatus 101 to execute will be described. The interactive object display unit 161 determines whether the information processing device 101 is in the connected state or the non-connected state. The connected state is a state in which the support-side connection portion 141 is connected to the device-side connection portion 112 . The non-connected state is a state in which the support-side connection portion 141 is not connected to the device-side connection portion 112 . The interactive object display unit 161 starts interactive object display processing in response to the change of the information processing apparatus 101 from the disconnected state to the connected state.

In another example, the interactive object display unit 161 starts interactive object display processing when the information processing apparatus 101 changes from the disconnected state to the connected state and other conditions are satisfied. . Another condition is, for example, that the information processing apparatus 101 is within a predetermined angle range with respect to the horizontal by the posture detection unit 153 (for example, the screen 111 is directed downward from the vertical). Other conditions include, for example, the receipt of signals from other sensors not shown and the initiation of charging.

In another example, interactive object display 161 determines whether internal wireless power transfer device 162 is in a transmitting state or a non-transmitting state. The transmission state is a state in which power is being transmitted between the internal wireless power transmission device 162 and the external wireless power transmission device 163, or a state in which power transmission is possible. It refers to a state in which communication required for power transmission is established with the external wireless power transmission device 163 . The non-transmission state is a state in which power is not transmitted between the internal wireless power transmission device 162 and the external wireless power transmission device 163, or a state in which power transmission is impossible, for example, the internal wireless power transmission device. 162 and the external wireless power transmission device 163 in which communication required for power transmission has not been established. The interactive object display unit 161 initiates interactive object display processing in response to the internal wireless power transmission device 162 changing from the non-transmitting state to the transmitting state.

In another example, the interactive object display unit 161 performs interactive object display processing when the internal wireless power transmission device 162 changes from the non-transmitting state to the transmitting state and other conditions are satisfied. Start. Another condition is, for example, that the information processing apparatus 101 is within a predetermined angle range with respect to the horizontal by the posture detection unit 153 (for example, the screen 111 is directed downward from the vertical). Other conditions include, for example, the receipt of signals from other sensors not shown and the initiation of charging.

(Interactive object display processing)
Next, the interactive object display processing that the interactive object display unit 161 (FIG. 5) causes the information processing apparatus 101 (FIG. 5) to execute will be described with reference to the flowchart of FIG. 8 and FIG. Examples of the cases where the interactive object display process is executed include cases where it is started by the above starting process, cases where it is started by a user's operation, cases where it is started by other triggers, and the like. The interactive object display process is repeatedly executed at an appropriate timing. Appropriate timing is a fixed time interval in one example, timing at which a change in the first angle θA is detected based on angle information detected by the posture detection unit 153 (FIG. 5) in another example, and others. at the time of

Interactable object display section 161 (FIG. 5) first acquires angle information corresponding to first angle θA from orientation detection section 153 (FIG. 5) in step 190 . The angle information does not have to be the first angle θA itself. For example, it may be information that allows the first angle θA to be calculated by calculation. In this embodiment, assume that the measurement reference plane 170 is horizontal and the angle information represents a first angle θA with respect to the horizontal.

Interactable object display unit 161 (FIG. 5) then determines line-of-sight angle φ in step 191 based on the acquired angle information and a second correspondence described later.

The lower part of FIG. 6 is a diagram for explaining the line-of-sight angle φ of the virtual space in the first state shown in FIG. 1 . A shooting reference plane 176 is set in the virtual space. The imaging reference plane 176 is a plane corresponding to the measurement reference plane 170 in the physical space. In this embodiment, it represents a horizontal plane in the virtual space. The line-of-sight angle φ is measured in a plane orthogonal to the imaging reference plane 176 with the clockwise direction in FIG. 6 as the positive direction. The line-of-sight angle φ is the angle of the photographing direction 177 with respect to the photographing reference plane 176 for pseudo-photographing the interactive object 105 .

The second correspondence is defined based on the first correspondence to express the relationship between the first angle θA and the line-of-sight angle φ. Based on the first correspondence relationship, the second angle θB is automatically determined when the first angle θA is determined, and as a result, the virtual image plane 175 does not rotate regardless of the first angle θA. It means that you know That is, the line-of-sight angle φ can be determined without considering the rotation of the virtual image plane 175 . In this example, the second correspondence is that the line-of-sight angle φ with respect to the imaging reference plane 176 is equal to the first angle θA. Therefore, as shown in FIG. 6, the line-of-sight angle φ=0 in the first state. As shown in FIG. 7, the line-of-sight angle φ=π/6 in the second state.

Interactable object display unit 161 (FIG. 5) then uses the determined line-of-sight angle φ to create an image of interactable object 105 in the virtual space at step 192 . The image created is a two-dimensional image. Next, at step 193, interactive object display unit 161 (FIG. 5) displays the created image on screen 111 (FIG. 5). Note that the created image may be a still image or a moving image. In this embodiment, a three-dimensional human-like character moves its body like a real human being. In order to make the user feel the effect of changing the viewing angle φ, the interactable object 105 preferably has different appearances depending on the viewing angle φ.

FIG. 9 is a correspondence diagram for explaining the relationship between the physical space and the virtual space in the first state of the modified example. Differences from FIG. 6 will be described. The second correspondence relationship of the modified example is that the line-of-sight angle φ with respect to the imaging reference plane 176 is the sum of the first angle θA and a correction value C within a range greater than 0 and less than θB. As a result, when the user views the virtual image plane 175 in the real space, if the angle of the line-of-sight direction 174 is θA+C, the user can appropriately view the image at the same angle as in the virtual space. When the first plane 171 moves to angles other than those shown in FIG. 9, each angle similarly results in a line-of-sight angle φ of θA+C.

In another example, if interactive object display 100 (FIG. 1) is installed such that measurement reference plane 170 (FIG. 6) is not horizontal (eg, screen 111 (FIG. 1) is not horizontal in the first state). If not), the line-of-sight angle φ in the virtual space may be modified so that an optimal image is displayed as viewed from the line-of-sight angle φ in the real space. For example, the line-of-sight angle φ with respect to the imaging reference plane 176 (FIG. 6) is corrected to a value obtained by adding the angle of inclination of the measurement reference plane 170 (FIG. 6) to the horizontal to the line-of-sight angle φ in real space.

With reference to FIG. 5, the change of the image based on the sound in the interactive object display process will be described. First, the user inputs voice to the information processing apparatus 101 via the voice input unit 154 . The interactive object display unit 161 performs voice recognition processing, changes an image based on the input voice, and performs processing corresponding to the voice. In other examples, input may be provided by means other than voice, eg, similar to input unit 158 described above.

For example, when a greeting such as "Hello" is input, a response greeting such as "Hello" is output to the voice output unit 155, and a moving image of the character on the screen 111 bowing is displayed on the screen 111.例文帳に追加For example, when the question "What is ?" is input, an external search site is accessed and searched, and the search result is output as a voice such as "It is" from the voice output unit 155, and the character thinks. A moving image of the action is displayed on the screen 111 . For example, when a request such as "Play music" is input, a music reproduction program is started to reproduce some music, and a moving image of a character moving in time with the music is displayed on the screen 111 .

When the user views the image reflected in the half mirror 136 as shown in FIG. 2, the interactive object 105 appears to exist behind the half mirror 136 . Also, since the interactable object 105 and the object in the real space can be seen at the same time, it appears as if the interactable object 105 exists in the real space. For example, a humanoid interactable object 105 appears to be standing on a desk. For example, the interactive object display unit 161 (FIG. 5) performs image processing to move the interactive object 105, so that the interactive object 105 appears to move in the real space.

In the interactive object display device 100 of FIG. 1, the vertical fixing portion 121 and the horizontal fixing portion 120 of the support base 102 (that is, the members serving as the reference for movement of the mounting portion 130 and the half mirror 136) are stationary with respect to an external object. (i.e., situations in which measurement reference plane 170 of FIG. 6 is not rotated). By using the measurement reference plane 170 in a situation where it does not rotate regardless of the user's operation, it is possible to prevent the interactive object 105 from appearing unnatural due to changes in the image regardless of the movement of the mounting unit 130 caused by the user's operation. be able to.

(summary)
According to this embodiment, the adjusting mechanism 123 is configured such that the first angle θA of the screen 111 and the second angle θB of the reflecting surface 137 are changed based on the first correspondence relationship, and the second is defined based on the first correspondence so as to express the relationship between the first angle θA and the line-of-sight angle φ. Then, the line-of-sight angle φ is determined based on the angle information corresponding to the first angle θA and the second correspondence. That is, even if the second angle θB of the reflecting surface 137 is not measured when determining the line-of-sight angle φ, the positional relationship between the screen 111 and the reflecting surface 137 is indirectly reflected by the second correspondence relationship. Furthermore, it reflects that the virtual image plane does not rotate. As a result, an appropriate line-of-sight angle φ for viewing the virtual image of the three-dimensional interactable object 105 in the virtual space can be automatically determined with a simple configuration according to the positions of the information processing device 101 and the half mirror 136, enabling interaction. An image of the object 105 can be displayed.

Further, since the line-of-sight angle φ is determined based on the angle information, unlike the case where the angle information is not used, it is possible to provide the user with an image reflecting the posture of the information processing apparatus 101 in the real space. Also, since the amount of change in the first angle θA: the amount of change in the second angle θB=2:1, the virtual image plane 175 on which the virtual image of the screen 111 is positioned in the physical space is maintained perpendicular to the measurement reference plane 170. be done. Therefore, the user viewing the virtual image at the line-of-sight angle φ with respect to the measurement reference plane 170 can enjoy the effect of the difference in the line-of-sight angle φ without feeling the tilt of the virtual image plane 175 regardless of the positions of the information processing device 101 and the half mirror 136 . You can feel it accurately.

According to this embodiment, since the line-of-sight angle φ is equal to the first angle θA, the direction parallel to the first plane 171 along the screen 111 is the optimum angle for viewing the virtual image. As a result, the user's motion to optimally view the virtual image is likely to be induced, and the user can be naturally guided to a position where it is difficult to view the screen 111 directly.

According to this embodiment, since the line-of-sight angle φ is a value obtained by adding the correction value C to the first angle θA, the direction forming an angle larger than the first angle θA, that is, the direction viewed from the back side of the screen 111 is the virtual image. the best angle for viewing. As a result, the user's motion to optimally view the virtual image is likely to be induced, and the user can be naturally guided to a position where the screen 111 cannot be directly viewed.

According to the present embodiment, when the information processing apparatus 101 is mounted on the mounting section 130, the support-side connection section 141 is connected to the device-side connection section 112 and the interactive object display process is started. The interactive object display process can be started by a simple operation.

According to this embodiment, when the information processing device 101 is mounted on the mounting unit 130, power can be transmitted between the external wireless power transmission device 163 and the internal wireless power transmission device 162, resulting in interactive object display processing. is started, the user can start interactive object display processing with a simple operation.

According to this embodiment, since the image changes based on the sound, it is possible to diversify the display.

DESCRIPTION OF SYMBOLS 100... Interactive object display apparatus, 101... Information processing apparatus, 102... Support stand,
103... network, 104... external computer, 105... interactable object,
DESCRIPTION OF SYMBOLS 110... Display part 111... Screen 112... Device side connection part 120... Lateral fixed part,
121...vertical fixing part, 122...stopping part, 123...adjusting mechanism, 124...first gear,
125... Second gear, 126... Third gear, 127... Mirror beam, 128... Stop bar,
DESCRIPTION OF SYMBOLS 130... Mounting part, 131... Mounting beam, 132... Cushioning material, 133... Side wall, 134... Rear wall,
135... opening, 136... half mirror, 137... reflective surface, 140... connection cable,
141 Support side connection unit 150 Communication device 151 Storage device 152 Processor
153... Posture detection unit, 154... Audio input unit, 155... Audio output unit,
156... interface, 157... display input unit, 158... input unit, 159... storage battery,
160 ... interactive object display program, 161 ... interactive object display unit,
162... Internal wireless power transmission device, 163... External wireless power transmission device,
170... measurement reference plane, 171... first plane, 172... second plane, 173... line of intersection,
174...line-of-sight direction 175...virtual image plane 176...shooting reference plane 177...shooting direction 178...reflected optical path

Claims (7)

An interactive object display program executed in an interactive object display device for displaying a three-dimensional interactive object in a virtual space in a real space for interaction with a user, comprising:
The interactable object display device comprises:
an information processing device;
a support base capable of detachably supporting the information processing device;
with
The information processing device
an attitude detection unit;
a display unit including a screen for displaying images;
a processor capable of executing the interactive object display program;
with
The support base
a half mirror having a reflecting surface capable of at least partially reflecting the screen;
a mounting unit on which the information processing device can be detachably mounted with the screen facing the reflecting surface;
an adjusting mechanism capable of adjusting an angle defined between the screen of the information processing device mounted on the mounting portion and the reflecting surface of the half mirror;
with
The adjustment mechanism is configured such that the first angle and the second angle change based on the first correspondence,
the first angle is an angle between a virtual measurement reference plane in the real space and a first plane extending along the screen;
wherein the second angle is the angle between the measurement reference plane and a second plane extending along the reflective surface;
the first correspondence is that when the first angle is θA radians, the second angle θB radians is (−π/4+θA/2) radians;
the posture detection unit is capable of detecting angle information corresponding to the first angle;
causing the information processing device to execute an interactive object display process when the interactive object display program is executed by the processor;
The interactable object display process includes:
determining a line-of-sight angle between a line-of-sight direction suitable for viewing a virtual image of the image formed by the reflective surface and the measurement reference plane based on the angle information and a second correspondence;
creating the image of the interactive object photographed in the virtual space from a photographing direction inclined by the line-of-sight angle with respect to the photographing reference plane set in the virtual space as a plane corresponding to the measurement reference plane in the physical space; and
displaying the created image on the screen;
including
wherein the second correspondence is defined based on the first correspondence to represent the relationship between the first angle and the line-of-sight angle;
An interactive object display program. wherein the second correspondence relationship is that the line-of-sight angle with respect to the imaging reference plane is equal to the first angle;
An interactive object display program according to claim 1. the second correspondence is that the line-of-sight angle φ radians with respect to the imaging reference plane is a value obtained by adding a correction value C radians to the first angle θA radians;
the correction value C radians is greater than 0 radians and less than θB radians;
An interactive object display program according to claim 1. The support base has a support-side connection for transmitting an electrical signal,
The information processing device comprises a device-side connection unit for transmitting an electrical signal,
The support-side connection portion is arranged at a position where the support-side connection portion is connected to the device-side connection portion when the information processing device is mounted on the mounting portion,
causing the information processing device to perform a startup process when the interactive object display program is executed by the processor;
The starting process is
detecting whether or not the support-side connection portion is connected to the device-side connection portion;
the interaction in response to a change from a non-connected state in which the supporting-side connecting portion is not connected to the device-side connecting portion to a connected state in which the supporting-side connecting portion is connected to the device-side connecting portion; starting a possible object display process;
including,
An interactive object display program according to any one of claims 1 to 3. wherein the interactive object display device comprises an external wireless power transmission device capable of transmitting power wirelessly;
The information processing device includes an internal wireless power transmission device capable of wirelessly receiving power,
The external wireless power transmission device is arranged at a position where power can be transmitted between the external wireless power transmission device and the internal wireless power transmission device when the information processing device is mounted on the mounting portion. ,
causing the information processing device to perform a startup process when the interactive object display program is executed by the processor;
The starting process is
detecting whether power can be transmitted between the external wireless power transmission device and the internal wireless power transmission device;
From a non-transmission state in which power transmission is impossible between the external wireless power transmission device and the internal wireless power transmission device, power transmission is started between the external wireless power transmission device and the internal wireless power transmission device. initiating the interactive object display process in response to changing to an enabled transmission state;
including,
An interactive object display program according to any one of claims 1 to 3. The information processing device includes a voice input unit capable of inputting voice from the outside,
wherein said interactable object display processing includes changing said image based on said audio;
Interactive object display program according to any one of claims 1 to 5. An interactive object display device for displaying a three-dimensional interactive object in virtual space in real space for interaction with a user, comprising:
The interactable object display device comprises:
an information processing device;
a support base capable of detachably supporting the information processing device;
with
The information processing device
an attitude detection unit;
a display unit including a screen for displaying images;
a processor capable of executing an interactive object display program;
with
The support base
a half mirror having a reflecting surface capable of at least partially reflecting the screen;
a mounting unit on which the information processing device can be detachably mounted with the screen facing the reflecting surface;
an adjusting mechanism capable of adjusting an angle defined between the screen of the information processing device mounted on the mounting portion and the reflecting surface of the half mirror;
with
The adjustment mechanism is configured such that the first angle and the second angle change based on the first correspondence,
the first angle is an angle between a virtual measurement reference plane in the real space and a first plane extending along the screen;
wherein the second angle is the angle between the measurement reference plane and a second plane extending along the reflective surface;
the first correspondence is that when the first angle is θA radians, the second angle θB radians is (−π/4+θA/2) radians;
the posture detection unit is capable of detecting angle information corresponding to the first angle;
causing the information processing device to execute an interactive object display process when the interactive object display program is executed by the processor;
The interactable object display process includes:
determining a line-of-sight angle between a line-of-sight direction suitable for viewing a virtual image of the image formed by the reflective surface and the measurement reference plane based on the angle information and a second correspondence;
creating the image of the interactive object photographed in the virtual space from a photographing direction inclined by the line-of-sight angle with respect to the photographing reference plane set in the virtual space as a plane corresponding to the measurement reference plane in the physical space; and
displaying the created image on the screen;
including
wherein the second correspondence is defined based on the first correspondence to represent the relationship between the first angle and the line-of-sight angle;
Interactive Object Display.

Images