WO2023145892A1

WO2023145892A1 - Display control device, and server

Info

Publication number: WO2023145892A1
Application number: PCT/JP2023/002690
Authority: WO
Inventors: 智仁山▲崎▼; 進関野
Original assignee: 株式会社Ｎｔｔドコモ
Priority date: 2022-01-31
Filing date: 2023-01-27
Publication date: 2023-08-03

Abstract

This display control device comprises an acquiring unit for acquiring a plurality of messages, a generating unit for generating a plurality of individual objects corresponding on a one-to-one basis to the plurality of messages, and a display control unit for causing a display device to display a virtual object, which is a collection of the plurality of individual objects, wherein the display control unit increases a size of the virtual object and increases a distance from a center of a virtual space to a center of the virtual object in accordance with an increase in the number of the plurality of messages.

Description

Display control device and server

The present invention relates to a display control device and a server. More particularly, the present invention relates to a display control device and a server that cause a display device to display a virtual object corresponding to a message.

By means of XR technology, including VR (Virtual Reality) technology and AR (Augmented Reality) technology, a message indicated by a virtual object is displayed in the virtual space displayed on the XR glasses that the user wears on his or her head.

For example, Patent Literature 1 discloses a technology related to a method and apparatus related to an interactive virtual environment for communication. Specifically, Patent Literature 1 discloses a technique of displaying a virtual object representing a "doodle message" in a virtual space in which information can be exchanged between users.

Japanese Patent Publication No. 2010-535363

When displaying a number of virtual objects corresponding to the number of messages in the virtual space, as the number of messages increases, the number of virtual objects also increases. Therefore, the virtual space displayed on the XR glasses worn by the user on the head may be filled with a plurality of virtual objects. In other words, in the conventional technology, there is a problem that the user's view is obstructed and the convenience is lowered.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide a display control device capable of suppressing deterioration in user convenience even when the number of messages increases when virtual objects corresponding to the number of messages are displayed in a virtual space. and

A display control device according to a preferred aspect of the present invention is a display control device for displaying a virtual space including a virtual object on a display device worn on the head of a user, the display control device comprising: an acquisition unit for acquiring a plurality of messages; a generation unit for generating a plurality of individual objects corresponding to the plurality of messages on a one-to-one basis; and a display control unit for causing the display device to display a virtual object that is a collection of the plurality of individual objects, The display control unit sets the size of the virtual object to a first size when the number of the plurality of messages is a first number, and displays the size of the virtual object in the virtual space from the center of the virtual space. When the distance to the center is a first distance and the number of the plurality of messages is a second number larger than the first number, the size of the virtual object is a second number larger than the first size. and a second distance, which is longer than the first distance, from the center of the virtual space to the center of the virtual object in the virtual space.

According to the present invention, when displaying virtual objects corresponding to the number of messages in the virtual space, even if the number of messages increases, the user's convenience can be prevented from deteriorating.

The figure which shows the whole structure of the information processing system 1 which concerns on 1st Embodiment. 1 is a perspective view showing the appearance of XR glasses 20 according to the first embodiment; FIG. 2 is a block diagram showing a configuration example of the XR glasses 20 according to the first embodiment; FIG. 1 is a block diagram showing a configuration example of a terminal device 10 according to the first embodiment; FIG. 4 is an explanatory diagram showing an example of operations of a generation unit 113 and a display control unit 114; FIG. 4 is an explanatory diagram showing an example of operations of a generation unit 113 and a display control unit 114; FIG. FIG. 10 is a diagram showing an example of how a plurality of individual objects SO1 to SO10 are aligned; 3 is a block diagram showing a configuration example of a server 30; FIG. A table showing an example of a message database MD. 4 is a flowchart showing the operation of the terminal device 10 according to the first embodiment; The block diagram which shows the structural example of 10 A of terminal devices. FIG. 4 is an explanatory diagram of an operation example of a display control unit 114A and a reception unit 115; FIG. 4 is an explanatory diagram of an operation example of a display control unit 114A and a reception unit 115; 9 is a flowchart showing the operation of the terminal device 10A according to the second embodiment; FIG. 2 is a block diagram showing a configuration example of a terminal device 10B; 3 is a block diagram showing a configuration example of a server 30A; FIG. A table showing an example of a location information database LD. FIG. 4 is an explanatory diagram showing an example of operations of a determination unit 314, an extraction unit 315, and an output unit 312A; A flow chart which shows operation of server 30A. FIG. 4 is an explanatory diagram of a virtual object VO9 generated when the terminal device 10B and the server 30A are combined;

1: First Embodiment Hereinafter, a configuration of an information processing system 1 including a terminal device 10 as a display control device according to a first embodiment of the present invention will be described with reference to FIGS. 1 to 10. FIG.

1-1: Configuration of First Embodiment 1-1-1: Overall Configuration FIG. 1 is a diagram showing the overall configuration of an information processing system 1 according to the first embodiment of the present invention. The information processing system 1 is a system that uses XR technology to provide a virtual space to a user U1 wearing XR glasses 20, which will be described later.

The information processing system 1 includes a terminal device 10, XR glasses 20, and a server 30. The terminal device 10 is an example of a display control device. In the information processing system 1, the terminal device 10 and the server 30 are communicably connected to each other via a communication network NET. Also, the terminal device 10 and the XR glasses 20 are connected so as to be able to communicate with each other. In the following description, when distinguishing between the terminal device 10 and the XR glasses 20 used for each user, the suffix "-X" is used for the reference numerals. X is an arbitrary integer of 1 or more. In addition, the same applies to the constituent elements of the terminal device 10 and the XR glasses 20, respectively. In FIG. 1, the terminal device 10-1 and the XR glasses 20 are connected so as to be able to communicate with each other. In addition, although two terminal devices 10 and one XR glass 20 are shown in FIG. A glass 20 may be provided.

　In FIG. 1, it is assumed that the user U1 uses a set of the terminal device 10-1 and the XR glasses 20. In the information processing system 1 shown in FIG. 1, the XR glasses 20 display a plurality of individual objects, which will be described later, corresponding to messages addressed to the user U1. The message may include a message transmitted from the terminal device 10-1 to the terminal device 10-2. The message may also include a message sent from another terminal device (not shown in FIG. 1) to the terminal device 10-1. Furthermore, the message may be a message generated by the terminal device 10-1 itself.

The server 30 provides various data and cloud services to the terminal device 10 via the communication network NET.

The terminal device 10-1 displays virtual objects placed in the virtual space on the XR glasses 20 worn by the user on the head. The virtual space is, for example, a celestial space. The virtual objects are, for example, virtual objects representing data such as still images, moving images, 3DCG models, HTML files, and text files, and virtual objects representing applications. Examples of text files include memos, source codes, diaries, and recipes. Examples of applications include browsers, applications for using SNS, and applications for generating document files. Note that the terminal device 10 is preferably a mobile terminal device such as a smart phone and a tablet, for example. Note that the terminal device 10-1 is an example of a display control device.

The terminal device 10-2 is a device for user U2 to send a message to user U1. The terminal device 10-2 may display a virtual object placed in the virtual space on the display 14 described later or XR glasses (not shown) connected to the terminal device 10-2. In particular, when the terminal device 10-2 is a device that displays a virtual object, the configuration of the terminal device 10-2 is basically the same as that of the terminal device 10-1. Also, similarly to the terminal device 10-1, the terminal device 10-2 is preferably a mobile terminal device such as a smart phone and a tablet, for example.

The XR glasses 20 are a see-through wearable display worn on the head of user U1. The XR glasses 20 display a virtual object on the display panel provided for each of the binocular lenses under the control of the terminal device 10-1. Note that the XR glass 20 is an example of a display device. Also, an example in which the XR glasses 20 are AR glasses will be described below. However, the fact that the XR glasses 20 are AR glasses is only an example, and the XR glasses 20 may be VR glasses or MR (Mixed Reality) glasses.

1-1-2: Configuration of XR Glasses FIG. 2 is a perspective view showing the appearance of the XR glasses 20. As shown in FIG. As shown in FIG. 2, the XR glasses 20 have

temples

91 and 92, a bridge 93,

frames

94 and 95, and

lenses

41L and 41R, like general eyeglasses.

An imaging device 26 is provided on the bridge 93 . The imaging device 26 images the outside world. The imaging device 26 also outputs imaging information indicating the captured image.

Each of the

lenses

41L and 41R has a half mirror. A frame 94 is provided with a liquid crystal panel or an organic EL panel for the left eye. A liquid crystal panel or an organic EL panel is hereinafter generically referred to as a display panel. The frame 94 is provided with an optical member that guides the light emitted from the display panel for the left eye to the lens 41L. The half mirror provided in the lens 41L transmits external light and guides it to the left eye, and reflects the light guided by the optical member to enter the left eye. The frame 95 is provided with a right-eye display panel and an optical member that guides light emitted from the right-eye display panel to the lens 41R. The half mirror provided in the lens 41R transmits external light and guides it to the right eye, and reflects the light guided by the optical member to enter the right eye.

The display 28, which will be described later, includes a lens 41L, a left-eye display panel, a left-eye optical member, and a lens 41R, a right-eye display panel, and a right-eye optical member.

With the above configuration, the user U1 can observe the image displayed by the display panel in a see-through state in which the image is superimposed on the appearance of the outside world. Further, in the XR glasses 20, of the binocular images with parallax, the image for the left eye is displayed on the display panel for the left eye, and the image for the right eye is displayed on the display panel for the right eye. On the other hand, it is possible to perceive the displayed image as if it had depth and stereoscopic effect.

FIG. 3 is a block diagram showing a configuration example of the XR glasses 20. As shown in FIG. The XR glasses 20 include a processing device 21 , a storage device 22 , a line-of-sight detection device 23 , a GPS device 24 , a motion detection device 25 , an imaging device 26 , a communication device 27 and a display 28 . Each element of the XR glasses 20 is interconnected by one or more buses for communicating information. Note that the term "apparatus" in this specification may be replaced with another term such as a circuit, a device, or a unit.

The processing device 21 is a processor that controls the XR glasses 20 as a whole. The processing device 21 is configured using, for example, one or more chips. The processing device 21 is configured using, for example, a central processing unit (CPU) including an interface with peripheral devices, an arithmetic device, registers, and the like. Some or all of the functions of the processing device 21 are implemented by hardware such as DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), PLD (Programmable Logic Device), and FPGA (Field Programmable Gate Array). may be realized. The processing device 21 executes various processes in parallel or sequentially.

The storage device 22 is a recording medium that can be read and written by the processing device 21 . The storage device 22 also stores a plurality of programs including the control program PR1 executed by the processing device 21 .

The line-of-sight detection device 23 detects the line-of-sight of the user U1 and generates line-of-sight information indicating the detection result. Any method may be used to detect the line of sight by the line of sight detection device 23 . The line-of-sight detection device 23 may detect line-of-sight information based on, for example, the position of the inner corner of the eye and the position of the iris. The line-of-sight information indicates the line-of-sight direction of the user U1. The line-of-sight detection device 23 supplies the line-of-sight information to the processing device 21, which will be described later. The line-of-sight information supplied to the processing device 21 is transmitted to the terminal device 10 via the communication device 27 .

The GPS device 24 receives radio waves from multiple satellites. The GPS device 24 also generates position information from the received radio waves. The positional information indicates the position of the XR glasses 20 . The location information may be in any format as long as the location can be specified. The position information indicates the latitude and longitude of the XR glasses 20, for example. As an example, location information is obtained from GPS device 24 . However, the XR glasses 20 may acquire position information by any method. The acquired position information is supplied to the processing device 21 . The position information output to the processing device 21 is transmitted to the terminal device 10 via the communication device 27 .

The motion detection device 25 detects motion of the XR glasses 20 . The motion detection device 25 corresponds to an inertial sensor such as an acceleration sensor that detects acceleration and a gyro sensor that detects angular acceleration. The acceleration sensor detects acceleration in orthogonal X-, Y-, and Z-axes. The gyro sensor detects angular acceleration around the X-, Y-, and Z-axes. The motion detection device 25 can generate posture information indicating the posture of the XR glasses 20 based on the output information of the gyro sensor. The motion information includes acceleration data indicating three-axis acceleration and angular acceleration data indicating three-axis angular acceleration. In addition, the motion detection device 25 supplies posture information indicating the posture of the XR glasses 20 and motion information related to the motion of the XR glasses 20 to the processing device 21 . The posture information and motion information supplied to the processing device 21 are transmitted to the terminal device 10 via the communication device 27 .

The imaging device 26 outputs imaging information obtained by imaging the outside world. Also, the imaging device 26 includes, for example, a lens, an imaging element, an amplifier, and an AD converter. The light condensed through the lens is converted into an image pickup signal, which is an analog signal, by the image pickup device. The amplifier amplifies the imaging signal and outputs it to the AD converter. The AD converter converts the amplified imaging signal, which is an analog signal, into imaging information, which is a digital signal. The converted imaging information is supplied to the processing device 21 . The imaging information supplied to the processing device 21 is transmitted to the terminal device 10 via the communication device 27 .

The communication device 27 is hardware as a transmission/reception device for communicating with other devices. The communication device 27 is also called a network device, a network controller, a network card, a communication module, etc., for example. The communication device 27 may include a connector for wired connection and an interface circuit corresponding to the connector. Further, the communication device 27 may have a wireless communication interface. Products conforming to wired LAN, IEEE1394, and USB are examples of connectors and interface circuits for wired connection. Also, as a wireless communication interface, there are products conforming to wireless LAN, Bluetooth (registered trademark), and the like.

The display 28 is a device that displays images. The display 28 displays various images under the control of the processing device 21 . The display 28 includes the lens 41L, the left-eye display panel, the left-eye optical member, and the lens 41R, the right-eye display panel, and the right-eye optical member, as described above. Various display panels such as a liquid crystal display panel and an organic EL display panel are preferably used as the display panel.

The processing device 21 functions as an acquisition unit 211 and a display control unit 212, for example, by reading the control program PR1 from the storage device 22 and executing it.

The acquisition unit 211 acquires image information indicating an image displayed on the XR glasses 20 from the terminal device 10-1.

The acquisition unit 211 also receives line-of-sight information supplied from the line-of-sight detection device 23 , position information supplied from the GPS device 24 , posture information and motion information supplied from the motion detection device 25 , and information supplied from the imaging device 26 . Acquire imaging information. After that, the acquisition unit 211 supplies the acquired line-of-sight information, position information, posture information, motion information, and imaging information to the communication device 27 .

Based on the image information acquired from the terminal device 10-1 by the acquisition unit 211, the display control unit 212 causes the display 28 to display an image indicated by the image information.

1-1-3: Configuration of Terminal Device FIG. 4 is a block diagram showing a configuration example of the terminal device 10. As shown in FIG. The terminal device 10 includes a processing device 11 , a storage device 12 , a communication device 13 , a display 14 , an input device 15 and an inertial sensor 16 . Elements of the terminal device 10 are interconnected by one or more buses for communicating information. As the configuration of the terminal device 10, the configuration of the terminal device 10-1 will be basically described below.

The processing device 11 is a processor that controls the terminal device 10 as a whole. Also, the processing device 11 is configured using, for example, a single chip or a plurality of chips. The processing unit 11 is configured using, for example, a central processing unit (CPU) including interfaces with peripheral devices, arithmetic units, registers, and the like. A part or all of the functions of the processing device 11 may be implemented by hardware such as DSP, ASIC, PLD, and FPGA. The processing device 11 executes various processes in parallel or sequentially.

The storage device 12 is a recording medium readable and writable by the processing device 11 . The storage device 12 also stores a plurality of programs including the control program PR2 executed by the processing device 11 .

The communication device 13 is hardware as a transmission/reception device for communicating with other devices. The communication device 13 is also called a network device, a network controller, a network card, a communication module, or the like, for example. The communication device 13 may include a connector for wired connection and an interface circuit corresponding to the connector. Further, the communication device 13 may have a wireless communication interface. Products conforming to wired LAN, IEEE1394, and USB are examples of connectors and interface circuits for wired connection. Also, as a wireless communication interface, there are products conforming to wireless LAN, Bluetooth (registered trademark), and the like.

The display 14 is a device that displays images and character information. The display 14 displays various images under the control of the processing device 11 . For example, various display panels such as a liquid crystal display panel and an organic EL (Electro Luminescence) display panel are preferably used as the display 14 . Note that when the XR glasses 20 are connected to the terminal device 10, the display 14 may not be an essential component. In this case, the XR glasses 20 further have the same function as the display 14 .

The input device 15 accepts operations from the user U1 who wears the XR glasses 20 on his head. For example, the input device 15 includes a pointing device such as a keyboard, touch pad, touch panel, or mouse. Here, when the input device 15 includes a touch panel, the input device 15 may also serve as the display 14 .

The inertial sensor 16 is a sensor that detects inertial force. The inertial sensor 16 includes, for example, one or more of an acceleration sensor, an angular velocity sensor, and a gyro sensor. The processing device 11 detects the orientation of the terminal device 10 based on the output information from the inertial sensor 16 . Further, the processing device 11 receives selection of the virtual object VO, input of characters, and input of instructions in the celestial sphere virtual space VS based on the orientation of the terminal device 10 . For example, the user U1 directs the central axis of the terminal device 10 toward a predetermined area of the virtual space VS, and operates the input device 15 to select the virtual object VO arranged in the predetermined area. The user U1's operation on the input device 15 is, for example, a double tap. By operating the terminal device 10 in this way, the user U1 can select the virtual object VO without looking at the input device 15 of the terminal device 10 .

If the XR glasses 20 are not connected to the terminal device 10, the terminal device 10 preferably has a GPS device similar to the GPS device 24 provided in the XR glasses 20.

The processing device 11 functions as an output unit 111, an acquisition unit 112, a generation unit 113, and a display control unit 114 by reading the control program PR2 from the storage device 12 and executing it.

The output unit 111 outputs a message created by the user U1 using the input device 15 to the server 30. The message specifies the sender of the message, the receiver of the message, and the content of the message. The content of the message includes at least one of text and images.

Furthermore, when the terminal device 10 is the terminal device 10-1 used by the user U1 who is the first user, the output unit 111 outputs a message addressed to the user U1 acquired by the acquisition unit 112-1, which will be described later. When it is read by the user U1, information indicating that it has been read is transmitted to the server 30. FIG.

The acquisition unit 112 acquires a plurality of messages addressed to the user U from the server 30. If the terminal device 10 is the terminal device 10-1 used by the user U1 who is the first user, the obtaining unit 112-1 obtains from the server 30 a plurality of messages addressed to the user U1.

The generating unit 113 generates multiple individual objects corresponding to the multiple messages acquired by the acquiring unit 112 on a one-to-one basis.

The display control unit 114 causes the XR glasses 20 as a display device to display the virtual object, which is a collection of multiple individual objects generated by the generation unit 113 .

As a result, user U1 can visually confirm the number of multiple messages.

5 and 6 are explanatory diagrams showing an example of the operation of the generation unit 113 and the display control unit 114. FIG. In the following description, it is assumed that X, Y and Z axes are orthogonal to each other in the virtual space VS. As an example, the X-axis extends in the front-rear direction of user U1. Furthermore, as viewed from the user U1, the forward direction along the X axis is the X1 direction, and the backward direction along the X axis is the X2 direction. Also, the Y-axis extends in the horizontal direction of the user U1. Furthermore, as viewed from the user U1, the right direction along the Y axis is the Y1 direction, and the left direction along the Y axis is the X2 direction. A horizontal plane is formed by these X-axis and Y-axis. Also, the Z-axis is orthogonal to the XY plane and extends in the vertical direction of the user U1. Furthermore, when viewed from the user U1, the downward direction along the Z axis is the Z1 direction, and the upward direction along the Z axis is the Z2 direction.

Referring to FIG. 5, it is assumed that user U1 is located at coordinates (x, y, z)=(x _u1 , y _u1 , z _u1 ) in virtual space VS. It is assumed that the coordinates of this user U1 are approximately at the center of the virtual space VS. “Substantially the center” is a position that includes not only the center of the virtual space VS but also an area within a range from the center that does not hinder the operation of the information processing system 1 according to the present invention. The coordinates of the user U1 in the virtual space VS correspond to the position of the user U1 in the real space. Also, the position of the user U1 in the physical space is indicated by position information generated in the XR glasses 20 worn on the head of the user U1.

In the example shown in FIG. 5, the virtual object VO1 displayed at coordinates (x, y, z)=(x ₁ , y ₁ , z ₁ ) by the display control unit 114 is a message addressed to the user U1. is a set of individual objects SO1 to SO3 corresponding one-to-one to . In FIG. 5, the virtual object VO1 and the individual objects SO1 to SO3 are, for example, spheres. Also, the coordinates (x, y, z) of the individual object SO1 = ( _x2 , _y2 , _z2 ), the coordinates (x, y, z) of the individual object SO2 = ( _x3 , _y3 , _z3 ), And the coordinates (x, y, z)=(x ₄ , y ₄ , z ₄ ) of the individual object SO3 are all coordinates within the virtual object VO1. Further, each sphere as the individual objects SO1 to SO3 exists entirely inside the sphere as the virtual object VO1. That is, the virtual object VO1 includes the same number of individual objects SO1 to SO3 as the number of messages addressed to the user U1.

The image information used by the display control unit 114 to display the individual objects SO1 to SO3 on the XR glasses 20 may be information stored in the storage device 12. Alternatively, the image information may be information acquired from the server 30 by the acquisition unit 112 .

Also, individual objects SO1 to SO3 may correspond one-to-one to all messages addressed to user U1. Alternatively, individual objects SO1 to SO3 may correspond one-to-one only to unread messages among all messages addressed to user U1. Alternatively, one individual object may correspond to a plurality of read messages.

Also, in FIG. 5, the viewing angle when the user U1 visually recognizes the virtual object VO1 is _θ1 .

Returning to FIG. 4, the display control unit 114 further increases the size of the virtual object VO1 as the number of messages acquired by the acquisition unit 112 increases. That is, the display control unit 114 increases the size of the virtual object VO1 as the number of individual objects SO included in the virtual object VO1 increases. Further, the display control unit 114 arranges the virtual object VO1 at a position farther from the user U1 in the virtual space VS as the number of the plurality of messages is larger. In other words, the display control unit 114 increases the distance from the center of the virtual space VS to the center of the virtual object VO1 as the number of messages increases. That is, when the number of the plurality of messages is the first number, the display control unit 114 sets the size of the virtual object VO1 to the first size, and displays the size of the virtual object VO1 from the center of the virtual space VS. Let the distance to the center be the first distance. Further, when the number of the plurality of messages is a second number larger than the first number, the display control unit 114 sets the size of the virtual object VO1 to a second size larger than the first size. At the same time, the distance from the center of the virtual space VS to the center of the virtual object VO1 is set to a second distance longer than the first distance.

In the example shown in FIG. 6, virtual objects VO2 displayed at coordinates (x, y, z)=(x ₅ , y ₅ , z ₅ ) correspond one-to-one to messages addressed to user U1. It is a set of individual objects SO1 to SO6 that are connected to each other. In FIG. 6, the shapes of the virtual object VO2 and the individual objects SO1 to SO6 are, for example, spheres. The virtual object VO2 contains the same number of individual objects SO1 to SO6 as the number of messages addressed to the user U1.

Comparing the virtual object VO1 shown in FIG. 5 and the virtual object VO2 shown in FIG. 6, the virtual object VO2 contains more individual objects SO than the virtual object VO1. Also, the virtual object VO2 is larger than the virtual object VO1. Furthermore, the coordinates (x, y, z)=(x ₅ , y ₅ , z ₅ ) of the virtual object VO2 are the coordinates (x, y, z)=(x ₁ , y ₁ , z ₁ ) of the virtual object VO1. It is positioned farther from the coordinates (x, y, z)=(x _U1 , y _U1 , z _U1 ) of the user U1, which is approximately the center of the virtual space VS.

Also, in FIG. 6, the viewing angle when the user U1 visually recognizes the virtual object VO2 is _θ2 . 5 and 6, the viewing angle θ ₁ when the user views the virtual object VO1 in FIG. 5 and the viewing angle θ ₂ when the user views the virtual object VO2 in FIG. Equal is preferred. When θ ₁ and θ ₂ are equal, the proportion occupied by the virtual object VO1 and the proportion occupied by the virtual object VO2 in the field of view of the user U1 are equal. Even if the number of individual objects SO that correspond one-to-one to the message addressed to user U1 increases, the area of virtual object VO2 in the field of view of user U1 remains equal to the area of virtual object VO1. I don't feel like my field of vision has narrowed. However, when the number of individual objects SO increases, the area of each individual object SO in the field of view of user U1 becomes smaller. Note that the viewing angle _θ1 and the viewing angle _θ2 may not necessarily be equal.

For convenience of explanation, different symbols are used for the virtual object VO1 and the virtual object VO2. VO1 may gradually increase.

Also, the display control unit 114 changes the display mode of the virtual object VO1 according to the number of messages addressed to the user U1. For example, as the number of individual objects SO included in the virtual object VO1 increases, the display control unit 114 changes at least one of the color of the virtual object VO1 and the color of the individual objects SO included in the virtual object VO1. You may let Alternatively, the display control unit 114 may change the shape of the virtual object VO1 as the number of individual objects SO increases. For example, each time the number of individual objects SO reaches a predetermined number, a plurality of individual objects SO may be arranged to represent a specific character in the virtual object VO1. FIG. 7 is a diagram showing an example of how a plurality of individual objects SO1 to SO10 are aligned. In the example shown in FIG. 7, in the virtual object VO3, the individual objects SO1 to SO10 are aligned to represent the letter "N".

As a result, when the number of multiple messages increases, the terminal device 10 can more appeal to the user U1 that the number of multiple messages has increased.

Further, the display control unit 114 may vary the display modes of the individual objects SO according to the transmission source devices corresponding to each of the plurality of messages addressed to the user U1. For example, the display control unit 114 may change at least one of the shape and color of the individual object SO according to the device that sent the message.

As a result, the user U1 can distinguish the plurality of individual objects SO according to the transmission source devices of the plurality of messages, simply by visually recognizing the plurality of individual objects SO.

1-1-4: Server Configuration FIG. 8 is a block diagram showing a configuration example of the server 30. As shown in FIG. The server 30 comprises a processing device 31 , a storage device 32 , a communication device 33 , a display 34 and an input device 35 . Each element of server 30 is interconnected by one or more buses for communicating information.

The processing device 31 is a processor that controls the server 30 as a whole. Also, the processing device 31 is configured using, for example, a single chip or a plurality of chips. The processing unit 31 is configured using, for example, a central processing unit (CPU) including interfaces with peripheral devices, arithmetic units, registers, and the like. A part or all of the functions of the processing device 31 may be realized by hardware such as DSP, ASIC, PLD, and FPGA. The processing device 31 executes various processes in parallel or sequentially.

The storage device 32 is a recording medium readable and writable by the processing device 31 . The storage device 32 also stores a plurality of programs including the control program PR3 executed by the processing device 31 . Furthermore, the storage device 32 stores a message database MD in which information related to messages transmitted and received between a plurality of users U is stored.

FIG. 9 is a table showing an example of the message database MD. As will be described later, the acquisition unit 311 provided in the server 30 acquires messages transmitted and received between the users U from the terminal device 10 . More specifically, the acquisition unit 311 acquires information indicating the sender of a message output from the terminal device 10, information indicating the recipient of the message, and information indicating the content of the message. These information are stored in the message database MD. Further, the acquisition unit 311 acquires information indicating that each message has been read by the user U. Based on this information, the message management unit 313, which will be described later, adds a flag indicating whether or not each message has been read in the message database MD. As an example, a flag with a value of '0' indicates that the message is unread. On the other hand, a flag with a value of "1" indicates that the message has already been read. As an example, the message database MD shown in FIG. 9 stores the message ID "1", the sender of the message with the message ID=1 "U1", the recipient "U2", the content of the message, and a flag "0" indicating unread are associated with each other and stored. In addition, in the table shown in FIG. 9, "n" is an integer of 2 or more.

Returning to FIG. 8, the communication device 33 is hardware as a transmission/reception device for communicating with other devices. The communication device 33 is also called a network device, a network controller, a network card, a communication module, etc., for example. The communication device 33 may include a connector for wired connection and an interface circuit corresponding to the connector. Further, the communication device 33 may have a wireless communication interface. Products conforming to wired LAN, IEEE1394, and USB are examples of connectors and interface circuits for wired connection. Also, as a wireless communication interface, there are products conforming to wireless LAN, Bluetooth (registered trademark), and the like.

The display 34 is a device that displays images and character information. The display 34 displays various images under the control of the processing device 31 . For example, various display panels such as a liquid crystal display panel and an organic EL display panel are preferably used as the display 34 .

The input device 35 is a device that accepts operations by the administrator of the information processing system 1 . For example, the input device 35 includes a pointing device such as a keyboard, touch pad, touch panel, or mouse. Here, when the input device 35 includes a touch panel, the input device 35 may also serve as the display 34 .

The processing device 31 functions as an acquisition unit 311, an output unit 312, and a message management unit 313, for example, by reading the control program PR3 from the storage device 32 and executing it.

The acquisition unit 311 acquires various data from the terminal device 10 via the communication device 33 . The data includes, for example, data indicating the operation content for the virtual object VO, which is input to the terminal device 10 by the user U1 wearing the XR glasses 20 on the head.

Above all, the acquisition unit 311 acquires messages transmitted and received between users U. Furthermore, as an example, when a message addressed to user U1 has been read by user U1 in terminal device 10-1, acquisition unit 311 acquires information indicating that the message has been read.

The output unit 312 transmits image information indicating an image displayed on the XR glasses 20 to the terminal device 10 . The image information may be stored in the storage device 32 . Alternatively, the image information may be generated by a generating unit (not shown).

Also, as an example, the output unit 312 transmits a message addressed to the user U1 stored in the message database MD to the terminal device 10-1. The same applies to other terminal devices 10 as well.

The message management unit 313 manages the message database MD. As an example, in the terminal device 10-1, when the acquisition unit 311 acquires information indicating that the message addressed to the user U1 has been read by the user U1, the message management unit 313 links the message to the message. The flag "0" indicating unread is changed to the flag "1" indicating read.

1-2: Operation of First Embodiment FIG. 10 is a flow chart showing the operation of the terminal device 10 according to the first embodiment, especially the terminal device 10-1 used by the user U1. The operation of the terminal device 10-1 will be described below with reference to FIG.

In step S1, the processing device 11-1 functions as an acquisition unit 112-1. Processing device 11-1 acquires a plurality of messages addressed to user U1.

In step S2, the processing device 11-1 functions as the generator 113-1. The processing device 11-1 generates a plurality of individual objects SO corresponding to the plurality of messages on a one-to-one basis.

In step S3, the processing device 11-1 functions as the display control unit 114-1. The processing device 11-1 causes the XR glasses 20 as a display device to display a virtual object VO, which is an aggregate of a plurality of individual objects SO. In particular, the processing device 11-1 increases the size of the virtual object VO as the number of messages acquired in step S1 increases. Also, the processing device 11-1 arranges the virtual object VO1 at a position farther from the user U1 in the virtual space VS as the number of the plurality of messages acquired in step S1 increases. After that, the processing device 11-1 executes the process of step S1.

1-3: Effect of the First Embodiment According to the above description, the terminal device 10 as a display control device has a virtual space including a virtual object VO on the XR glasses 20 as a display device worn on the head. It is a display control device for displaying VS. The terminal device 10 includes an acquisition unit 112 , a generation unit 113 and a display control unit 114 . Acquisition unit 112 acquires a plurality of messages. The generation unit 113 generates a plurality of individual objects SO corresponding to a plurality of messages on a one-to-one basis. The display control unit 114 causes the XR glasses 20 as a display device to display a virtual object VO, which is an aggregate of a plurality of individual objects SO. Further, when the number of the plurality of messages is the first number, the display control unit 114 sets the size of the virtual object VO to the first size, and sets the size of the virtual object VO to the first size in the virtual space VS. Let the distance to the center of the virtual object VO be the first distance. Furthermore, when the number of messages is a second number that is larger than the first number, the display control unit 114 sets the size of the virtual object VO to a second size that is larger than the first size. , in the virtual space VS, the distance from the center of the virtual space VS to the center of the virtual object VO is set as a second distance longer than the first distance.

Since the terminal device 10 has the above configuration, it is possible to display the virtual objects VO corresponding to the number of messages in the virtual space VS, and to suppress the deterioration of the user's convenience even if the number of messages increases. It becomes possible. Specifically, in the virtual space VS, as the number of individual objects SO corresponding to the number of messages increases, the terminal device 10 increases the size of the virtual object VO, which is a collection of the individual objects SO. At the same time, the virtual object VO is displayed at a position farther from the user U1. Since the terminal device 10 has this configuration, even if the number of individual objects SO increases on the display 28 provided on the XR glasses 20 worn on the head of the user U1, the viewability thereof is ensured. As a result, deterioration in user convenience is suppressed.

Also, according to the above description, the display control unit 114 changes the display mode of the virtual object VO according to the number of messages.

Since the terminal device 10 has the above configuration, when the number of multiple messages increases, it becomes possible to further appeal to the user U1 that the number of multiple messages has increased.

Also, according to the above description, the display control unit 114 makes the display modes of the plurality of individual objects SO different from each other according to the transmission source device corresponding to each of the plurality of messages.

Since the terminal device 10 has the above configuration, the user U1 can distinguish the plurality of individual objects SO according to the transmission source devices of the plurality of messages simply by visually recognizing the plurality of individual objects SO. becomes.

2: Second Embodiment Hereinafter, a configuration of an information processing system 1A including a terminal device 10A as a display control device according to a second embodiment of the present invention will be described with reference to FIGS. 11 to 14. FIG.

2-1: Configuration of Second Embodiment 2-1-1: Overall Configuration An information processing system 1A according to the second embodiment of the present invention has terminal devices as compared with the information processing system 1 according to the first embodiment. 10 in that a terminal device 10A is provided instead. Otherwise, the overall configuration of the information processing system 1A is the same as the overall configuration of the information processing system 1 according to the first embodiment shown in FIG. 1, so illustration and description thereof will be omitted.

2-1-2: Configuration of Terminal Device FIG. 11 is a block diagram showing a configuration example of the terminal device 10A. Unlike the terminal device 10, the terminal device 10A includes a processing device 11A instead of the processing device 11 and a storage device 12A instead of the storage device 12. FIG.

Unlike the storage device 12, the storage device 12A stores the control program PR2A instead of the control program PR2.

The processing device 11A includes a display control unit 114A instead of the display control unit 114 included in the processing device 11. The processing device 11A also includes a reception unit 115 and a determination unit 116 in addition to the components included in the processing device 11 .

The reception unit 115 receives an operation from the user U1 on the virtual object VO. For example, the accepting unit 115 may accept an operation using the input device 15 by the user U1. Alternatively, the accepting unit 115 may accept an operation of the user U1 touching the virtual object VO in the virtual space VS. The operation of selecting the virtual object VO in the virtual space VS is an example of the first operation.

The receiving unit 115 may also receive an operation by the user U1 to select one message from a list of multiple messages displayed in the virtual space VS, as will be described later. There is a one-to-one correspondence between these multiple messages and multiple individual objects SO. One individual object SO is selected by the user U1 selecting one message from a list of a plurality of messages. Alternatively, the accepting unit 115 may accept an operation of the user U1 touching one individual object SO in the virtual space VS. The operation of selecting one individual object SO in the virtual space VS is an example of the second operation.

The display control unit 114A has the same function as the display control unit 114. Further, when the operation by the user U1 is the first operation, the display control unit 114A displays a list of multiple messages in the virtual space VS. On the other hand, when the operation by the user U1 is the second operation, the display control unit 114A causes the virtual space VS to display the content of the message corresponding to one individual object SO designated by the second operation. .

12 and 13 are explanatory diagrams of operation examples of the display control unit 114A and the reception unit 115. FIG. In FIG. 12, when the user U1 performs an operation of selecting the virtual object VO2 using the terminal device 10-1, the display control unit 114A displays a list L of a plurality of messages in the virtual space VS. More specifically, as shown in FIG. 12, a list of titles of messages A to F is displayed as list L in virtual space VS by display control unit 114A. Messages A to F correspond one-to-one to individual objects SO1 to SO6, respectively. Note that in FIG. 12, the list L is displayed on the left side of the virtual object VO2 as seen from the user U1, but this display location is merely an example. The list L may be displayed anywhere in the virtual space VS. In particular, the list L is preferably displayed at a position that does not overlap the virtual object VO2 when viewed from the user U1. The same applies when the user U1 performs an operation of touching the virtual object VO7 in the virtual space VS.

As a result, the user U1 can visually recognize, in a list format, a plurality of messages corresponding one-to-one to the plurality of individual objects SO included in the virtual object VO.

　In FIG. 12, when the user U1 performs an operation to select one message from a plurality of messages shown in the list L, a message object MO1 indicating the contents of the message is displayed as shown in FIG. The message object MO1 includes at least one of text and images. In the example shown in FIG. 13, the user U1 performs an operation to select "message F" corresponding to the individual object SO6 from the plurality of messages shown in the list L. The same is true when the user U1 performs an operation to touch the individual object SO6 in the virtual space VS.

As a result, the user U1 can visually recognize the specific contents of each of the multiple messages corresponding to the multiple individual objects SO included in the virtual object VO.

Returning the description to FIG. 11, the determination unit 116 determines the importance of each of the plurality of messages. For example, the determination unit 116 may determine the importance by analyzing the contents of each of the multiple messages. Alternatively, the determination unit 116 may determine the importance based on the transmission source device corresponding to each of the multiple messages. Alternatively, the determining unit 116 may determine the degree of importance based on the operation of the user U1 using the input device 15. FIG. For example, as shown in FIG. 13, while the contents of a message corresponding to one individual object SO are displayed, the user U1 judges the importance of the message and inputs the judgment result using the input device 15. do. The determining unit 116 may determine the degree of importance based on the content of input from the input device 15 .

Of the plurality of individual objects SO included in the virtual object VO, the display control unit 114A displays the individual objects SO corresponding to the messages having a degree of importance equal to or greater than a predetermined value, and the individual objects SO corresponding to the messages having a degree of importance less than the predetermined value. is displayed near the user U1, that is, near the center of the virtual space VS. Further, the display control unit 114A may display an individual object SO having a higher degree of importance in the virtual object VO near the user U1, that is, near the approximate center of the virtual space VS. The "predetermined value" described above is an example of the "first value".

As a result, user U1 can preferentially check messages that are more important to him/herself.

2-2: Operation of Second Embodiment FIG. 14 is a flowchart showing the operation of the terminal device 10A according to the second embodiment, especially the terminal device 10A-1 used by the user U1. The operation of the terminal device 10A-1 will be described below with reference to FIG.

In step S11, the processing device 11A-1 functions as an acquisition unit 112-1. The processing device 11A-1 acquires a plurality of messages addressed to the user U1.

In step S12, the processing device 11A-1 functions as the generator 113-1. The processing device 11A-1 generates a plurality of individual objects SO corresponding to the plurality of messages on a one-to-one basis.

In step S13, the processing device 11A-1 functions as the display control section 114A-1. The processing device 11A-1 causes the XR glasses 20 as a display device to display a virtual object VO, which is an aggregate of a plurality of individual objects SO. The processing device 11A-1 increases the size of the virtual object VO as the number of messages acquired in step S11 increases. Also, the processing device 11A-1 arranges the virtual object VO1 at a position farther from the user U1 in the virtual space VS as the number of the plurality of messages acquired in step S11 increases.

In step S14, the processing device 11A-1 functions as the reception unit 115. Processing device 11A-1 receives an operation from user U1. If the operation from the user U1 is the first operation on the virtual object VO, the processing device 11A-1 executes the process of step S15. If the operation from the user U1 is the second operation on the individual object SO, the processing device 11A-1 executes the process of step S16.

In step S15, the processing device 11A-1 functions as the display control section 114A. The processing device 11A-1 causes the list L of a plurality of messages to be displayed in the virtual space VS. After that, the processing device 11A-1 executes the process of step S14.

In step S16, the processing device 11A-1 functions as the display control section 114A. The processing device 11A-1 causes the contents of the message corresponding to one individual object SO to be displayed in the virtual space VS. After that, the processing device 11A-1 executes the process of step S11.

2-3: Effects of the Second Embodiment According to the above description, the terminal device 10A as a display control device further includes the reception unit 115 that receives an operation on the virtual object VO. If the above operation is the first operation, the display control unit 114 causes the list L of the plurality of messages to be displayed in the virtual space VS.

Since the terminal device 10A has the above configuration, the user U1 can visually recognize, in a list format, a plurality of messages corresponding one-to-one to a plurality of individual objects SO included in the virtual object VO.

Further, according to the above description, in the terminal device 10A as the display control device, the display control unit 114, when the above operation is the second operation of designating one individual object SO among the plurality of individual objects SO, The content of the message corresponding to one individual object SO is displayed in the virtual space VS.

Since the terminal device 10A has the above configuration, the user U1 can visually recognize the specific contents of each of the multiple messages corresponding one-to-one to the multiple individual objects SO included in the virtual object VO.

Also, according to the above description, the terminal device 10A as a display control device further includes the determination unit 116 that determines the importance of each of a plurality of messages. Of the plurality of individual objects SO included in the virtual object VO, the display control unit 114 selects individual objects SO corresponding to messages having a degree of importance greater than or equal to the first value, and displays individual objects SO corresponding to messages having a degree of importance less than the first value. The individual object SO is displayed near the user U1 as compared with the individual object SO.

Since the terminal device 10A has the above configuration, the user U can preferentially check messages that are more important to him/herself.

3: Third Embodiment A configuration of an information processing system 1B including a terminal device 10B as a display control device according to a third embodiment of the present invention will be described below with reference to FIG.

3-1: Configuration of Third Embodiment 3-1-1: Overall Configuration An information processing system 1B according to the third embodiment of the present invention has terminal devices as compared with the information processing system 1 according to the first embodiment. 10 in that a terminal device 10B is provided instead. Otherwise, the overall configuration of the information processing system 1B is the same as the overall configuration of the information processing system 1 according to the first embodiment shown in FIG. 1, so illustration and description thereof will be omitted.

3-1-2: Configuration of Terminal Device FIG. 15 is a block diagram showing a configuration example of the terminal device 10B. Unlike the terminal device 10, the terminal device 10B includes a processing device 11B instead of the processing device 11 and a storage device 12B instead of the storage device 12. FIG. The terminal device 10</b>B also includes a sound pickup device 17 in addition to the components included in the terminal device 10 .

The sound pickup device 17 picks up the voice of the user U1 and converts the picked-up voice into an electric signal. The sound collecting device 17 is specifically a microphone. An electrical signal converted from the voice by the sound collecting device 17 is output to the voice recognition unit 117, which will be described later.

Unlike the storage device 12, the storage device 12B stores the control program PR2B instead of the control program PR2.

Unlike the processing device 11, the processing device 11B includes an acquisition unit 112B instead of the acquisition unit 112. The processing device 11B also includes a speech recognition unit 117 and a message generation unit 118 in addition to the components included in the processing device 11. FIG.

The voice recognition unit 117 recognizes the voice collected by the sound collection device 17. More specifically, the speech recognition unit 117 generates text by performing speech recognition based on the electrical signal acquired from the sound pickup device 17 .

The message generation unit 118 generates a message corresponding to the text generated by the speech recognition unit 117. 112 A of acquisition parts acquire the message which the message production|generation part 118 produced|generated. That is, the messages generated by the message generator 118 are included in the plurality of messages acquired by the acquirer 112A. The plurality of messages acquired by acquisition unit 112A may include messages generated by multiple users U including user U1 in addition to messages generated by message generation unit 118 . In other words, the plurality of messages are generated by the terminal device 10B-1 as a display control device and one or more terminal devices 10B connected to the terminal device 10B-1 via the communication network NET.

Since the terminal device 10B has the above configuration, it can generate a message based on the voice uttered by the user U1 and generate the individual object SO based on the generated message.

3-2: Operation of the Third Embodiment The operation of the terminal device 10B according to the third embodiment, particularly the terminal device 10B-1 used by the user U1, basically corresponds to the operation of the terminal device according to the first embodiment shown in FIG. Since the operation is the same as that of the device 10-1, its illustration and detailed description are omitted. In the operation of the terminal device 10B-1, the messages generated by the message generator 118 are included in the plurality of messages acquired in step S1.

3-3: Effect of the Third Embodiment According to the above description, in the terminal device 10B-1 as the display control device, a plurality of messages can be sent to the terminal device 10B-1 as the display control device and the communication network NET. is generated by one or a plurality of terminal devices 10B connected to the terminal device 10B-1 via.

Since the terminal device 10B has the above configuration, the user U1 can confirm messages generated by users U other than the user U1.

According to the above description, the terminal device 10B as a display control device further includes the sound pickup device 17, the speech recognition section 117, and the message generation section 118. The sound pickup device 17 picks up the voice of the user U and outputs an electrical signal representing the voice. The speech recognition unit 117 generates text based on the electrical signal output from the sound pickup device 17 . Message generator 118 generates a message corresponding to the text generated by speech recognizer 117 . Also, the plurality of messages includes messages generated by the message generator 118 .

Since the terminal device 10B has the above configuration, it can generate the individual object SO based on the voice uttered by the user U1.

4: Fourth Embodiment A configuration of an information processing system 1C including a terminal device 10C as a display control device according to a fourth embodiment of the present invention will be described below with reference to FIGS. 16 to 19. FIG.

4-1: Configuration of Fourth Embodiment 4-1-1: Overall Configuration An information processing system 1C according to the fourth embodiment of the present invention has terminal devices as compared with the information processing system 1 according to the first embodiment. 10 and a server 30A instead of the server 30, respectively. Otherwise, the overall configuration of the information processing system 1C is the same as the overall configuration of the information processing system 1 according to the first embodiment shown in FIG. 1, so illustration and description thereof will be omitted.

4-1-2: Configuration of Terminal Device Unlike the terminal device 10, the terminal device 10C includes a processing device 11C instead of the processing device 11 and a storage device 12C instead of the storage device 12. FIG. Unlike the storage device 12, the storage device 12C stores a control program PR2C instead of the control program PR2. Unlike the processing device 11, the processing device 11C includes an output section 111C instead of the output section 111. FIG. Otherwise, the configuration of the terminal device 10C is the same as the configuration of the terminal device 10 according to the first embodiment shown in FIG. 4, so illustration and description thereof will be omitted.

The output unit 111C has the same function as the output unit 111 has. Further, the output unit 111C outputs the device ID (identifer) of the terminal device 10, the user name using the terminal device 10, and the location information acquired by the terminal device 10 from the XR glasses 20 to the server 30A. For example, if the terminal device 10 is the terminal device 10-1 used by user U1 who is the first user, the output unit 111C-1 outputs the device ID of the terminal device 10-1, the user name of the terminal device 10-1, and the device ID of the terminal device 10-1. and position information generated by the XR glasses 20 worn on the head of the user U1 are output to the server 30A. The terminal device 10-1 is an example of a first display control device. The output unit 111C also outputs coordinates indicating the display position in the virtual space VS of the virtual object VO displayed in the virtual space VS by the display control unit 114 to the server 30A. A virtual space VS including a virtual object VO is displayed on the XR glasses 20 connected to the terminal device 10-1. XR glasses 20 are an example of a first display device. The virtual space VS displayed on the XR glasses 20 is an example of the first virtual space.

4-1-3: Server Configuration FIG. 16 is a block diagram showing a configuration example of the server 30A. Unlike the server 30, the server 30A includes a processing device 31A instead of the processing device 31 and a storage device 32A instead of the storage device 32. FIG.

Unlike the storage device 32, the storage device 32A stores the control program PR3A instead of the control program PR3. In addition to the components stored by the storage device 32, the storage device 32A also stores a location information database LD.

FIG. 17 is a table showing an example of the location information database LD. As described above, the terminal device 10C provides the server 30A with the device ID of the terminal device 10C, the user name using the terminal device 10, and the location information acquired by the terminal device 10C from the XR glasses 20, or the terminal device 10C outputs the generated location information. The location information database LD stores these device IDs, user names, and location information. As an example, the location information database LD shown in FIG. (x, y, z)=(x _u1 , y _u1 , z _u1 ), where 1 is the positional information acquired from the XR glasses 20, are stored in a mutually linked state. In addition, in the table shown in FIG. 17, "L" is an integer of 2 or more.

Unlike the processing device 31, the processing device 31A includes an acquisition unit 311A instead of the acquisition unit 311 and an output unit 312A instead of the output unit 312. The processing device 31A also includes a determination unit 314 and an extraction unit 315 in addition to the constituent elements included in the processing device 31 .

The determination unit 314 determines whether or not the number of messages output by the output unit 312A to the terminal device 10C-1 is equal to or greater than a predetermined number. When the determination unit 314 determines that the number of the plurality of messages is equal to or greater than the predetermined number, the acquisition unit 311A receives the virtual message displayed in the virtual space VS by the display control unit 114-1 from the terminal device 10C-1. Coordinates indicating the display position of the object VO in the virtual space VS are acquired. Note that the obtaining unit 311A may obtain the coordinates indicating the display position of the individual object SO in addition to the coordinates indicating the display position of the virtual object VO.

By referring to the location information database LD, the extracting unit 315 determines, among the users U who are permitted to share the virtual space VS with the user U1, the location within a predetermined distance from the display position of the virtual object VO in the virtual space VS. to extract users U other than the user U1. The extracted other user U is an example of the second user. The second user is a user U who exists within a predetermined distance from the display position in the virtual space VS and is permitted to share the virtual space VS when the number of messages is equal to or greater than a predetermined number. It should be noted that this "predetermined distance" preferably increases according to the number of messages output to the terminal device 10C-1 by the output unit 312A.

The output unit 312A supplies the terminal device 10C used by the user U extracted by the extraction unit 315 with a plurality of messages identical to the plurality of messages output to the terminal device 10C-1 and the virtual object VO in the virtual space VS. and the coordinates indicating the display position of the . The same multiple messages and the coordinates indicating the display position of the virtual object VO are examples of control information. Also, the terminal device 10C to which the output unit 312A transmits the control information is an example of a second display control device. Further, the XR glass 20 connected to the terminal device 10C or the display 14 provided in the terminal device 10C is an example of a second display device. The terminal device 10C that has obtained the same plurality of messages generates a plurality of individual objects SO corresponding to the same plurality of messages on a one-to-one basis, similar to the terminal device 10C-1. In addition, the terminal device 10C displays the virtual space VS in which the virtual object VO, which is an aggregate of the plurality of individual objects SO, is arranged at the display position obtained from the server 30A. It is displayed on the display 14 provided in the device 10C. The virtual space VS displayed on the XR glasses 20 connected to the terminal device 10C or the display 14 provided in the terminal device 10C is an example of the second virtual space. Here, the same plurality of messages output from the output unit 312A and the coordinates indicating the display position of the virtual object VO are information for displaying the second virtual space including the virtual object VO on the second display device. is. As described above, when the acquiring unit 311A acquires the coordinates indicating the display position of the individual object SO in addition to the coordinates indicating the display position of the virtual object VO, the output unit 312A outputs the coordinates indicating the display position of the individual object SO to the terminal device 10C. On the other hand, it outputs the coordinates indicating the display position of the individual object SO in the virtual space VS. In this case, the terminal device 10C displays the individual object SO at the display position in the virtual space VS obtained from the server 30A.

As a result, in the virtual space VS, when the number of individual objects SO included in the virtual object VO visually recognized by the user U1 is equal to or greater than a predetermined number, the other user U2 can also visually recognize the virtual object VO.

FIG. 18 is an explanatory diagram showing an example of operations of the determination unit 314, the extraction unit 315, and the output unit 312A. It is assumed that a virtual space VS including a virtual object VO4 is displayed on the XR glasses 20 worn on the head by the user U1. As the number of messages addressed to user U1 increases, the number of individual objects SO included in virtual object VO4 also increases. Also, as the number of individual objects SO included in the virtual object VO4 increases, the virtual object VO4 moves away from the user U1. When the determining unit 314 provided in the server 30A determines that the number of messages addressed to the user U1 is equal to or greater than a predetermined number, the extracting unit 315 extracts the display position of the virtual object VO4 in the virtual space VS (x, y, z)=(x ₁₂ , y ₁₂ , z ₁₂ ), extracting users U other than user U1 who are present within a predetermined distance. In the example shown in FIG. 18, it is assumed that the extraction unit 315 has extracted the user U2 located at (x,y,z)=( _xU2 , _yU2 , _zU2 ) as the user U other than the user U1. . The output unit 312A transmits to the terminal device 10C-2 used by the user U2 the same plurality of messages that were transmitted to the terminal device 10C-1. The acquiring unit 112 provided in the terminal device 10C-2 acquires the same plurality of messages and the coordinates indicating the display position of the virtual object VO4 in the virtual space VS from the server 30A. The acquisition unit 112 provided in the terminal device 10C-2 generates a plurality of individual objects SO corresponding to the same plurality of messages on a one-to-one basis. The display control unit 114 provided in the terminal device 10C-2 displays the virtual object VO4, which is an aggregate of the plurality of individual objects SO, at the display position in the virtual space VS acquired from the server 30A (x, y, z)= Display at (x ₁₂ , y ₁₂ , z ₁₂ ). As a result, both the user U1 and the user U2 can visually recognize the virtual object VO9 in the virtual space VS.

4-2: Operation of Fourth Embodiment FIG. 19 is a flow chart showing the operation of the server 30A according to the fourth embodiment. The operation of the server 30A will be described below with reference to FIG.

In step S21, the processing device 31A functions as the output unit 312A. The processing device 31A transmits a plurality of messages to the terminal device 10C-1 used by the user U1. The terminal device 10C-1 causes the XR glasses 20 to display the virtual object VO4 based on the plurality of messages.

In step S22, the processing device 31A functions as the determination unit 314. The processing device 31A determines whether or not the number of messages sent to the terminal device 10C-1 is equal to or greater than a predetermined number. When the determination result is affirmative, that is, when the processing device 31A determines that the number of the plurality of messages is equal to or greater than the predetermined number, the processing device 31A executes the process of step S23. When the determination result is negative, that is, when the processing device 31A determines that the number of the plurality of messages is less than the predetermined number, the processing device 31A executes the process of step S21.

In step S23, the processing device 31A functions as an acquisition unit 311A. The processing device 31A acquires, from the terminal device 10C-1, the coordinates indicating the display position in the virtual space VS of the virtual object VO4 displayed in the virtual space VS by the display control unit 114-1.

In step S24, the processing device 31A functions as the extraction unit 315. The processing device 31A extracts users U other than the user U1 who exist within a predetermined distance from the display position of the virtual object VO4 in the virtual space VS from among the users U who share the virtual space VS with the user U1. Here, as an example, the processing device 31A should extract the user U2.

In step S25, the processing device 31A functions as the output unit 312A. The processing device 31A indicates, to the terminal device 10C-2 used by the user U2, a plurality of messages identical to the plurality of messages output to the terminal device 10C-1, and the display position of the virtual object VO4 in the virtual space VS. Send coordinates and The terminal device 10C-2 that has obtained the same plurality of messages generates a plurality of individual objects SO corresponding to the same plurality of messages on a one-to-one basis, similar to the terminal device 10C-1. Also, the terminal device 10C-2 displays the virtual object VO4, which is an aggregate of the plurality of individual objects SO, at the display position in the virtual space VS obtained from the server 30A.

4-3: Effects of the Fourth Embodiment According to the above description, the server 30A is a server that transmits a plurality of messages to the terminal devices 10 to 10C as the first display control device. be. In this case, the XR glasses 20 as the first display device are worn on the head of the user U1 as the first user. The server 30A includes an acquisition unit 311A and an output unit 312A. The obtaining unit 311A obtains the display position of the virtual object VO4 in the first virtual space VS. When the number of messages is equal to or greater than a predetermined number, the output unit 312A outputs a second message that exists within a predetermined distance from the display position in the first virtual space VS and is allowed to share the virtual object VO. Control information is transmitted to terminal devices 10 to 10C as display control devices. The terminal devices 10 to 10C as the second display control device display the second virtual space on the XR glasses 20 as the second display device worn on the head of the user U2 as the second user. . The above control information is information for displaying the virtual object VO4 in the second virtual space on the second display device.

Since the server 30A has the above configuration, when the number of individual objects SO included in the virtual object VO4 displayed on the XR glasses 20 is equal to or greater than a predetermined number, the other user U2 can also The virtual object VO4 can be visually recognized.

5: Modifications The present disclosure is not limited to the embodiments illustrated above. Specific modification modes are exemplified below. Two or more aspects arbitrarily selected from the following examples may be combined.

5-1: Modification 1
The components and technical features included in each of the information processing system 1 according to the first embodiment to the information processing system 1C according to the fourth embodiment may be combined with each other. For example, the terminal device 10B according to the third embodiment and the server 30A according to the fourth embodiment may be combined. FIG. 20 is an explanatory diagram of the virtual object VO5 generated when the terminal device 10B and the server 30A are combined. For example, the terminal device 10B generates a plurality of individual objects SO based on the cheers of individual spectators in a soccer stadium, and the XR glasses 20 as AR glasses are configured by the plurality of individual objects SO above the soccer stadium. The virtual object VO5 to be displayed may be displayed. The virtual object VO5 is shared by a plurality of spectators wearing XR glasses 20 as AR glasses on their heads. may be arranged in a shape representing In this case, the virtual object VO5 and the virtual space VS, which are composed of individual objects SO corresponding to a plurality of messages on a one-to-one basis, are shared by a plurality of users U who gather at a predetermined location. The predetermined place may be a venue for some event or a public facility such as a school. Furthermore, multiple users U who share the virtual object VO5 and the virtual space VS may participate in the same event. For example, an e-sports tournament corresponds to the event.

In addition to the information processing system 1B according to the third embodiment, the terminal devices 10 to 10C also receive messages from the second user U, in addition to messages addressed to the user U1 and messages generated by the user U1 himself. A message sent to a third user U may be retrieved.

5-2: Modification 2
In the information processing system 1 according to the first embodiment to the information processing system 1C according to the fourth embodiment, the terminal devices 10 to 10C each include a display control section 114 or a display control section 114A. However, instead of the terminal devices 10 to 10C, the servers 30 to 30A may be configured to include the display control unit 114 or the display control unit 114A. Alternatively, the servers 30 to 30A may set coordinates indicating the display positions of the virtual object VO and the individual object SO in the virtual space VS.

5-3: Modification 3
In the information processing system 1</b>C according to the fourth embodiment, the server 30</b>A has a determination section 314 . However, the terminal device 10C may be configured to include the determination unit 314 instead of the server 30A. Specifically, the terminal device 10C determines whether the number of acquired messages or the number of individual objects SO displayed on the XR glasses 20 is equal to or greater than a predetermined number, and sends the determination result to the server. It may be configured to output to 30A.

5-4: Modification 4
In the information processing system 1 according to the first embodiment to the information processing system 1C according to the fourth embodiment, the display control unit 114 or the display control unit 114A provided in the terminal devices 10 to 10C is controlled by the user U when the contents of the message are displayed. The read individual object SO may be erased. In particular, in the server 30A provided in the information processing system 1C according to the fourth embodiment, the determination unit 314 determines whether the number of unread messages is equal to or greater than a predetermined number, not the number of multiple messages output to the terminal device 10C-1. It may be determined whether

5-5: Modification 5
In the information processing system 1 according to the first embodiment to the information processing system 1C according to the fourth embodiment, the acquisition unit 112 provided in the terminal device 10 to the terminal device 10C acquires a plurality of messages from the server 30 or the server 30A. . However, the acquiring unit 112 may acquire only message IDs corresponding to each of the plurality of messages from the server 30 or server 30A. In this case, the generation unit 113 generates individual objects SO that correspond one-to-one with a plurality of message IDs instead of a plurality of messages. Also, in this case, as shown in FIG. 13 as an example, when the user U1 visually recognizes the content of the message corresponding to the individual object SO, the acquiring unit 112 acquires the content of the message from the server 30 or the server 30A for the first time. can be obtained. After that, the display control unit 114 or the display control unit 114A may display the content of the message in the virtual space VS.

5-6: Modification 6
In the information processing system 1 according to the first embodiment to the information processing system 1C according to the fourth embodiment, the individual object SO corresponds to the message generated by the user U on a one-to-one basis. However, the messages to which the individual object SO corresponds are not limited to messages generated by the user U. For example, the message may be a notification to user U generated by an application.

5-7: Modification 7
In the information processing system 1 according to the first embodiment to the information processing system 1C according to the fourth embodiment, the server 30 or the server 30A sends messages stored in the message database MD to the terminal devices 10 to 10C. Output. However, the method of outputting a message from server 30 or server 30A to terminal devices 10 to 10C is not limited to this. For example, the information processing system 1 according to the first embodiment to the information processing system 1C according to the fourth embodiment may further include a content server. More specifically, the server 30 or the server 30A may acquire content including a message for the user U from the content server, and output the acquired content to the terminal devices 10 to 10C.

5-8: Modification 8
In the information processing system 1 according to the first embodiment to the information processing system 1C according to the fourth embodiment, the terminal devices 10 to 10C and the XR glasses 20 are implemented separately. However, the method of realizing the terminal devices 10 to 10C and the XR glasses 20 in the embodiment of the present invention is not limited to this. For example, the XR glasses 20 may have the same functions as the terminal device 10 . In other words, the terminal devices 10 to 10C and the XR glasses 20 may be implemented within a single housing. The same applies to the information processing system 1A according to the second embodiment to the information processing system 1C according to the fourth embodiment.

5-9: Modification 9
The information processing system 1 according to the first embodiment to the information processing system 1C according to the fourth embodiment include, as an example, XR glasses 20 as AR glasses. However, in the information processing systems 1 to 1C, the XR glass 20 includes an HMD (Head Mounted Display) employing VR (Virtual Reality) technology, an HMD employing MR (Mixed Reality) technology, and MR technology. may be provided with any one of the MR glasses employing Alternatively, the information processing system 1 to information processing system 1C may include, instead of the XR glasses 20, one of ordinary smartphones and tablets equipped with imaging devices. These HMDs, MR glasses, smartphones, and tablets are examples of display devices.

6: Others (1) In the above-described embodiments, the storage devices 12 to 12C, 22, and 32 to 32A are examples of ROM and RAM, but flexible disks, magneto-optical disks ( compact discs, digital versatile discs, Blu-ray discs), smart cards, flash memory devices (e.g. cards, sticks, key drives), CD-ROMs (Compact Disc-ROM), registers, removable A disk, hard disk, floppy disk, magnetic strip, database, server or other suitable storage medium. Also, the program may be transmitted from a network via an electric communication line. Also, the program may be transmitted from the communication network NET via an electric communication line.

(2) In the embodiments described above, the information, signals, etc. described may be represented using any of a variety of different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may refer to voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, light fields or photons, or any of these. may be represented by a combination of

(3) In the above-described embodiments, input/output information and the like may be stored in a specific location (for example, memory), or may be managed using a management table. Input/output information and the like can be overwritten, updated, or appended. The output information and the like may be deleted. The entered information and the like may be transmitted to another device.

(4) In the above-described embodiment, the determination may be made by a value (0 or 1) represented using 1 bit, or by a true/false value (Boolean: true or false). Alternatively, it may be performed by numerical comparison (for example, comparison with a predetermined value).

(5) The order of the processing procedures, sequences, flowcharts, etc. exemplified in the above embodiments may be changed as long as there is no contradiction. For example, the methods described in this disclosure present elements of the various steps using a sample order, and are not limited to the specific order presented.

(6) Each function illustrated in FIGS. 1 to 20 is implemented by any combination of at least one of hardware and software. Also, the method of realizing each functional block is not particularly limited. That is, each functional block may be implemented using one device physically or logically coupled, or directly or indirectly using two or more physically or logically separated devices (e.g. , wired, wireless, etc.) and may be implemented using these multiple devices. A functional block may be implemented by combining software in the one device or the plurality of devices.

(7) The programs illustrated in the above embodiments, whether referred to as software, firmware, middleware, microcode, hardware description language or by other names, instructions, instruction sets, code, code shall be interpreted broadly to mean segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executables, threads of execution, procedures, functions, and the like.

In addition, software, instructions, information, etc. may be transmitted and received via a transmission medium. For example, the software uses at least one of wired technology (coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), etc.) and wireless technology (infrared, microwave, etc.) to website, Wired and/or wireless technologies are included within the definition of transmission medium when sent from a server or other remote source.

(8) In each of the above aspects, the terms "system" and "network" are used interchangeably.

(9) Information, parameters, etc. described in this disclosure may be expressed using absolute values, may be expressed using relative values from a predetermined value, or may be expressed using corresponding other information. may be represented as

(10) In the above-described embodiments, the terminal device 10 to terminal device 10C and the server 30 to server 30A may be mobile stations (MS). A mobile station is defined by those skilled in the art as subscriber station, mobile unit, subscriber unit, wireless unit, remote unit, mobile device, wireless device, wireless communication device, remote device, mobile subscriber station, access terminal, mobile terminal, wireless It may also be called a terminal, remote terminal, handset, user agent, mobile client, client, or some other suitable term. Also, in the present disclosure, terms such as "mobile station", "user terminal", "user equipment (UE)", "terminal", etc. may be used interchangeably.

(11) In the above-described embodiments, the terms "connected," "coupled," or any variation thereof refer to any direct or indirect connection between two or more elements. Any connection or coupling is meant, including the presence of one or more intermediate elements between two elements that are "connected" or "coupled" to each other. Couplings or connections between elements may be physical couplings or connections, logical couplings or connections, or a combination thereof. For example, "connection" may be replaced with "access." As used in this disclosure, two elements are defined using at least one of one or more wires, cables, and printed electrical connections and, as some non-limiting and non-exhaustive examples, in the radio frequency domain. , electromagnetic energy having wavelengths in the microwave and optical (both visible and invisible) regions, and the like.

(12) In the above-described embodiments, the phrase "based on" does not mean "based only on," unless expressly specified otherwise. In other words, the phrase "based on" means both "based only on" and "based at least on."

(13) The terms "determining" and "determining" as used in this disclosure may encompass a wide variety of actions. "Judgement" and "determination" are, for example, judging, calculating, computing, processing, deriving, investigating, looking up, searching, inquiring (eg, lookup in a table, database, or other data structure); Also, "judgment" and "determination" are used for receiving (e.g., receiving information), transmitting (e.g., transmitting information), input, output, access (accessing) (for example, accessing data in memory) may include deeming that a "judgment" or "decision" has been made. In addition, "judgment" and "decision" are considered to be "judgment" and "decision" by resolving, selecting, choosing, establishing, comparing, etc. can contain. In other words, "judgment" and "decision" may include considering that some action is "judgment" and "decision". Also, "judgment (decision)" may be replaced by "assuming", "expecting", "considering", and the like.

(14) In the above-described embodiments, where "include," "including," and variations thereof are used, these terms are synonymous with the term "comprising." , is intended to be inclusive. Furthermore, the term "or" as used in this disclosure is not intended to be an exclusive OR.

(15) In this disclosure, where articles have been added by translation, such as a, an, and the in English, the disclosure includes the plural nouns following these articles. good.

(16) In the present disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean that "A and B are different from C". Terms such as "separate," "coupled," etc. may also be interpreted in the same manner as "different."

(17) Each aspect/embodiment described in the present disclosure may be used alone, may be used in combination, or may be used by switching according to execution. In addition, notification of predetermined information (for example, notification of “being X”) is not limited to explicit notification, but is performed implicitly (for example, not notification of the predetermined information). good too.

Although the present disclosure has been described in detail above, it is clear to those skilled in the art that the present disclosure is not limited to the embodiments described in this disclosure. The present disclosure can be practiced with modifications and variations without departing from the spirit and scope of the present disclosure as defined by the claims. Accordingly, the description of the present disclosure is for illustrative purposes and is not meant to be limiting in any way on the present disclosure.

1 to 1C... Information processing system 10 to 10C... Terminal device 11 to 11C... Processing device 12 to 12C... Storage device 13... Communication device 14... Display 15... Input device 16... Inertial sensor 17... Sound collecting device 20 AR glasses 21 processing device 22 storage device 23 line-of-sight detection device 24 GPS device 25 motion detection device 26 imaging device 27 communication device 28

display

30, 30A... server, 31, 31A... processing device, 32, 32A... storage device, 33... communication device, 34... display, 35... input device, 41L, 41R... lens, 91, 92... temple, 93... bridge, 94, 95... frame, 111, 111C... output unit, 112 to 112B... acquisition unit, 113... generation unit, 114, 114A... display control unit, 115... reception unit, 116... determination unit, 117... voice recognition unit, 118 ...

message generation unit

311, 311A ...

acquisition unit

312, 312A ... output unit 313 ... message management unit 314 ... determination unit 315 ... extraction unit MO1 ... message object PR1 to PR3A ... control program SO, SO1 ~ SO10 individual objects, U, U1 to U2 users, VO, VO1 to VO5 virtual objects

Claims

A display control device for displaying a virtual space including a virtual object on a display device worn on a user's head,
an acquisition unit that acquires a plurality of messages;
a generation unit that generates a plurality of individual objects corresponding to the plurality of messages on a one-to-one basis;
a display control unit that causes the display device to display a virtual object that is an aggregate of the plurality of individual objects;
The display control unit
When the number of the plurality of messages is a first number, the size of the virtual object is set to the first size, and in the virtual space, the distance from the center of the virtual space to the center of the virtual object is Let the first distance be
when the number of the plurality of messages is a second number larger than the first number, the size of the virtual object is set to a second size larger than the first size, and in the virtual space, setting a distance from the center of the virtual space to the center of the virtual object as a second distance longer than the first distance;
Display controller.
The display control device according to claim 1, wherein the display control unit changes the display mode of the virtual object according to the number of the plurality of messages.
3. The display control device according to claim 1 or 2, wherein the display control unit makes the display modes of the plurality of individual objects different from each other according to the transmission source device corresponding to each of the plurality of messages.
further comprising a reception unit that receives the user's operation on the virtual object;
4. The display control device according to any one of claims 1 to 3, wherein said display control unit displays a list of said plurality of messages in said virtual space when said operation is a first operation.
further comprising a reception unit that receives the user's operation on the virtual object;
When the operation is a second operation of designating one individual object among the plurality of individual objects, the display control unit causes the content of a message corresponding to the one individual object to be displayed in the virtual space. The display control device according to any one of claims 1 to 3.
further comprising a determination unit that determines the importance of each of the plurality of messages;
The display control unit selects, from among the plurality of individual objects included in the virtual object, a first individual object corresponding to a message having a degree of importance equal to or greater than a first value, 6. The display control device according to any one of claims 1 to 5, which is displayed near the user in comparison with a second individual object corresponding to the message of .
The display control device according to any one of claims 1 to 6, wherein the plurality of messages are generated by the display control device and one or more terminal devices connected to the display control device.
a sound collecting device that collects the voice of the user and outputs an electrical signal representing the voice;
a speech recognition unit that generates text based on the electrical signal output from the sound pickup device;
a message generation unit that generates a message corresponding to the text generated by the speech recognition unit;
8. The display control device according to claim 7, wherein said plurality of messages include messages generated by said message generator.
A server that transmits the plurality of messages to the display control device according to any one of claims 1 to 8,
The display control device is a first display control device, the display device is a first display device worn on the head of the user as a first user, and the virtual space is a first virtual space. ,
an acquisition unit that acquires the display position of the virtual object in the first virtual space from the first display control device;
to a second display control device that exists within a predetermined distance from the display position in the first virtual space and that is allowed to share the virtual object when the number of the plurality of messages is equal to or greater than a predetermined number; and an output unit for transmitting control information,
The second display control device causes a second display device mounted on the head of a second user to display a second virtual space,
The control information is information for causing the second display device to display the virtual object in the second virtual space.
server.