WO2023079875A1 - Information processing device - Google Patents

Information processing device Download PDF

Info

Publication number
WO2023079875A1
WO2023079875A1 PCT/JP2022/036695 JP2022036695W WO2023079875A1 WO 2023079875 A1 WO2023079875 A1 WO 2023079875A1 JP 2022036695 W JP2022036695 W JP 2022036695W WO 2023079875 A1 WO2023079875 A1 WO 2023079875A1
Authority
WO
WIPO (PCT)
Prior art keywords
virtual object
nickname
user
display
virtual
Prior art date
Application number
PCT/JP2022/036695
Other languages
French (fr)
Japanese (ja)
Inventor
有希 中村
康夫 森永
充宏 後藤
達哉 西▲崎▼
怜央 水田
Original Assignee
株式会社Ｎｔｔドコモ
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社Ｎｔｔドコモ filed Critical 株式会社Ｎｔｔドコモ
Publication of WO2023079875A1 publication Critical patent/WO2023079875A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16—Sound input; Sound output
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue

Definitions

  • the present invention relates to an information processing device.
  • AR Augmented Reality
  • the real environment perceived by the user is augmented by a computer.
  • this technology for example, it is possible to superimpose and display a virtual space on a real space visually recognized through AR glasses worn on the user's head.
  • tags are sometimes attached to virtual objects placed in the virtual space.
  • face authentication processing is performed on a captured image of a person captured by a head-mounted display, and the name of the person, which is the face recognition result, is processed for a face image extracted from the captured image. is disclosed as tag information.
  • Patent Document 2 discloses a technology for setting a tag indicating an object included in the captured image. In the technique disclosed in Patent Document 2, it is possible to search for a photographed image in which an object indicated by the tag is captured using the tag.
  • JP 2019-012536 A Japanese Patent No. 6908953
  • the AR technology when a tag is set for a virtual object and a virtual object corresponding to the tag is specified using the tag, as the number of pairs of the virtual object and the tag increases, the user can identify each virtual object. and each tag, and it becomes difficult for the user to memorize the tag itself. As a result, if the user cannot remember the tag corresponding to the virtual object that the user wants to specify, the user cannot easily specify the virtual object.
  • An object of the present invention is to provide an information processing apparatus that can easily remind a user of a tag as a name for specifying a virtual object placed in a virtual space.
  • An information processing apparatus includes a display control unit that displays a plurality of virtual objects arranged in a virtual space on a display device worn on the user's head, and A virtual object specifying unit for specifying a first virtual object among the plurality of virtual objects based on generated instruction information, and a name corresponding to the first virtual object specified by the virtual object specifying unit are stored in a storage device. and a call name specifying unit that specifies the corresponding call name as a first call name when stored, wherein the display control unit causes the display device to display the first call name.
  • FIG. 2 is a perspective view showing the appearance of the AR glasses 20; Schematic diagram of virtual space VS. Schematic diagram of virtual space VS.
  • FIG. 2 is a block diagram showing a configuration example of the AR glasses 20; 2 is a block diagram showing a configuration example of a terminal device 10;
  • FIG. 4 is a diagram showing an example of first information IF1;
  • 3 is a functional block diagram showing the configuration of an identifying unit 113;
  • FIG. 4 is an explanatory diagram of a first operation example of the display control unit 114;
  • FIG. 4 is an explanatory diagram of a first operation example of the display control unit 114;
  • FIG. 9 is an explanatory diagram of a second operation example of the display control unit 114;
  • FIG. 9 is an explanatory diagram of a second operation example of the display control unit 114;
  • FIG. 11 is an explanatory diagram of a third operation example of the display control unit 114;
  • FIG. 11 is an explanatory diagram of a third operation example of the display control unit 114;
  • FIG. 11 is an explanatory diagram of a fourth operation example of the display control unit 114;
  • FIG. 4 is an explanatory diagram of a first operation example of the display control unit 114;
  • FIG. 9 is an explanatory diagram of a second operation example of the display control unit 114;
  • FIG. 11 is an explanatory diagram of a third operation example of the display control unit 114;
  • FIG. 11 is
  • FIG. 11 is an explanatory diagram of a fourth operation example of the display control unit 114; 3 is a block diagram showing a configuration example of a server 30; FIG. 4 is a flowchart showing a first operation of the information processing system 1 according to the first embodiment; 4 is a flowchart showing a second operation of the information processing system 1 according to the first embodiment; 8 is a flowchart showing a third operation of the information processing system 1 according to the first embodiment; 9 is a flowchart showing a fourth operation of the information processing system 1 according to the first embodiment; The figure which shows the whole structure of 1 A of information processing systems which concern on 2nd Embodiment. The block diagram which shows the structural example of 10 A of terminal devices.
  • FIG. 4 is a flowchart showing a first operation of the information processing system 1 according to the first embodiment
  • 4 is a flowchart showing a second operation of the information processing system 1 according to the first embodiment
  • 8 is a flowchart showing a third operation of the information processing system 1 according to the first embodiment
  • FIG. 3 is a functional block diagram showing the configuration of an identifying unit 113A; Explanatory drawing about the 1st operation example of 114 A of display control parts. Explanatory drawing about the 2nd operation example of 114 A of display control parts.
  • FIG. 11 is an explanatory diagram of a third operation example of the display control unit 114A;
  • FIG. 4 is an explanatory diagram showing functions of an updating unit 117;
  • FIG. 4 is an explanatory diagram showing functions of an updating unit 117;
  • 4 is a flowchart for explaining a first operation of the information processing system 1A; 6 is a flowchart for explaining a second operation of the information processing system 1A; A flow chart explaining the 3rd operation of information processing system 1A.
  • FIG. 1 First Embodiment
  • FIG. 1 First Embodiment
  • FIG. 1 the configuration of an information processing system 1 including an information processing apparatus according to a first embodiment of the present invention will be described with reference to FIGS. 1 to 19.
  • FIG. 1 the configuration of an information processing system 1 including an information processing apparatus according to a first embodiment of the present invention will be described with reference to FIGS. 1 to 19.
  • FIG. 1 the configuration of an information processing system 1 including an information processing apparatus according to a first embodiment of the present invention
  • FIG. 1 is a diagram showing the overall configuration of an information processing system 1 according to the first embodiment of the present invention.
  • the information processing system 1 is a system that provides a virtual space using AR technology to a user U1 wearing AR glasses 20, which will be described later.
  • the information processing system 1 includes a terminal device 10, AR glasses 20, and a server 30.
  • the terminal device 10 and the AR glasses 20 are communicably connected to each other.
  • the terminal device 10 and the server 30 are communicably connected to each other via a communication network NET.
  • the terminal device 10 and the AR glasses 20 are combined as a pair of the terminal device 10-1 and the AR glasses 20-1, a pair of the terminal device 10-2 and the AR glasses 20-2, and a terminal device 10-1 and the AR glasses 20-2.
  • a total of three pairs of devices 10-3 and AR glasses 20-3 are described.
  • the number of sets is merely an example, and the information processing system 1 can include any number of sets of terminal devices 10 and AR glasses 20 .
  • the terminal device 10 is an example of an information processing device.
  • the terminal device 10 is a device for displaying a virtual object arranged in a virtual space on the AR glasses 20 worn on the user's head.
  • the virtual space is, for example, a celestial space.
  • the virtual objects are, for example, virtual objects representing data such as still images, moving images, 3DCG models, HTML files, and text files, and virtual objects representing applications. Examples of text files include memos, source codes, diaries, and recipes. Examples of applications include browsers, applications for using SNS, and applications for generating document files.
  • the terminal device 10 is preferably a mobile terminal device such as a smart phone and a tablet, for example.
  • the AR glasses 20 are see-through wearable displays worn on the user's head. Under the control of the terminal device 10, the AR glasses 20 display a virtual object on the display panel provided for each of the binocular lenses. Note that the AR glasses 20 are an example of a display device.
  • the server 30 provides various data and cloud services to the terminal device 10 via the communication network NET.
  • FIG. 2 is a perspective view showing the appearance of the AR glasses 20.
  • the AR glasses 20 have temples 91 and 92, a bridge 93, trunks 94 and 95, and lenses 41L and 41R, like common spectacles.
  • An imaging device 27 is provided in the bridge 93 .
  • the imaging device 27 captures an image of the outside world and outputs imaging data representing the captured image.
  • each of the temples 91 and 92 is provided with a sound pickup device 24 that picks up sound.
  • the sound collection device 24 outputs sound data representing the collected sound. Note that the position of the sound pickup device 24 is not limited to the temples 91 and 92, and may be, for example, the bridge 93 and any one of the trunks 94 and 95.
  • Each of the lenses 41L and 41R has a half mirror.
  • the body portion 94 is provided with a liquid crystal panel or an organic EL panel for the left eye (hereinafter collectively referred to as a display panel) and an optical member for guiding light emitted from the display panel for the left eye to the lens 41L. .
  • the half mirror provided in the lens 41L transmits external light and guides it to the left eye, and reflects the light guided by the optical member to enter the left eye.
  • the body portion 95 is provided with a right-eye display panel and an optical member that guides light emitted from the right-eye display panel to the lens 41R.
  • the half mirror provided in the lens 41R transmits external light and guides it to the right eye, and reflects the light guided by the optical member to enter the right eye.
  • the display 29, which will be described later, includes a lens 41L, a left-eye display panel, a left-eye optical member, and a lens 41R, a right-eye display panel, and a right-eye optical member.
  • the user can observe the image on the display panel superimposed on the state of the outside world.
  • the image for the left eye is displayed on the display panel for the left eye
  • the image for the right eye is displayed on the display panel for the right eye.
  • binocular parallax the user U1 can perceive the displayed image as if it had depth and stereoscopic effect.
  • FIG. 3 and 4 are schematic diagrams of the virtual space VS provided to the user U1 by using the AR glasses 20.
  • FIG. As shown in FIG. 3, in the virtual space VS, virtual objects VO1 to VO5 representing various contents such as browsers, cloud services, images, and moving images are arranged.
  • the user U1 walks around the public space while wearing the AR glasses 20 on which the virtual objects VO1 to VO5 arranged in the virtual space VS are displayed. It becomes possible to experience the space VS.
  • the user U1 can act in the public space while receiving benefits brought about by the virtual objects VO1 to VO5 placed in the virtual space VS.
  • a plurality of users U1 to U3 can share the virtual space VS.
  • the plurality of users U1-U3 share one or a plurality of virtual objects VO, and the users U1-U3 can communicate with each other.
  • FIG. 5 is a block diagram showing a configuration example of the AR glasses 20.
  • the AR glasses 20 include a processing device 21 , a storage device 22 , a line-of-sight detection device 23 , a sound collection device 24 , a GPS device 25 , a motion detection device 26 , an imaging device 27 , a communication device 28 and a display 29 .
  • Each element of the AR glasses 20 is interconnected by one or more buses for communicating information.
  • the processing device 21 is a processor that controls the entire AR glasses 20, and is configured using, for example, one or more chips.
  • the processing device 21 is configured using, for example, a central processing unit (CPU) including an interface with peripheral devices, an arithmetic device, registers, and the like.
  • CPU central processing unit
  • Some or all of the functions of the processing device 21 are realized by hardware such as DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), PLD (Programmable Logic Device), FPGA (Field Programmable Gate Array), and the like.
  • DSP Digital Signal Processor
  • ASIC Application Specific Integrated Circuit
  • PLD Programmable Logic Device
  • FPGA Field Programmable Gate Array
  • the processing device 21 executes various processes in parallel or sequentially.
  • the storage device 22 is a recording medium readable and writable by the processing device 21, and stores a plurality of programs including the control program PR1 executed by the processing device 21.
  • the line-of-sight detection device 23 detects the line of sight of the user U1, and outputs line-of-sight data indicating the direction of the line of sight of the user U1 to the processing device 21, which will be described later.
  • the line-of-sight detection device 23 may detect the line-of-sight by any method. For example, line-of-sight data may be detected based on the position of the inner corner of the eye and the position of the iris.
  • the sound collection device 24 collects sound and outputs sound data based on the collected sound to the processing device 21, which will be described later.
  • the GPS device 25 receives radio waves from multiple satellites and generates position data from the received radio waves.
  • the position data indicates the position of the AR glasses 20.
  • the location data may be in any format as long as the location can be specified.
  • the position data indicates the latitude and longitude of the AR glasses 20, for example.
  • position data is obtained from GPS device 25 .
  • the AR glasses 20 may acquire position data by any method.
  • the acquired position data is output to the processing device 21 .
  • the motion detection device 26 detects motion of the AR glasses 20 and outputs motion data to the processing device 21 .
  • Examples of the motion detection device 26 include inertial sensors such as an acceleration sensor that detects acceleration and a gyro sensor that detects angular acceleration.
  • the acceleration sensor detects acceleration in orthogonal X-, Y-, and Z-axes.
  • the gyro sensor detects angular acceleration around the X-, Y-, and Z-axes.
  • the motion detection device 26 can generate orientation information indicating the orientation of the AR glasses 20 based on the output information of the gyro sensor.
  • the motion data includes acceleration data respectively indicating three-axis accelerations and angular acceleration data respectively indicating three-axis angular accelerations.
  • the imaging device 27 outputs imaging data obtained by imaging the outside world.
  • the imaging device 27 includes, for example, a lens, an imaging element, an amplifier, and an AD converter.
  • the light condensed through the lens is converted into an image pickup signal, which is an analog signal, by the image pickup device.
  • the amplifier amplifies the imaging signal and outputs it to the AD converter.
  • the AD converter converts the amplified imaging signal, which is an analog signal, into imaging data, which is a digital signal.
  • the converted imaging data is output to the processing device 21 .
  • the imaging data output to the processing device 21 is output to the terminal device 10 via the communication device 28 .
  • the terminal device 10 recognizes various gestures of the user U1 based on the imaging data, and controls the terminal device 10 according to the recognized gestures. That is, the imaging device 27 functions as an input device for inputting instructions from the user U1, like a pointing device and a touch panel.
  • the communication device 28 is hardware as a transmission/reception device for communicating with other devices.
  • the communication device 28 is also called, for example, a network device, a network controller, a network card, a communication module, or the like.
  • the communication device 28 may include a connector for wired connection and an interface circuit corresponding to the connector.
  • the communication device 28 may also have a wireless communication interface. Products conforming to wired LAN, IEEE1394, or USB can be used as connectors and interface circuits for wired connection. Also, as a wireless communication interface, there are products conforming to wireless LAN, Bluetooth (registered trademark), and the like.
  • the display 29 is a device that displays images.
  • the display 29 displays various images under the control of the processing device 21 .
  • the display 29 includes the lens 41L, the left-eye display panel, the left-eye optical member, and the lens 41R, the right-eye display panel, and the right-eye optical member, as described above.
  • Various display panels such as a liquid crystal display panel and an organic EL display panel are preferably used as the display panel.
  • the processing device 21 functions as an acquisition unit 211 and a display control unit 212, for example, by reading the control program PR1 from the storage device 22 and executing it.
  • the acquisition unit 211 acquires the control signal from the terminal device 10 . More specifically, the acquisition unit 211 acquires a control signal for controlling display on the AR glasses 20 generated by a display control unit 114 provided in the terminal device 10 and described later.
  • the acquisition unit 211 also receives line-of-sight data input from the line-of-sight detection device 23, audio data input from the sound collection device 24, position data input from the GPS device 25, motion data input from the motion detection device 26, and acquires imaging data input from the imaging device 27 . After that, the acquisition unit 211 outputs the acquired line-of-sight data, audio data, position data, motion data, and imaging data to the communication device 28 .
  • the display control unit 212 controls display on the display 29 based on the control signal from the terminal device 10 acquired by the acquisition unit 211 .
  • FIG. 6 is a block diagram showing a configuration example of the terminal device 10. As shown in FIG.
  • the terminal device 10 includes a processing device 11 , a storage device 12 , a communication device 13 , a display 14 , an input device 15 and an inertial sensor 16 .
  • the elements of terminal 10 are interconnected by a bus or buses for communicating information. Note that the term "apparatus" in this specification may be replaced with another term such as a circuit, a device, or a unit.
  • the processing device 11 is a processor that controls the entire terminal device 10, and is configured using, for example, one or more chips.
  • the processing unit 11 is configured using, for example, a central processing unit (CPU) including interfaces with peripheral devices, arithmetic units, registers, and the like.
  • CPU central processing unit
  • a part or all of the functions of the processing device 11 may be realized by hardware such as DSP, ASIC, PLD, and FPGA.
  • the processing device 11 executes various processes in parallel or sequentially.
  • the storage device 12 is a recording medium readable and writable by the processing device 11, and stores a plurality of programs including a control program PR2 executed by the processing device 11, first information IF1, and second information IF2.
  • FIG. 7 is a diagram showing an example of the first information IF1.
  • the first information IF1 is tabular information.
  • the first information IF1 includes identification information that uniquely identifies the virtual object VO, a tag TG corresponding to the virtual object VO, position information indicating the position of the virtual object VO in the celestial virtual space VS, and an image of each virtual object VO. Match information.
  • the identification information is hereinafter referred to as ID.
  • the position information is three-dimensional coordinates in the virtual space VS. If each virtual object VO does not have a tag TG corresponding to itself, the tag TG column is blank.
  • the plurality of virtual objects VO arranged in the virtual space VS are associated one-to-one with the plurality of tags TG.
  • the tag TG is an example of the name of the virtual object VO.
  • the plurality of virtual objects VO arranged in the virtual space VS are the plurality of virtual objects VO that can be visually recognized by the user U1 changing the posture.
  • the virtual object VO can be placed in the virtual space VS based on an instruction from the user U1.
  • the virtual object VO may be arranged in the virtual space VS when a predetermined condition is satisfied without an instruction from the user U1. For example, K virtual objects VO out of J virtual objects VO are arranged in the virtual space VS.
  • the first information IF1 is information about K virtual objects VO. Note that the first information IF ⁇ b>1 may be acquired from the server 30 via the communication device 13 .
  • FIG. 8 is a diagram showing an example of the second information IF2.
  • the second information IF2 is tabular information.
  • the second information IF2 associates the ID of the virtual object VO arranged in the virtual space VS with the attributes of the virtual object VO.
  • the second information IF2 is information about K virtual objects VO.
  • the second information IF2 may be acquired from the server 30 via the communication device 13 .
  • the "attribute" is an item that classifies the content of each virtual object VO according to its features or properties.
  • the virtual object VO includes data such as still images, moving images, 3DCG models, HTML files and text files, and applications.
  • the communication device 13 is hardware as a transmission/reception device for communicating with other devices.
  • the communication device 13 is also called, for example, a network device, a network controller, a network card, or a communication module.
  • the communication device 13 may include a connector for wired connection and an interface circuit corresponding to the connector. Further, the communication device 13 may have a wireless communication interface. Products conforming to wired LAN, IEEE1394, or USB can be used as connectors and interface circuits for wired connection. Also, as a wireless communication interface, there are products conforming to wireless LAN, Bluetooth (registered trademark), and the like.
  • the display 14 is a device that displays images and character information.
  • the display 14 displays various images under the control of the processing device 11 .
  • various display panels such as a liquid crystal display panel and an organic EL (Electro Luminescence) display panel are preferably used as the display 14 .
  • the input device 15 accepts operations from the user U1 wearing the AR glasses 20 on his head.
  • the input device 15 includes a pointing device such as a keyboard, touch pad, touch panel, or mouse.
  • the input device 15 may also serve as the display 14 .
  • the inertial sensor 16 is a sensor that detects inertial force.
  • the inertial sensor 16 includes, for example, one or more of an acceleration sensor, an angular velocity sensor, and a gyro sensor.
  • the processing device 11 detects the orientation of the terminal device 10 based on the output information from the inertial sensor 16 . Further, the processing device 11 receives selection of the virtual object VO, input of characters, and input of instructions in the celestial sphere virtual space VS based on the orientation of the terminal device 10 .
  • the user U1 directs the central axis of the terminal device 10 toward a predetermined area of the virtual space VS, and operates the input device 15 to select the virtual object VO arranged in the predetermined area.
  • the user U1's operation on the input device 15 is, for example, a double tap. By operating the terminal device 10 in this way, the user U1 can select the virtual object VO without looking at the input device 15 of the terminal device 10 .
  • characters are input by operating the input device 15 with the central axis of the terminal device 10 facing the key that the user U1 wants to input. Further, for example, when the user U1 presses down the input device 15 and moves the terminal device 10 left or right, a predetermined instruction is input.
  • the terminal device 10 functions as a portable controller that controls the virtual space VS.
  • the processing device 11 functions as an acquisition unit 111, an action recognition unit 112, an identification unit 113, a display control unit 114, a determination unit 115, and a voice recognition unit 116 by reading and executing the control program PR2 from the storage device 12. .
  • the acquisition unit 111 acquires instruction information according to the motion of the user U1 wearing the AR glasses 20 on the head.
  • the instruction information is information that designates a specific virtual object VO.
  • the operation of the user U1 is, for example, inputting to the terminal device 10 by the user U1 using the input device 15 . More specifically, the action of the user U1 may be pressing of a specific part as the input device 15 provided in the terminal device 10 . Alternatively, the user U1's action may be an operation using the terminal device 10 as a portable controller.
  • the action of the user U1 may be visual observation of the AR glasses 20 by the user U1. If the action of the user U1 is visual observation, the instruction information is the viewpoint of the user U1 on the AR glasses 20 . In this case, the instruction information is transmitted from the AR glasses 20 to the terminal device 10 .
  • the action of user U1 may be a gesture of user U1.
  • the action recognition unit 112 recognizes various gestures of the user U1.
  • the acquisition unit 111 may acquire instruction information according to various gestures of the user U1.
  • the acquisition unit 111 acquires the first information IF1 and the second information IF2 from the server 30 by using the communication device 13 .
  • the acquisition unit 111 stores the acquired first information IF1 and second information IF2 in the storage device 12 . Furthermore, the acquisition unit 111 acquires the first information IF1 and the second information IF2 from the storage device 12 .
  • the motion recognition unit 112 recognizes various gestures of the user U1 based on the imaging data obtained from the AR glasses 20. More specifically, as described above, the imaging device 27 provided in the AR glasses 20 outputs imaging data obtained by imaging the outside world. When the imaging data includes a part of the body of the user U1 wearing the AR glasses 20 on the head, the action recognition unit 112 performs various actions of the user U1 based on the imaging data acquired from the AR glasses 20. gestures.
  • the identification unit 113 includes a virtual object identification unit 113-1 and a nickname identification unit 113-2.
  • the virtual object specifying unit 113-1 specifies one virtual object VO among a plurality of virtual objects VO arranged in the virtual space VS.
  • One virtual object VO specified by the virtual object specifying unit 113-1 is hereinafter referred to as a first virtual object VO.
  • the nickname specifying unit 113-2 specifies the one tag TG.
  • One tag TG is an example of a first nickname.
  • the nickname identifying unit 113-2 identifies the tag TG corresponding to the first virtual object VO by referring to the first information IF1. In other words, when the tag TG corresponding to the first virtual object VO is stored in the storage device 12, the nickname identifying unit 113-2 identifies the corresponding tag TG as the first nickname.
  • the nickname identifying unit 113-2 identifies the tag TG of "#FOX”.
  • Figs. 10A to 10C are explanatory diagrams of how to use the tag TG.
  • virtual objects VO6 to VO8 representing a deer, a fox, and a horse are arranged in a virtual space VS perceived by the user U1.
  • the character string "#FOX" is registered as the tag TG7 in the fox virtual object VO7.
  • the user U1 utters a character string indicated by a tag TG corresponding to each content while pressing a specific portion of the input device 15 provided in the terminal device 10.
  • the display control unit 114 causes the AR glasses 20 as a display device to display a plurality of virtual objects VO placed in the virtual space VS.
  • the display control unit 114 also causes the AR glasses 20 to display the tag TG specified by the name specifying unit 113-2. More specifically, the display control unit 114 generates image data to be displayed on the AR glasses 20 and transmits the generated image data to the AR glasses 20 via the communication device 13 .
  • FIG. 11A and 11B are explanatory diagrams of a first operation example of the display control unit 114.
  • the acquisition unit 111 acquires an operation signal generated by an operation on the input device 15 by the user U1 as a trigger.
  • the display control unit 114 divides the celestial sphere-shaped virtual space VS into a plurality of regions R1 to R17 by a plurality of straight lines corresponding to the latitude and longitude lines of the celestial sphere.
  • the display control unit 114 planarizes each of the plurality of divided regions R1 to R17. Furthermore, as shown in FIG.
  • the display control unit 114 causes the display 29 of the AR glasses 20 to display a two-dimensional image SI obtained by planarizing the plurality of flattened regions R1 to R17. Specifically, the display control unit 114 displays regions R5 and R6 located on the right side of the zenith region R17 as viewed from the user U1 located in the center of the celestial space in the two-dimensional image SI. It is arranged to the right of the region R17. In addition, the display control unit 114 displays regions R13 and R14 positioned to the left of the zenith region R17 as viewed from the user U1 who is positioned in the center of the celestial sphere space in the two-dimensional image SI. Place on the left.
  • the display control unit 114 displays the regions R9 and R10 positioned in front of the zenith region R17 as viewed from the user U1 positioned in the center of the celestial sphere as follows in the two-dimensional image SI below the region R17. to be placed.
  • the display control unit 114 causes the regions R1 and R2 positioned behind the zenith region R17 as viewed from the user U1 positioned at the center of the celestial space to be displayed above the region R17 in the two-dimensional image SI. to be placed.
  • the display control unit 114 displays regions R7 and R8, which are positioned to the right and front of the zenith region R17 as viewed from the user U1 who is positioned at the center of the celestial space, in the two-dimensional image SI. Place it in the lower right corner.
  • the display control unit 114 causes the regions R11 and R12, which are positioned in front left of the zenith region R17 as viewed from the user U1 positioned in the center of the celestial space, to be displayed in the two-dimensional image SI as the region R17. Place it in the lower left corner.
  • the display control unit 114 causes the regions R3 and R4 positioned to the right rear of the zenith region R17 as viewed from the user U1 positioned in the center of the celestial space to be displayed as the regions R17 in the two-dimensional image SI. Place it in the upper right corner.
  • the display control unit 114 displays regions R15 and R16 positioned to the left rear of the zenith region R17 as viewed from the user U1 positioned at the center of the celestial sphere space in the two-dimensional image SI. Place it on the upper left. Furthermore, as shown in FIG.
  • the vertical and horizontal directions of each of the regions R1 to R17 are the same as those viewed from the user U1 when the user U1, who is positioned in the center of the celestial space, faces the regions R1 to R17. Matches up, down, left, and right.
  • the display control unit 114 causes the display 29 of the AR glasses 20 to be adjacent to the region R containing the virtual object VO specified by the virtual object specifying unit 113-1, and the virtual object VO specified by the name specifying unit 113-2. Display the specified tag TG.
  • the display control unit 114 causes the tag specified by the nickname specifying unit 113-2 to be adjacent to the region R3 containing the virtual object VO7 specified by the virtual object specifying unit 113-1. Display TG7.
  • the display control unit 114 planarly develops the plurality of regions R1 to R17 using an operation signal corresponding to the operation of the input device 15 by the user U1 as a trigger.
  • the operation signal that triggers planar development is not limited to being generated in response to an operation on the input device 15 .
  • the processing device 11 functions as the motion recognition unit 112 .
  • the operation signal may be generated according to the gesture of the user U1 detected by the processing device 11 functioning as the motion recognition unit 112 .
  • the above operation signal may be generated according to the attitude of the terminal device 10 .
  • FIG. 12A and 12B are explanatory diagrams of a second operation example of the display control unit 114.
  • FIG. 12A during normal operation, the display control unit 114 causes the AR glasses 20 worn on the head of the user U1 to have a celestial virtual space VS in which the user U1 is positioned at the nadir. display on the display 29. 12B, the display control unit 114 controls the display 29 of the AR glasses 20 based on the operation signal generated by the operation of the input device 15 by the user U1. is displayed as a reduced three-dimensional image TI.
  • the display control unit 114 causes the virtual object VO specified by the virtual object specifying unit 113-1 and the tag TG corresponding to the virtual object VO to be displayed adjacent to each other on the three-dimensional image TI.
  • the display control unit 114 causes the virtual object VO7 specified by the virtual object specifying unit 113-1 and the tag TG7 corresponding to the virtual object VO7 to be adjacent to each other on the three-dimensional image TI. display.
  • the display control unit 114 causes the display 29 to display the three-dimensional image TI based on the operation signal generated by the operation of the input device 15 by the user U1.
  • the operation signal that triggers the display of the three-dimensional image TI is not limited to being generated in response to the operation on the input device 15 .
  • the above operation signal may be generated according to a gesture of the user U1 detected by the processing device 11 functioning as the action recognition unit 112 .
  • the above operation signal may be generated according to the attitude of the terminal device 10 .
  • the determination unit 115 determines whether or not the viewpoint of the user U1 is positioned within a specific virtual object VO displayed on the AR glasses 20 as a display device for a predetermined time or longer.
  • the AR glasses 20 are equipped with the line-of-sight detection device 23.
  • the line-of-sight detection device 23 detects line-of-sight data based on, for example, the position of the inner corner of the eye and the position of the iris of the user U1.
  • the determination unit 115 determines the coordinates of the point where the line-of-sight direction indicated by the line-of-sight data collides with the spherical surface of the celestial sphere as the virtual space VS as the viewpoint position of the user U1.
  • the first information IF1 includes position information of each first virtual object VO.
  • the first virtual object VO is arranged at the position indicated by the position information.
  • the determination unit 115 determines whether or not the viewpoint of the user U1 is located within the area where the first virtual object VO is arranged for a predetermined time or longer.
  • FIGS. 13A and 13B are explanatory diagrams of a third operation example of the display control unit 114 especially in cooperation with the determination unit 115.
  • FIG. 13A in the virtual space VS, when it is determined that the viewpoint of the user U1 is positioned within the specific virtual object VO for a predetermined time or longer, the display control unit 114 performs the following operations as shown in FIG. 13B.
  • the tag TG corresponding to the virtual object VO is displayed near the virtual object VO.
  • the “neighborhood” of the virtual object VO specifically means a range within a predetermined distance from the virtual object VO. In the example shown in FIGS.
  • the display control unit 114 displays a "#FOX", which is the tag TG7 corresponding to the virtual object VO7, is displayed.
  • the speech recognition unit 116 recognizes the speech uttered by the user U1.
  • the AR glasses 20 are equipped with the sound pickup device 24.
  • a sound uttered by the user U1 wearing the AR glasses 20 on the head is picked up by the sound pickup device 24 and converted into sound data.
  • Audio data is output from the AR glasses 20 to the terminal device 10 .
  • the voice recognition unit 116 recognizes the contents of the utterance based on the voice data acquired from the AR glasses 20 . More specifically, speech recognition unit 116 converts speech data into text data.
  • FIGS. 14A and 14B are explanatory diagrams of a fourth operation example of the display control unit 114 especially in cooperation with the speech recognition unit 116.
  • FIG. 14A when the user U1 wearing the AR glasses 20 on his or her head utters the attribute of the virtual object VO, the voice recognition unit 116 recognizes the voice data representing the voice uttered by the user U1. do.
  • the recognition result by the speech recognition unit 116 indicates one attribute included in the second information IF2
  • the virtual object specifying unit 113-1 generates one or more attributes corresponding to the one attribute based on the second information IF2. to identify the virtual object VO.
  • the voice recognition unit 116 recognizes the voice uttered by the user U1 as the character string "animal".
  • the virtual objects VO corresponding to the recognition result "animal" are a virtual object VO6 that is a 3D model of a deer, a virtual object VO7 that is a 3D model of a fox, and a virtual object VO8 that is a 3D model of a horse. Therefore, the virtual object specifying unit 113-1 specifies virtual objects VO6 to VO8 as virtual objects VO corresponding to the attribute "animal" among the plurality of virtual objects VO.
  • the call name specifying unit 113-2 provides one virtual object VO corresponding to each of a part or all of the one or more virtual objects VO specified by the virtual object specifying unit 113-1. Identify the above tag TG.
  • examples of one or more virtual objects VO identified by the virtual object identifying unit 113-1 include a virtual object VO6 that is a 3D model of a deer, a virtual object VO7 that is a 3D model of a fox, and a virtual object VO7 that is a 3D model of a fox.
  • a virtual object VO8, which is a 3D model of a horse, can be mentioned.
  • the one or more tags TG corresponding to each of the partial virtual objects VO of the identified one or more virtual objects VO are divided into two tags TG, a tag TG7 of "#FOX” and a tag TG8 of "#HORSE". becomes. Therefore, the calling name identifying unit 113-2 identifies two tags TG, ie, the tag TG7 of "#FOX” and the tag TG8 of "#HORSE". On the other hand, if "#DEER" is assigned as the tag TG6 corresponding to the virtual object VO6, which is a 3D model of a deer, the calling name identifying unit 113-2 is identified by the virtual object identifying unit 113-1. In addition, one or more tags TG corresponding to each of all virtual objects VO of one or more virtual objects VO are specified.
  • the display control unit 114 displays one or more virtual objects VO specified by the virtual object specifying unit 113-1 based on the second information IF2, as shown in FIG. 14B, on the AR glasses 20 as the display device. display. Further, when a tag TG corresponding to the virtual object VO exists, the display control unit 114 displays the tag TG in association with the corresponding virtual object VO.
  • icons IC6 to IC8 indicating virtual objects VO6 to VO8 identified by the virtual object identifying unit 113-1 are displayed in the popup P1. Further, in the popup P1, the tag TG7 of "#FOX” is added to the icon IC7, and the tag TG8 of "#HORSE” is added to the icon IC8.
  • the display control unit 114 changes the display content regarding the first virtual object VO corresponding to the tag TG. .
  • the display control unit 114 may move the virtual object VO corresponding to the tag TG in the line-of-sight direction of the user U1.
  • the display control unit 114 may display a screen for selecting whether to start the application.
  • the terminal device 10 may include a GPS device similar to the GPS device 25 provided in the AR glasses 20. In this case, the AR glasses 20 do not have to be equipped with the GPS device 25 .
  • FIG. 15 is a block diagram showing a configuration example of the server 30.
  • the server 30 comprises a processing device 31 , a storage device 32 , a communication device 33 , a display 34 and an input device 35 .
  • Each element of server 30 is interconnected by a bus or buses for communicating information.
  • the processing device 31 is a processor that controls the entire server 30, and is configured using, for example, one or more chips.
  • the processing unit 31 is configured using, for example, a central processing unit (CPU) including interfaces with peripheral devices, arithmetic units, registers, and the like.
  • CPU central processing unit
  • a part or all of the functions of the processing device 31 may be realized by hardware such as DSP, ASIC, PLD, or FPGA.
  • the processing device 31 executes various processes in parallel or sequentially.
  • the storage device 32 is a recording medium readable and writable by the processing device 31, and stores a plurality of programs including the control program PR3 executed by the processing device 31, first information IF1, and second information IF2.
  • the communication device 33 is hardware as a transmission/reception device for communicating with other devices.
  • the communication device 33 is also called, for example, a network device, a network controller, a network card, or a communication module.
  • the communication device 33 may include a connector for wired connection and an interface circuit corresponding to the connector. Further, the communication device 33 may have a wireless communication interface. Products conforming to wired LAN, IEEE1394, and USB are examples of connectors and interface circuits for wired connection. Also, as a wireless communication interface, there are products conforming to wireless LAN, Bluetooth (registered trademark), and the like.
  • the display 34 is a device that displays images and character information.
  • the display 34 displays various images under the control of the processing device 31 .
  • various display panels such as a liquid crystal display panel and an organic EL display panel are preferably used as the display 34 .
  • the input device 35 is a device that accepts operations by the administrator of the information processing system 1 .
  • the input device 35 includes a pointing device such as a keyboard, touch pad, touch panel, or mouse.
  • the input device 35 may also serve as the display 34 .
  • the administrator of the information processing system 1 can use the input device 35 to input and edit the first information IF1 and the second information IF2.
  • the processing device 31 functions as an output unit 311 and an acquisition unit 312 by reading and executing the control program PR3 from the storage device 32, for example.
  • the output unit 311 outputs the first information IF1 and the second information IF2 stored in the storage device 32 to the terminal device 10 by using the communication device 33 .
  • the output unit 311 outputs to the terminal device 10 data necessary for the terminal device 10 to provide the virtual space VS to the user U1 wearing the AR glasses 20 on the head.
  • the data includes data related to the virtual object VO itself and data related to an application (not shown) for using the cloud service.
  • the acquisition unit 312 acquires various data from the terminal device 10 by using the communication device 33 .
  • the data includes, for example, data indicating the operation content for the virtual object VO, which is input to the terminal device 10 by the user U1 wearing the AR glasses 20 on the head. Further, when the user U1 uses the above cloud service, the data includes input data to the above application.
  • FIGS. 16 to 19 are flowcharts showing the operation of the information processing system 1 according to the first embodiment. The operation of the information processing system 1 will be described below with reference to FIGS. 16 to 19. FIG.
  • FIG. 16 is a flow chart explaining the first operation of the information processing system 1 .
  • step S ⁇ b>1 the processing device 11 acquires an operation signal by functioning as the acquisition unit 111 .
  • the operation signal is a trigger for causing the display 29 provided in the AR glasses 20 to display a two-dimensional image SI obtained by planarly developing the celestial virtual space VS. is.
  • step S2 the processing device 11 functions as the display control unit 114 to cause the display 29 provided in the AR glasses 20 to display a two-dimensional image SI obtained by planarly developing the celestial virtual space VS.
  • step S3 the processing device 11 functions as the acquisition unit 111 to acquire instruction information.
  • the instruction information is information for specifying the first virtual object VO among the plurality of virtual objects VO arranged in the virtual space VS.
  • step S4 the processing device 11, by functioning as the virtual object specifying unit 113-1, specifies the first virtual object VO among the plurality of virtual objects VO based on the instruction information. Further, the processing device 11 functions as the virtual object specifying unit 113-1, and as a result of specifying the first virtual object VO, an ID corresponding to the first virtual object VO is output.
  • step S5 the processing device 11 refers to the first information IF1 stored in the storage device 12 by functioning as the calling name identification unit 113-2. After that, the processing device 11 functions as the call name identifying unit 113-2 so that the tag TG corresponding to the first virtual object VO identified by functioning as the virtual object identifying unit 113-1 is changed to the first virtual object VO. 1 information IF1. Specifically, the processing device 11 functions as the calling name identification unit 113-2 to determine whether the tag TG corresponding to the ID output in step S4 is included in the first information IF1. If the tag TG is included in the first information IF1, that is, if the determination result in step S5 is YES, the processing device 11 functions as the nickname specifying unit 113-2 to specify the tag TG as the first nickname. After that, the process of step S6 is executed. If the tag TG is not included in the first information IF1, that is, if the determination result in step S5 is NO, the processing device 11 ends all the processes.
  • step S6 the processing device 11 functions as the display control unit 114 to display the tag TG specified by the name specifying unit 113-2 on the display 29 of the AR glasses 20. More specifically, the processing device 11 functions as the display control unit 114 to display the specified tag TG in the two-dimensional image SI displayed on the display 29 . Note that if the tag TG is not specified in step S5, the processing device 11 omits the processing of step S6.
  • FIG. 17 is a flow chart explaining the second operation of the information processing system 1 .
  • step S ⁇ b>11 the processing device 11 acquires the operation signal by functioning as the acquisition unit 111 .
  • the operation signal is a trigger for causing the display 29 provided in the AR glasses 20 to display a three-dimensional image TI obtained by reducing the celestial sphere virtual space VS. .
  • step S12 the processing device 11 functions as the display control unit 114 to cause the display 29 provided in the AR glasses 20 to display a three-dimensional image TI obtained by reducing the celestial sphere-shaped virtual space VS.
  • step S ⁇ b>13 the processing device 11 acquires the instruction information by functioning as the acquisition unit 111 .
  • the instruction information is information for specifying the first virtual object VO among the plurality of virtual objects VO arranged in the virtual space VS.
  • step S14 the processing device 11, by functioning as the virtual object specifying unit 113-1, specifies the first virtual object VO among the plurality of virtual objects VO based on the instruction information. Further, the processing device 11 functions as the virtual object specifying unit 113-1, and as a result of specifying the first virtual object VO, an ID corresponding to the first virtual object VO is output.
  • step S15 the processing device 11 refers to the first information IF1 stored in the storage device 12 by functioning as the calling name identification unit 113-2. After that, the processing device 11 functions as the call name identifying unit 113-2 so that the tag TG corresponding to the first virtual object VO identified by functioning as the virtual object identifying unit 113-1 is changed to the first virtual object VO. 1 information IF1. Specifically, the processing device 11 functions as the nickname identification unit 113-2 to determine whether the tag TG corresponding to the ID output in step S14 is included in the first information IF1. If the tag TG is included in the first information IF1, that is, if the determination result in step S15 is YES, the processing device 11 functions as the nickname identification unit 113-2 to identify the tag TG as the first nickname. After that, the process of step S16 is executed. If the tag TG is not included in the first information IF1, that is, if the determination result in step S15 is NO, the processing device 11 ends all the processes.
  • step S16 the processing device 11 functions as the display control unit 114 to display the tag TG specified by the name specifying unit 113-2 on the display 29 of the AR glasses 20. More specifically, the processing device 11 functions as the display control unit 114 to display the specified tag TG in the three-dimensional image TI displayed on the display 29 . Note that if the tag TG is not specified in step S15, the processing device 11 omits the process of step S16.
  • FIG. 18 is a flow chart explaining the third operation of the information processing system 1 .
  • step S ⁇ b>21 the processing device 11 functions as the acquisition unit 111 to acquire line-of-sight data related to the line of sight of the user U ⁇ b>1 on the AR glasses 20 . More specifically, the processing device 21 of the AR glasses 20 outputs the acquired line-of-sight data to the communication device 28 by functioning as the acquisition unit 211 . The communication device 28 outputs the line-of-sight data acquired from the processing device 21 to the terminal device 10 . The processing device 11 of the terminal device 10 functions as the acquisition unit 111 to acquire line-of-sight data from the AR glasses 20 using the communication device 13 .
  • step S22 the processing device 11 functions as the determination unit 115 to determine whether or not the viewpoint of the user U1 is positioned within the first virtual object VO displayed on the AR glasses 20 for a predetermined time or longer. . More specifically, the processing device 11 functions as the determination unit 115 to acquire the viewpoint position of the user U1 as instruction information based on the line-of-sight data acquired in step S21. After that, the processing device 11, as the determination unit 115, determines whether or not the viewpoint of the user U1 has been positioned at the specific virtual object VO for a predetermined time or longer. When the determination result is true, that is, when the determination result of step S22 is "YES", the processing device 11 executes the process of step S23. When the determination result is false, that is, when the determination result of step S22 is "NO", the processing device 11 executes the process of step S21.
  • step S23 the processing device 11 functions as the virtual object specifying unit 113-1 to specify the first virtual object VO among the plurality of virtual objects VO based on the determination result in step S22. More specifically, the processing device 11, by functioning as the virtual object specifying unit 113-1, determines in step S22 that the virtual object VO for which the viewpoint of the user U1 has been positioned for a predetermined period of time or more is assigned to the first virtual object VO. Identify as object VO. Further, the processing device 11 functions as the virtual object specifying unit 113-1, and as a result of specifying the first virtual object VO, an ID corresponding to the first virtual object VO is output.
  • step S24 the processing device 11 refers to the first information IF1 stored in the storage device 12 by functioning as the calling name identification unit 113-2. After that, the processing device 11 functions as the call name identifying unit 113-2 so that the tag TG corresponding to the first virtual object VO identified by functioning as the virtual object identifying unit 113-1 is changed to the first virtual object VO. 1 information IF1. Specifically, the processing device 11 functions as the calling name identification unit 113-2 to determine whether the tag TG corresponding to the ID output in step S23 is included in the first information IF1. If the tag TG is included in the first information IF1, that is, if the determination result in step S24 is YES, the processing device 11 functions as the nickname identification unit 113-2 to identify the tag TG as the first nickname. After that, the process of step S25 is executed. If the tag TG is not included in the first information IF1, that is, if the determination result in step S24 is NO, the processing device 11 terminates all the processes.
  • step S ⁇ b>25 the processing device 11 functions as the display control unit 114 to display the tag TG specified as the first nickname on the display 29 provided in the AR glasses 20 . More specifically, the processing device 11 functions as the display control unit 114 to cause the display 29 to display the specified tag TG in the vicinity of the first virtual object VO.
  • the “neighborhood” of the first virtual object VO specifically means a range within a predetermined distance from the first virtual object VO. Note that if the tag TG is not identified in step S25, the processing device 11 omits the processing of step S26.
  • FIG. 19 is a flow chart explaining the fourth operation of the information processing system 1 .
  • step S31 the processing device 11 functions as the speech recognition unit 116 to recognize the speech uttered by the user U1. More specifically, the processing device 21 of the AR glasses 20 acquires voice data representing the voice of the user U1 from the sound pickup device 24 by functioning as the acquisition unit 211 . Also, the processing device 21 of the AR glasses 20 outputs the acquired audio data to the communication device 28 by functioning as the acquisition unit 211 . The communication device 28 outputs the audio data acquired from the processing device 21 to the terminal device 10 . The processing device 11 of the terminal device 10 functions as the acquisition unit 111 to acquire audio data from the AR glasses 20 using the communication device 13 . Further, the processing device 11 of the terminal device 10 performs voice recognition on voice data by functioning as a voice recognition unit 116 .
  • a character string as a speech recognition result corresponds to instruction information in the first to third operations described above. In this operation example, it is assumed that a character string as a speech recognition result is a character string indicating an attribute.
  • step S32 the processing device 11 identifies the virtual object VO by functioning as the virtual object identification unit 113-1. More specifically, the processing device 11 refers to the second information IF2 stored in the storage device 12 by functioning as the virtual object identification unit 113-1. Further, the processing device 11, by functioning as the virtual object specifying unit 113-1, specifies one or more virtual objects VO corresponding to the attribute whose character string as the speech recognition result is included in the second information IF2. . Further, the processing device 11 functions as the virtual object specifying unit 113-1, and as a result of specifying one or more virtual objects VO, an ID corresponding to the one or more virtual objects VO is output.
  • step S33 the processing device 11 refers to the first information IF1 stored in the storage device 12 by functioning as the nickname identification unit 113-2. After that, the processing device 11 functions as the call name identifying unit 113-2 so that the tag TG corresponding to the first virtual object VO identified by functioning as the virtual object identifying unit 113-1 is changed to the first virtual object VO. 1 information IF1. Specifically, the processing device 11 functions as the nickname identification unit 113-2 to determine whether the tag TG corresponding to the ID output in step S32 is included in the first information IF1. If the tag TG is included in the first information IF1, that is, if the determination result in step S33 is YES, the processing device 11 functions as the nickname specifying unit 113-2 to specify the tag TG as the first nickname. After that, the process of step S34 is executed. If the tag TG is not included in the first information IF1, that is, if the determination result in step S33 is NO, the processing device 11 ends all the processes.
  • step S ⁇ b>34 the processing device 11 functions as the display control unit 114 to display the tag TG specified as the first nickname on the display 29 provided in the AR glasses 20 .
  • the processing device 11 functions as the display control unit 114 to cause the display 29 to display the specified tag TG in a form attached to the virtual object VO within the popup P1. Note that if the tag TG is not specified in step S33, the processing device 11 omits the processing of step S34.
  • the terminal device 10 as an information processing device includes the display control unit 114, the virtual object identification unit 113-1, and the name identification unit 113-2.
  • the display control unit 114 causes the AR glasses 20 as a display device worn on the head of the user U1 to display a plurality of virtual objects VO arranged in the virtual space VS.
  • the virtual object specifying unit 113-1 specifies the first virtual object VO among the plurality of virtual objects VO based on instruction information generated according to the user U1's action.
  • the call name identifying unit 113-2 identifies the corresponding tag TG as the first virtual object VO. identified as a nickname for The display control unit 114 causes the AR glasses 20 as the display device to display the tag TG as the first name.
  • the terminal device 10 as an information processing device identifies the first virtual object VO arranged in the virtual space VS for the user U1 wearing the AR glasses 20 on the head. It is possible to easily recall the tag TG as the first name of the. Specifically, it may be difficult for the user U1 to grasp the correspondence between each virtual object VO and each tag TG and to store the tag TG.
  • the AR glasses 20 as a display device that the user U1 wears on the head are associated with the virtual object VO specified based on the instruction information generated according to the user U1's motion. to display the tag TG corresponding to the virtual object VO. By visually recognizing the displayed tag TG on the AR glasses 20, the user U1 can recall the tag TG corresponding to the virtual object VO.
  • the display control unit 114 causes the AR glasses 20 as a display device to display a two-dimensional image SI obtained by planarly developing the virtual space VS. Further, the display control unit 114 displays the tag TG as the first name in association with the first virtual object VO within the two-dimensional image SI.
  • the user U1 can grasp the position of the virtual object VO that the user wants to call in the virtual space VS, and then call the virtual object VO. It becomes possible to recall the corresponding tag TG.
  • the display control unit 114 causes the AR glasses 20 as a display device to display a three-dimensional image TI obtained by reducing the virtual space VS. Further, the display control unit 114 displays the tag TG as the first name in association with the first virtual object VO within the three-dimensional image TI.
  • the user U1 can grasp the position of the virtual object VO that the user wants to call in the virtual space VS, and then call the virtual object VO. It becomes possible to recall the corresponding tag TG.
  • the above operation of the user U1 is viewing the AR glasses 20 as a display device.
  • the above instruction information indicates the viewpoint of the user U1 on the AR glasses 20 as a display device.
  • the terminal device 10 as an information processing device further includes a determination unit 115 .
  • the determination unit 115 determines whether or not the viewpoint of the user U1 is positioned within the first virtual object VO displayed on the AR glasses 20 as a display device for a predetermined time or longer.
  • the display control unit 114 displays the first virtual object VO on the AR glasses 20 as a display device. to display the tag TG as the first nickname corresponding to .
  • the user U1 does not need to perform any motion other than the motion related to the line of sight with respect to the AR glasses 20 as the display device. It becomes possible to recall the tag TG as a name of 1.
  • the display control unit 114 changes the display content regarding the first virtual object VO when the recognition result of the voice uttered by the user U1 matches the tag TG as the first nickname.
  • the user U1 can, for example, display the first virtual object VO corresponding to the tag TG that matches the speech recognition result in the line of sight of the user U1. It is possible to move in the direction Alternatively, for example, if the first virtual object VO corresponding to the tag TG that matches the speech recognition result is an application, the user U1 can start the application.
  • the virtual object specifying unit 113-1 is the voice uttered by the user U1, which represents at least one attribute of the plurality of virtual objects VO.
  • One or more virtual objects VO are specified based on the speech recognition result.
  • the nickname specifying unit 113-2 specifies a tag TG as a nickname corresponding to each of the one or more virtual objects VO specified by the virtual object specifying unit 113-1.
  • the display control unit 114 causes the AR glasses 20 as the display device to display the tag TG as the corresponding nickname specified by the nickname specifying unit 113-2 in association with each of a part or all of the virtual objects VO. .
  • the user U1 narrows down a plurality of virtual objects VO arranged in the virtual space VS to one or more virtual objects VO corresponding to the same attribute. If the tags TG corresponding to the narrowed-down virtual objects VO include the tag TG corresponding to the virtual object VO that the user U1 wants to call, the user U1 can easily remember the tag TG. .
  • FIG. 2 Second Embodiment A configuration of an information processing system 1A including an information processing apparatus according to a second embodiment of the present invention will be described below with reference to FIGS. 20 to 29.
  • FIG. In the following description, for simplification of description, the same symbols are used for the same components as in the first embodiment, and the description of their functions may be omitted. Also, in the following description, for the sake of simplification of description, mainly the differences between the second embodiment and the first embodiment will be described.
  • FIG. 20 is a diagram showing the overall configuration of an information processing system 1A according to the second embodiment of the present invention.
  • the information processing system 1A differs from the information processing system 1 according to the first embodiment in that it includes a terminal device 10A instead of the terminal device 10.
  • FIG. 20 is a diagram showing the overall configuration of an information processing system 1A according to the second embodiment of the present invention.
  • the information processing system 1A differs from the information processing system 1 according to the first embodiment in that it includes a terminal device 10A instead of the terminal device 10.
  • FIG. 1A includes a terminal device 10A instead of the terminal device 10.
  • FIG. 21 is a block diagram showing a configuration example of the terminal device 10A.
  • the terminal device 10A differs from the terminal device 10 according to the first embodiment in that it includes a processing device 11A instead of the processing device 11 and a storage device 12A instead of the storage device 12 .
  • the storage device 12A differs from the storage device 12 according to the first embodiment in that it is not essential to store the second information IF2 and that it stores the learning model LM1.
  • the learning model LM1 is a learning model for use by the later-described calling name identification unit 113-2A. Specifically, the learning model LM1 is a learning model for calculating the degree of similarity between the first word and the second word. As an example, the learning model LM1 converts the meaning of a word into a numerical vector, and based on how much the direction of the vector of the first word and the direction of the vector of the second word are in the same direction, the Calculate the similarity between the first word and the second word.
  • the similarity calculation method described above is merely an example, and the present invention is not limited to this.
  • the learning model LM1 may use another method as long as it can calculate the degree of similarity between the first word and the second word.
  • the learning model LM1 is generated by learning teacher data in the learning phase.
  • the teacher data used to generate the learning model LM1 has a plurality of pairs of sets of first words and second words and numerical values indicating degrees of similarity.
  • the learning model LM1 is generated outside the terminal device 10A.
  • learning model LM1 is preferably generated in server 30 .
  • the terminal device 10A acquires the learning model LM1 from the server 30 via the communication network NET.
  • the processing device 11A functions as an acquisition unit 111, an action recognition unit 112, a specification unit 113A, a display control unit 114A, a voice recognition unit 116, and an update unit 117 by reading and executing the control program PR4 from the storage device 12A.
  • the acquisition unit 111, the action recognition unit 112, and the speech recognition unit 116 are the same as the acquisition unit 111, the action recognition unit 112, and the speech recognition unit 116 as functions of the processing device 11 according to the first embodiment. Therefore, its description is omitted.
  • FIG. 22 is a functional block diagram showing the configuration of the identification unit 113A.
  • the identification unit 113A differs from the identification unit 113 according to the first embodiment in that it includes a nickname identification unit 113-2A instead of the nickname identification unit 113-2.
  • the nickname identification unit 113-2A performs the same operation as the nickname identification unit 113-2 as the first operation. Specifically, the call name specifying unit 113-2A adds the tag TG corresponding to the first virtual object VO specified by the virtual object specifying unit 113-1 to the first information IF1 stored in the storage device 12. When included, the corresponding tag TG is identified as the first nickname.
  • the nickname identifying unit 113-2A identifies, as a first nickname, a plurality of tags TG corresponding to some or all of the plurality of virtual objects VO arranged in the virtual space VS. More specifically, the call name specifying unit 113-2A identifies the tag TG of the virtual object VO whose first information IF1 includes the tag TG among the plurality of virtual objects VO placed in the virtual space VS. Identify.
  • the nickname identification unit 113-2A performs a second operation in which the recognition result of the first speech uttered by the user U1 does not match any of the plurality of tags TG included in the first information IF1. , the tag TG that is most similar to the second nickname among the plurality of tags TG included in the first information IF1 is specified as the first nickname.
  • the nickname identification unit 113-2A identifies the character string as the recognition result of the first voice uttered by the user U1 and one tag TG among the plurality of tags TG included in the first information IF1. , into the learning model LM1 described above. The learning model LM1 outputs the degree of similarity between the character string as the recognition result of the first speech and the one tag TG. The nickname specifying unit 113-2A performs the same operation for all the tags TG described in the first information IF1, and determines the degree of similarity between the character string as the recognition result of the first speech and all the tags TG. to get Furthermore, the nickname identifying unit 113-2A identifies the tag TG with the highest similarity value among all the tag TGs.
  • the display control unit 114A causes the AR glasses 20 as a display device to display the tag TG specified by the name specifying unit 113-2A.
  • FIG. 23 is an explanatory diagram of a first operation example of the display control unit 114A.
  • the user U1 uttered "DOG" as the tag TG corresponding to the virtual object VO to be called. is a second nickname that is not included in the first information IF1.
  • the nickname identification unit 113-2A selects the tag TG "HORSE” as the tag TG most similar to the second nickname "DOG" among the plurality of tags TG contained in the first information IF1. Identify.
  • the display control unit 114A displays the popup P2 in the virtual space VS.
  • the display control unit 114A causes the user U1 to display a message in the popup P2 to confirm whether the tag TG that the user U1 was originally trying to utter is "HORSE".
  • FIG. 24 is an explanatory diagram of a second operation example of the display control unit 114A.
  • the user U1 uttered some character string as the tag TG corresponding to the virtual object VO that the user U1 wants to call, but the character string that is the recognition result of the uttered first voice is not the first information. Assume that it is a second nickname that is not included in IF1.
  • the call name identification unit 113-2A identifies a plurality of tags TG corresponding to some or all of the plurality of virtual objects VO placed in the virtual space VS based on the first information IF1.
  • the display control unit 114A causes the popup P3 to be displayed in the virtual space VS.
  • the display control unit 114A associates, in the popup P3, the icons of the multiple virtual objects VO corresponding to the specified multiple tags TG, that is, the reduced display, with the specified multiple tags TG, display the list.
  • FIG. 25 is an explanatory diagram of a third operation example of the display control unit 114A.
  • the character string that is the recognition result of the uttered first voice is the first information IF1.
  • the call name identification unit 113-2A identifies a plurality of tags TG corresponding to some or all of the plurality of virtual objects VO placed in the virtual space VS based on the first information IF1. That is, the call name identification unit 113-2A identifies all tags TG included in the first information IF1.
  • the display control unit 114A displays the plurality of tags TG in the vicinity of some or all of the plurality of virtual objects VO corresponding to the plurality of tags TG in the virtual space VS.
  • the “neighborhood” of some or all of the plurality of virtual objects VO specifically means a range within a predetermined distance from each virtual object VO.
  • the user U1 After the display control unit 114A has executed the operation shown in the first operation example, the user U1 utters the tag TG specified by the nickname specifying unit 113-2A. Alternatively, after display control unit 114A performs the operation shown in the second operation example or the third operation example, user U1 may select one or more tags TG specified by nickname specifying unit 113-2A. Suppose that one of the tags TG is uttered. In the operations shown in the first to third operation examples, the user U1 uttered some character string as the tag TG for the virtual object VO to be called, but the character string was not included in the first information IF1. If not, the uttered voice is the first voice.
  • the display control unit 114A displays the virtual object VO corresponding to the matching tag TG. change. Specifically, the display control unit 114A may move the virtual object VO corresponding to the matching tag TG in the line-of-sight direction of the user U1. Alternatively, if the virtual object VO corresponding to the matched tag TG is an application, the display control unit 114 may display a screen for selecting whether to activate the application.
  • the updating unit 117 determines that the recognition result of the first voice uttered by the user U1 is not included in the first information IF1, and the recognition result of the second voice is the first information IF1.
  • the number of matches with a specific tag TG included in IF1 reaches a predetermined number of times, the virtual object VO corresponding to the specific tag TG is treated as a tag TG as a second name, which is the recognition result of the first voice.
  • the user U1 uttered some character string as the tag TG for the virtual object VO to be called
  • the first information IF1 does not include the recognition result of the first speech by utterance.
  • the user U1 utters a character string as one of the tag TG from among the one or a plurality of tags TG specified by the nickname specifying unit 113-2A, and the recognition result of the second voice resulting from the utterance is obtained. is included in the first information IF1.
  • the first information IF1 includes the second speech recognition result after the first speech recognition result has not been included in the first information IF1 for a predetermined number of times. .
  • the updating unit 117 sets the character string as the first speech recognition result to a new tag TG, and then replaces the virtual object VO corresponding to the tag TG as the second speech recognition result with the new tag TG. Associate with a new tag TG.
  • the storage device 12A stores the virtual object VO, the recognition result of the first voice, and the recognition result of the second voice after the first voice is uttered in the first information IF1. and the number of times the test was performed are stored in a tabular form.
  • the update unit 117 determines that the first information IF1 does not include the first speech recognition result and the second speech recognition result is included in the first information IF1. It is determined whether or not the number of times of being held has reached a predetermined number of times. After that, when the number of times reaches a predetermined number of times, the update unit 117 sets the character string as the first speech recognition result to a new tag TG, and then sets the character string as the second speech recognition result. is associated with the new tag TG.
  • FIG. 26A and 26B are explanatory diagrams showing the function of the updating unit 117.
  • FIGS. 27 to 29 are flowcharts showing the operation of the information processing system 1A according to the second embodiment. The operation of the information processing system 1A will be described below with reference to FIGS. 27 to 29.
  • FIG. 27 is a flowchart describing the first operation of the information processing system 1A.
  • step S41 the processing device 11A recognizes the voice uttered by the user U1 by functioning as the voice recognition unit 116. More specifically, the processing device 21 of the AR glasses 20 acquires voice data representing the voice of the user U1 from the sound pickup device 24 by functioning as the acquisition unit 211 . Also, the processing device 21 of the AR glasses 20 outputs the acquired audio data to the communication device 28 by functioning as the acquisition unit 211 . The communication device 28 outputs the audio data acquired from the processing device 21 to the terminal device 10A. The processing device 11A of the terminal device 10A acquires audio data from the AR glasses 20 using the communication device 13 by functioning as the acquisition unit 111 . Further, the processing device 11A of the terminal device 10A functions as a voice recognition unit 116 to perform voice recognition on voice data. A character string as a speech recognition result corresponds to the instruction information in the first embodiment.
  • step S42 the processing device 11A functions as the nickname identification unit 113-2A, so that the recognition result of the voice uttered by the user U1 is included in the first information IF1 and corresponds to the plurality of virtual objects VO. It is determined whether or not it corresponds to any of the tags TG.
  • the processing device 11A executes the process of step S45.
  • the recognition result of the voice uttered by the user U1 is an example of the above first name.
  • the determination result is false, that is, when the determination result of step S42 is "NO”
  • the processing device 11A executes the process of step S43.
  • the recognition result of the voice uttered by the user U1 is an example of the above second name.
  • step S43 the processing device 11A functions as the nickname specifying unit 113-2A, thereby identifying the voice of the user U1 among the plurality of tags TG included in the first information IF1, that is, the plurality of first nicknames.
  • a tag TG that is most similar to the second nickname as a recognition result is specified as the first nickname.
  • step S44 the processing device 11A functions as the display control unit 114A to display the most similar tag TG identified in step S43 in the virtual space VS.
  • the processing device 11A displays the pop-up P2 in the virtual space VS by functioning as the display control unit 114A.
  • the processing device 11A functions as the display control unit 114A to display a message for the user U1 to confirm the tag TG that the user U1 was originally trying to utter in the popup P2. After that, the processing device 11A executes the process of step S41.
  • step S45 the processing device 11A functions as the display control unit 114A to change the display of the virtual object VO corresponding to the tag TG as the first nickname, which is the recognition result in step S42.
  • the update unit 117 may associate the virtual object VO corresponding to the specific tag TG with the tag TG as the second nickname that is the recognition result of the first voice. .
  • FIG. 28 is a flowchart illustrating the second operation of the information processing system 1A.
  • step S51 the processing device 11A recognizes the voice uttered by the user U1 by functioning as the voice recognition unit 116. Note that the details of the operation are the same as in step S41 in the first operation, so description thereof will be omitted.
  • step S52 the processing device 11A, by functioning as the nickname identification unit 113-2A, determines whether the recognition result of the voice uttered by the user U1 corresponds to any of the plurality of tags TG included in the first information IF1. determine whether or not When the determination result is true, that is, when the determination result of step S52 is "YES", the processing device 11A executes the process of step S55. In this case, the recognition result of the voice uttered by the user U1 is an example of the above first name. When the determination result is false, that is, when the determination result of step S52 is "NO", the processing device 11A executes the process of step S53. In this case, the recognition result of the voice uttered by the user U1 is an example of the above second name.
  • step S53 the processing device 11A functions as the call name specifying unit 113-2A, and based on the first information IF1, the plurality of virtual objects VO corresponding to some or all of the plurality of virtual objects VO arranged in the virtual space VS. identifies the tag TG of
  • step S54 the processing device 11A functions as the display control unit 114A to display the popup P3 in the virtual space VS. Furthermore, by functioning as the display control unit 114A, the processing device 11A, in the popup P3, displays the icons of the multiple virtual objects VO corresponding to the multiple tags TG identified in step S53, that is, reduced display, and A list is displayed in association with a plurality of specified tags TG. After that, the processing device 11A executes the process of step S51.
  • step S55 the processing device 11A functions as the display control unit 114A to change the display of the virtual object VO corresponding to the tag TG as the first nickname, which is the recognition result in step S52.
  • the first information IF1 does not include the recognition result of the first voice uttered by the user U1, and the first information IF1 includes the recognition result of the second voice. If the number of matches with the specified tag TG reaches a predetermined number, after step S55, the update unit 117 updates the virtual object VO corresponding to the specified tag TG to the first voice recognition result, which is the first voice recognition result. 2 may be associated with a tag TG as a nickname.
  • FIG. 29 is a flowchart illustrating the third operation of the information processing system 1A.
  • step S61 the processing device 11A recognizes the voice uttered by the user U1 by functioning as the voice recognition unit 116.
  • the details of the operation are the same as those in step S41 in the first operation and step S51 in the second operation, and thus description thereof is omitted.
  • step S62 the processing device 11A, by functioning as the nickname identification unit 113-2A, determines whether the recognition result of the voice uttered by the user U1 corresponds to any of the plurality of tags TG included in the first information IF1. determine whether or not When the determination result is true, that is, when the determination result of step S62 is "YES", the processing device 11A executes the process of step S65. In this case, the recognition result of the voice uttered by the user U1 is an example of the above first name. When the determination result is false, that is, when the determination result of step S62 is "NO", the processing device 11A executes the process of step S63. In this case, the recognition result of the voice uttered by the user U1 is an example of the above second name.
  • step S63 the processing device 11A functions as the call name specifying unit 113-2A, and based on the first information IF1, the plurality of virtual objects VO corresponding to some or all of the plurality of virtual objects VO arranged in the virtual space VS. identifies the tag TG of
  • step S64 the processing device 11A, by functioning as the display control unit 114A, displays the plurality of tags TG in the vicinity of some or all of the plurality of virtual objects VO corresponding to the plurality of tags TG identified in step S63.
  • Display TG Here, the “neighborhood” of some or all of the plurality of virtual objects VO specifically means a range within a predetermined distance from each virtual object VO.
  • the processing device 11A executes the process of step S61.
  • step S65 the processing device 11A functions as the display control unit 114A to change the display of the virtual object VO corresponding to the tag TG as the first nickname, which is the recognition result in step S62.
  • the first information IF1 does not include the recognition result of the first voice uttered by the user U1
  • the updating unit 117 updates the virtual object VO corresponding to the specific tag TG after step S65. , may be associated with the tag TG as the second nickname that is the recognition result of the first voice.
  • the terminal device 10A as an information processing device includes the display control section 114A and the nickname specifying section 113-2A.
  • the display control unit 114A causes the AR glasses 20 as a display device worn on the head of the user U1 to display a plurality of virtual objects VO arranged in the virtual space VS. If the recognition result of the first voice uttered by the user U1 is a second nickname that does not match any of the plurality of first nicknames corresponding to the plurality of virtual objects VO, the nickname specifying unit 113-2A , to identify a first nickname that is most similar to the second nickname among the plurality of first nicknames.
  • the display control unit 114A causes the AR glasses 20 as a display device to display the first nickname specified by the nickname specifying unit 113-2A.
  • the terminal device 10A As an information processing device, even if the speech recognition result of the user U1's utterance does not correspond to the tag TG included in the first information IF1, it is included in the first information IF1. It becomes possible to recall the tag TG. In particular, user U1 can recall one tag TG that is most similar to the recognition result of the voice uttered by him/herself.
  • the display control unit 114A changes the display of the virtual object VO corresponding to the specified first nickname among the plurality of virtual objects VO.
  • the terminal device 10A as an information processing device further includes an updating unit 117 that associates the virtual object VO with the second nickname when the number of times the recognition result of the first voice becomes the second nickname reaches a predetermined number of times. Prepare.
  • the terminal device 10A as an information processing device allows the user U1 to call a certain virtual object VO using a second name not included in the first information IF1. When the number of times of doing so reaches a predetermined number of times, it becomes possible to associate the second nickname with the virtual object VO.
  • the terminal device 10A as an information processing device includes a display control section 114A and a call name specifying section 113-2A.
  • the display control unit 114A causes the AR glasses 20 as a display device worn on the head of the user U1 to display a plurality of virtual objects VO arranged in the virtual space VS.
  • the nickname identification unit 113-2A identifies a plurality of first nicknames corresponding to some or all of the plurality of virtual objects VO.
  • display control unit 114A controls the plurality of names specified by nickname specifying unit 113-2A. Each of the first nicknames is displayed in association with the corresponding virtual object VO out of some or all of the plurality of virtual objects VO.
  • the terminal device 10A As an information processing device, even if the speech recognition result of the user U1's utterance does not correspond to the tag TG included in the first information IF1, the user U1 can obtain the first information It becomes possible to recall the tag TG contained in IF1. In particular, the user U1 can visually recognize all the tags TG included in the first information IF1 within the virtual space VS.
  • the recognition result of the second voice uttered by the user U1 is the specified plurality of names. If the virtual object VO matches any of the first nicknames, the display control unit 114A changes the display of the virtual object VO corresponding to the matching first nickname among the plurality of virtual objects VO. Further, the terminal device 10A as an information processing device further includes an updating unit 117 that associates the virtual object VO with the second nickname when the number of times the recognition result of the first voice becomes the second nickname reaches a predetermined number of times. Prepare.
  • the terminal device 10A as an information processing device allows the user U1 to call a certain virtual object VO using a second name not included in the first information IF1. When the number of times of doing so reaches a predetermined number of times, it becomes possible to associate the second nickname with the virtual object VO.
  • the terminal device 10 includes a speech recognition unit 116 as a function of the processing device 11 .
  • the terminal device 10A according to the second embodiment includes a speech recognition unit 116 as a function of the processing device 11A.
  • the terminal devices 10 and 10A do not have to include the speech recognition section 116 .
  • the voice recognition unit 116 may be an external device of the terminal devices 10 and 10A, and may be communicably connected to the terminal devices 10 and 10A.
  • the speech recognition device corresponding to the speech recognition unit 116 may exist on the cloud and be communicably connected to the terminal devices 10 and 10A via the communication network NET.
  • the terminal device 10 includes an acquisition unit 111 as a function of the processing device 11 .
  • the acquisition unit 111 acquires the first information IF1 and the second information IF2 from the storage device 12 .
  • the terminal device 10A according to the second embodiment includes an acquisition unit 111 as a function of the processing device 11A.
  • the acquiring unit 111 acquires the first information IF1 from the storage device 12A.
  • the acquisition source of the first information IF1 and the second information IF2 of the acquisition unit 111 may not be the storage device 12 or 12A.
  • the acquisition unit 111 may directly acquire the first information IF1 and the second information IF2 from the server 30 .
  • the terminal device 10 according to the first embodiment includes a motion recognition unit 112 as a function of the processing device 11 .
  • the terminal device 10A according to the second embodiment includes a motion recognition unit 112 as a function of the processing device 11A.
  • the action recognition unit 112 recognizes gestures of the user U1.
  • the method of recognizing gestures of user U1 is not limited to the above method.
  • the AR glasses 20 may recognize gestures of the user U1 by including a motion recognition unit similar to the motion recognition unit 112 .
  • the terminal device 10 includes, as a function of the processing device 11, a call name identifying section 113-2.
  • the terminal device 10A according to the second embodiment includes a call name identifying section 113-2A as a function of the processing device 11A.
  • the nickname identifying units 113-2 and 113-2A identify a tag TG as a nickname corresponding to the virtual object VO identified by the virtual object identifying unit 113-1.
  • the call name specifying units 113-2 and 113-2A do not specify the tag TG when there is no tag TG corresponding to the virtual object VO specified by the virtual object specifying unit 113-1.
  • the processing devices 11 and 11A may have a function of setting a new tag TG for a virtual object VO that does not have a corresponding tag TG.
  • the terminal device 10 and the AR glasses 20 are implemented separately.
  • the terminal device 10A and the AR glasses 20 are implemented separately.
  • the method of implementing the terminal device 10 or 10A and the AR glasses 20 in the embodiment of the present invention is not limited to this.
  • the terminal device 10 or 10A and the AR glasses 20 may be realized within a single housing by providing the AR glasses 20 with the same functions as the terminal device 10 or 10A.
  • Modification 6 10 A of terminal devices which concern on 2nd Embodiment are provided with the update part 117 as a function of 11 A of processing apparatuses.
  • the update unit 117 updates the number of times the recognition result of the first voice uttered by the user U1 is not included in the first information IF1 and the recognition result of the second voice matches the specific tag TG included in the first information IF1.
  • the virtual object VO corresponding to the specific tag TG is associated with the tag TG as the second name, which is the recognition result of the first voice. That is, the update unit 117 associates multiple tags TG with one virtual object VO.
  • the operation of the updating unit 117 is not limited to this. For example, instead of associating a plurality of tags TG with one virtual object VO, the updating unit 117 associates one tag TG with one virtual object VO, A set of one virtual object VO and one tag TG may be set.
  • the storage device 12, the storage device 12A, the storage device 22, and the storage device 32 are ROM and RAM, but flexible disks, magneto-optical disks (for example, compact disks) , Digital Versatile Discs, Blu-ray Discs), Smart Cards, Flash Memory Devices (e.g. Cards, Sticks, Key Drives), CD-ROMs (Compact Disc-ROMs), Registers, Removable Discs, Hard Disks, Floppy disk, magnetic strip, database, server or other suitable storage medium.
  • the program may be transmitted from a network via an electric communication line.
  • the program may be transmitted from the communication network NET via an electric communication line.
  • the information, signals, etc. described may be represented using any of a variety of different technologies.
  • data, instructions, commands, information, signals, bits, symbols, chips, etc. may refer to voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, light fields or photons, or any of these. may be represented by a combination of
  • input/output information and the like may be stored in a specific location (for example, memory), or may be managed using a management table. Input/output information and the like can be overwritten, updated, or appended. The output information and the like may be deleted. The entered information and the like may be transmitted to another device.
  • the determination may be made by a value (0 or 1) represented using 1 bit, or by a true/false value (Boolean: true or false). Alternatively, it may be performed by numerical comparison (for example, comparison with a predetermined value).
  • each function illustrated in FIGS. 1 to 29 is realized by any combination of at least one of hardware and software.
  • the method of realizing each functional block is not particularly limited. That is, each functional block may be implemented using one device that is physically or logically coupled, or directly or indirectly using two or more devices that are physically or logically separated (e.g. , wired, wireless, etc.) and may be implemented using these multiple devices.
  • a functional block may be implemented by combining software in the one device or the plurality of devices.
  • software, instructions, information, etc. may be transmitted and received via a transmission medium.
  • the software uses at least one of wired technology (coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), etc.) and wireless technology (infrared, microwave, etc.) to website, Wired and/or wireless technologies are included within the definition of transmission medium when sent from a server or other remote source.
  • wired technology coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), etc.
  • wireless technology infrared, microwave, etc.
  • system and “network” are used interchangeably.
  • Information, parameters, etc. described in this disclosure may be expressed using absolute values, may be expressed using relative values from a predetermined value, or may be expressed using corresponding other information. may be represented as
  • the terminal device 10, the terminal device 10A, and the server 30 may be mobile stations (MS).
  • a mobile station is defined by those skilled in the art as a subscriber station, mobile unit, subscriber unit, wireless unit, remote unit, mobile device, wireless device, wireless communication device, remote device, mobile subscriber station, access terminal, mobile terminal, wireless It may also be called a terminal, remote terminal, handset, user agent, mobile client, client, or some other suitable term. Also, in the present disclosure, terms such as “mobile station”, “user terminal”, “user equipment (UE)", “terminal”, etc. may be used interchangeably.
  • connection refers to any direct or indirect connection between two or more elements. Any connection or coupling is meant and can include the presence of one or more intermediate elements between two elements that are “connected” or “coupled” to each other. Couplings or connections between elements may be physical couplings or connections, logical couplings or connections, or a combination thereof. For example, “connection” may be replaced with "access”.
  • two elements are defined using at least one of one or more wires, cables, and printed electrical connections and, as some non-limiting and non-exhaustive examples, in the radio frequency domain. , electromagnetic energy having wavelengths in the microwave and optical (both visible and invisible) regions, and the like.
  • the phrase “based on” does not mean “based only on,” unless expressly specified otherwise. In other words, the phrase “based on” means both “based only on” and “based at least on.”
  • determining and “determining” as used in this disclosure may encompass a wide variety of actions.
  • “Judgement” and “determination” are, for example, judging, calculating, computing, processing, deriving, investigating, looking up, searching, inquiring (eg, lookup in a table, database, or other data structure), ascertaining as “judged” or “determined”, and the like.
  • "judgment” and “determination” are used for receiving (e.g., receiving information), transmitting (e.g., transmitting information), input, output, access (accessing) (for example, accessing data in memory) may include deeming that a "judgment” or “decision” has been made.
  • judgment and “decision” are considered to be “judgment” and “decision” by resolving, selecting, choosing, establishing, comparing, etc. can contain.
  • judgment and “decision” may include considering that some action is “judgment” and “decision”.
  • judgment (decision) may be read as “assuming”, “expecting”, “considering”, or the like.
  • the term "A and B are different” may mean “A and B are different from each other.” The term may also mean that "A and B are different from C”. Terms such as “separate,” “coupled,” etc. may also be interpreted in the same manner as “different.”
  • notification of predetermined information is not limited to explicit notification, but is performed implicitly (for example, not notification of the predetermined information). good too.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • General Physics & Mathematics (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Health & Medical Sciences (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Computational Linguistics (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • Software Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

This information processing device is provided with: a display control unit which, on a display device mounted on the user's head, displays multiple virtual objects arranged in a virtual space; a virtual object specifying unit which specifies a first virtual object from multiple virtual objects on the basis of instruction information generated in response to a user operation; and a name specifying part which, in the case that a name corresponding to the first virtual object specified by the virtual object specifying unit is stored in a storage device, specifies said corresponding name as a first name. The display control unit displays the first name on the display device.

Description

情報処理装置Information processing equipment
　本発明は、情報処理装置に関する。 The present invention relates to an information processing device.
　ＡＲ（Augmented Reality）技術において、ユーザが知覚する現実の環境はコンピュータにより拡張される。この技術を用いることにより、例えば、ユーザが頭部に装着するＡＲグラスを通じて視認する現実の空間に、仮想空間を重畳表示することが可能となる。 In AR (Augmented Reality) technology, the real environment perceived by the user is augmented by a computer. By using this technology, for example, it is possible to superimpose and display a virtual space on a real space visually recognized through AR glasses worn on the user's head.
　ＡＲ技術においては、仮想空間に配置される仮想オブジェクトに対して、タグが紐づけられることがある。例えば、特許文献１は、ヘッドマウントディスプレイによって撮像された人物の撮像画像に対して顔認証処理を実行し、当該撮像画像から抽出された顔画像に対して、顔認証結果である当該人物の名前をタグ情報として付与する技術を開示している。 In AR technology, tags are sometimes attached to virtual objects placed in the virtual space. For example, in Patent Document 1, face authentication processing is performed on a captured image of a person captured by a head-mounted display, and the name of the person, which is the face recognition result, is processed for a face image extracted from the captured image. is disclosed as tag information.
　一方で、現実空間を撮影する技術に関して、例えば特許文献２は、撮影画像に含まれるオブジェクトを示すタグを設定する技術を開示している。特許文献２に係る技術においては、当該タグを用いて、当該タグに示されるオブジェクトが撮像された撮影画像を検索することが可能である。 On the other hand, regarding the technology for capturing the real space, Patent Document 2, for example, discloses a technology for setting a tag indicating an object included in the captured image. In the technique disclosed in Patent Document 2, it is possible to search for a photographed image in which an object indicated by the tag is captured using the tag.
特開２０１９－０１２５３６号公報JP 2019-012536 A 特許６９０８９５３号公報Japanese Patent No. 6908953
　ＡＲ技術において、仮想オブジェクトに対しタグを設定すると共に、当該タグを用いて当該タグに対応する仮想オブジェクトを特定する場合、仮想オブジェクトとタグとの組の数が増加するに従い、ユーザが各仮想オブジェクトと各タグとの対応関係を把握すること、及びユーザが当該タグ自体を記憶することが困難となる。その結果、ユーザが自身の特定したい仮想オブジェクトに対応するタグを想起できない場合、当該仮想オブジェクトを簡便に特定することができなくなる。 In the AR technology, when a tag is set for a virtual object and a virtual object corresponding to the tag is specified using the tag, as the number of pairs of the virtual object and the tag increases, the user can identify each virtual object. and each tag, and it becomes difficult for the user to memorize the tag itself. As a result, if the user cannot remember the tag corresponding to the virtual object that the user wants to specify, the user cannot easily specify the virtual object.
　そこで本発明は、ユーザに対して、仮想空間に配置される仮想オブジェクトを特定するための呼び名としてのタグを、簡便に想起させることが可能な情報処理装置を提供することを目的とする。 An object of the present invention is to provide an information processing apparatus that can easily remind a user of a tag as a name for specifying a virtual object placed in a virtual space.
　本発明の好適な態様に係る情報処理装置は、ユーザの頭部に装着される表示装置に、仮想空間に配置される複数の仮想オブジェクトを表示させる表示制御部と、前記ユーザの動作に応じて生成される指示情報に基づいて前記複数の仮想オブジェクトのうち第１の仮想オブジェクトを特定する仮想オブジェクト特定部と、前記仮想オブジェクト特定部が特定した第１の仮想オブジェクトに対応する呼び名が記憶装置に記憶されている場合、当該対応する呼び名を第１の呼び名として特定する呼び名特定部とを備え、前記表示制御部は、前記表示装置に前記第１の呼び名を表示させる、情報処理装置である。 An information processing apparatus according to a preferred aspect of the present invention includes a display control unit that displays a plurality of virtual objects arranged in a virtual space on a display device worn on the user's head, and A virtual object specifying unit for specifying a first virtual object among the plurality of virtual objects based on generated instruction information, and a name corresponding to the first virtual object specified by the virtual object specifying unit are stored in a storage device. and a call name specifying unit that specifies the corresponding call name as a first call name when stored, wherein the display control unit causes the display device to display the first call name.
　本発明によれば、ユーザに対して、仮想空間に配置される仮想オブジェクトを特定するための呼び名を、簡便に想起させることが可能となる。 According to the present invention, it is possible to easily remind the user of the name for specifying the virtual object placed in the virtual space.
第１実施形態に係る情報処理システム１の全体構成を示す図。The figure which shows the whole structure of the information processing system 1 which concerns on 1st Embodiment. ＡＲグラス２０の外観を示す斜視図。FIG. 2 is a perspective view showing the appearance of the AR glasses 20; 仮想空間ＶＳの模式図。Schematic diagram of virtual space VS. 仮想空間ＶＳの模式図。Schematic diagram of virtual space VS. ＡＲグラス２０の構成例を示すブロック図。FIG. 2 is a block diagram showing a configuration example of the AR glasses 20; 端末装置１０の構成例を示すブロック図。2 is a block diagram showing a configuration example of a terminal device 10; FIG. 第１情報ＩＦ１の例を示す図。FIG. 4 is a diagram showing an example of first information IF1; 第２情報ＩＦ２の例を示す図。The figure which shows the example of 2nd information IF2. 特定部１１３の構成を示す機能ブロック図。3 is a functional block diagram showing the configuration of an identifying unit 113; FIG. タグＴＧの利用方法についての説明図。Explanatory drawing about the utilization method of tag TG. タグＴＧの利用方法についての説明図。Explanatory drawing about the utilization method of tag TG. タグＴＧの利用方法についての説明図。Explanatory drawing about the utilization method of tag TG. 表示制御部１１４の第１の動作例についての説明図。FIG. 4 is an explanatory diagram of a first operation example of the display control unit 114; 表示制御部１１４の第１の動作例についての説明図。FIG. 4 is an explanatory diagram of a first operation example of the display control unit 114; 表示制御部１１４の第２の動作例についての説明図。FIG. 9 is an explanatory diagram of a second operation example of the display control unit 114; 表示制御部１１４の第２の動作例についての説明図。FIG. 9 is an explanatory diagram of a second operation example of the display control unit 114; 表示制御部１１４の第３の動作例についての説明図。FIG. 11 is an explanatory diagram of a third operation example of the display control unit 114; 表示制御部１１４の第３の動作例についての説明図。FIG. 11 is an explanatory diagram of a third operation example of the display control unit 114; 表示制御部１１４の第４の動作例についての説明図。FIG. 11 is an explanatory diagram of a fourth operation example of the display control unit 114; 表示制御部１１４の第４の動作例についての説明図。FIG. 11 is an explanatory diagram of a fourth operation example of the display control unit 114; サーバ３０の構成例を示すブロック図。3 is a block diagram showing a configuration example of a server 30; FIG. 第１実施形態に係る情報処理システム１の第１の動作を示すフローチャート。4 is a flowchart showing a first operation of the information processing system 1 according to the first embodiment; 第１実施形態に係る情報処理システム１の第２の動作を示すフローチャート。4 is a flowchart showing a second operation of the information processing system 1 according to the first embodiment; 第１実施形態に係る情報処理システム１の第３の動作を示すフローチャート。8 is a flowchart showing a third operation of the information processing system 1 according to the first embodiment; 第１実施形態に係る情報処理システム１の第４の動作を示すフローチャート。9 is a flowchart showing a fourth operation of the information processing system 1 according to the first embodiment; 第２実施形態に係る情報処理システム１Ａの全体構成を示す図。The figure which shows the whole structure of 1 A of information processing systems which concern on 2nd Embodiment. 端末装置１０Ａの構成例を示すブロック図。The block diagram which shows the structural example of 10 A of terminal devices. 特定部１１３Ａの構成を示す機能ブロック図。FIG. 3 is a functional block diagram showing the configuration of an identifying unit 113A; 表示制御部１１４Ａの第１の動作例についての説明図。Explanatory drawing about the 1st operation example of 114 A of display control parts. 表示制御部１１４Ａの第２の動作例についての説明図。Explanatory drawing about the 2nd operation example of 114 A of display control parts. 表示制御部１１４Ａの第３の動作例についての説明図。FIG. 11 is an explanatory diagram of a third operation example of the display control unit 114A; 更新部１１７の機能を示す説明図。FIG. 4 is an explanatory diagram showing functions of an updating unit 117; 更新部１１７の機能を示す説明図。FIG. 4 is an explanatory diagram showing functions of an updating unit 117; 情報処理システム１Ａの第１の動作について説明するフローチャート。4 is a flowchart for explaining a first operation of the information processing system 1A; 情報処理システム１Ａの第２の動作について説明するフローチャート。6 is a flowchart for explaining a second operation of the information processing system 1A; 情報処理システム１Ａの第３の動作について説明するフローチャート。A flow chart explaining the 3rd operation of information processing system 1A.
１：第１実施形態
　以下、図１～図１９を参照することにより、本発明の第１実施形態に係る情報処理装置を含む情報処理システム１の構成について説明する。 1: First Embodiment Hereinafter, the configuration of an information processing system 1 including an information processing apparatus according to a first embodiment of the present invention will be described with reference to FIGS. 1 to 19. FIG.
１．１：第１実施形態の構成
１．１．１：全体構成
　図１は、本発明の第１実施形態に係る情報処理システム１の全体構成を示す図である。情報処理システム１は、後述のＡＲグラス２０を装着したユーザＵ１に対して、ＡＲ技術により、仮想空間を提供するシステムである。 1.1: Configuration of First Embodiment 1.1.1: Overall Configuration FIG. 1 is a diagram showing the overall configuration of an information processing system 1 according to the first embodiment of the present invention. The information processing system 1 is a system that provides a virtual space using AR technology to a user U1 wearing AR glasses 20, which will be described later.
　情報処理システム１は、端末装置１０、ＡＲグラス２０、及びサーバ３０を備える。端末装置１０とＡＲグラス２０とは互いに通信可能に接続される。また、端末装置１０とサーバ３０とは、通信網ＮＥＴを介して互いに通信可能に接続される。なお、図１において、端末装置１０とＡＲグラス２０との組として、端末装置１０－１とＡＲグラス２０－１との組、端末装置１０－２とＡＲグラス２０－２との組、及び端末装置１０－３とＡＲグラス２０－３との組の合計３組が記載される。しかし、当該組数はあくまで一例であって、情報処理システム１は、任意の数の端末装置１０とＡＲグラス２０との組を備えることが可能である。また、端末装置１０は、情報処理装置の一例である。 The information processing system 1 includes a terminal device 10, AR glasses 20, and a server 30. The terminal device 10 and the AR glasses 20 are communicably connected to each other. Also, the terminal device 10 and the server 30 are communicably connected to each other via a communication network NET. In FIG. 1, the terminal device 10 and the AR glasses 20 are combined as a pair of the terminal device 10-1 and the AR glasses 20-1, a pair of the terminal device 10-2 and the AR glasses 20-2, and a terminal device 10-1 and the AR glasses 20-2. A total of three pairs of devices 10-3 and AR glasses 20-3 are described. However, the number of sets is merely an example, and the information processing system 1 can include any number of sets of terminal devices 10 and AR glasses 20 . Also, the terminal device 10 is an example of an information processing device.
　端末装置１０は、ユーザが頭部に装着するＡＲグラス２０に対して、仮想空間に配置される仮想オブジェクトを表示させるための装置である。当該仮想空間は、一例として、天球型の空間である。また、仮想オブジェクトは、例として、静止画像、動画、３ＤＣＧモデル、ＨＴＭＬファイル、及びテキストファイル等のデータを示す仮想オブジェクト、及びアプリケーションを示す仮想オブジェクトである。ここで、テキストファイルとしては、例として、メモ、ソースコード、日記、及びレシピが挙げられる。また、アプリケーションとしては、例として、ブラウザ、ＳＮＳを用いるためのアプリケーション、及びドキュメントファイルを生成するためのアプリケーションが挙げられる。なお、端末装置１０は、例として、スマートフォン、及びタブレット等の携帯端末装置であることが好適である。 The terminal device 10 is a device for displaying a virtual object arranged in a virtual space on the AR glasses 20 worn on the user's head. The virtual space is, for example, a celestial space. The virtual objects are, for example, virtual objects representing data such as still images, moving images, 3DCG models, HTML files, and text files, and virtual objects representing applications. Examples of text files include memos, source codes, diaries, and recipes. Examples of applications include browsers, applications for using SNS, and applications for generating document files. Note that the terminal device 10 is preferably a mobile terminal device such as a smart phone and a tablet, for example.
　ＡＲグラス２０は、ユーザの頭部に装着するシースルー型のウエアラブルディスプレイである。ＡＲグラス２０は、端末装置１０の制御により、両眼用のレンズの各々に設けられた表示パネルに仮想オブジェクトを表示させる。なお、ＡＲグラス２０は、表示装置の一例である。 The AR glasses 20 are see-through wearable displays worn on the user's head. Under the control of the terminal device 10, the AR glasses 20 display a virtual object on the display panel provided for each of the binocular lenses. Note that the AR glasses 20 are an example of a display device.
　サーバ３０は、通信網ＮＥＴを介して、端末装置１０に対して各種データ及びクラウドサービスを提供する。 The server 30 provides various data and cloud services to the terminal device 10 via the communication network NET.
１．１．２：ＡＲグラスの構成
　図２は、ＡＲグラス２０の外観を示す斜視図である。図２に示されるようにＡＲグラス２０は、一般的な眼鏡と同様にテンプル９１及び９２、ブリッジ９３、胴部９４及び９５、及びレンズ４１Ｌ及び４１Ｒを有する。ブリッジ９３には撮像装置２７が設けられる。撮像装置２７は外界を撮像し、撮像した画像を示す撮像データを出力する。また、テンプル９１及び９２の各々には、音を収音する収音装置２４が設けられる。収音装置２４は、収音した音声を示す音声データを出力する。なお、収音装置２４の位置は、テンプル９１及びテンプル９２に限定されず、例えば、ブリッジ９３、及び胴部９４及び９５のいずれかであってもよい。 1.1.2: Configuration of AR Glasses FIG. 2 is a perspective view showing the appearance of the AR glasses 20. As shown in FIG. As shown in FIG. 2, the AR glasses 20 have temples 91 and 92, a bridge 93, trunks 94 and 95, and lenses 41L and 41R, like common spectacles. An imaging device 27 is provided in the bridge 93 . The imaging device 27 captures an image of the outside world and outputs imaging data representing the captured image. Also, each of the temples 91 and 92 is provided with a sound pickup device 24 that picks up sound. The sound collection device 24 outputs sound data representing the collected sound. Note that the position of the sound pickup device 24 is not limited to the temples 91 and 92, and may be, for example, the bridge 93 and any one of the trunks 94 and 95.
　レンズ４１Ｌ及び４１Ｒの各々は、ハーフミラーを備えている。胴部９４には、左眼用の液晶パネル又は有機ＥＬパネル（以下、表示パネルと総称する）と、左眼用の表示パネルから射出された光をレンズ４１Ｌに導光する光学部材が設けられる。レンズ４１Ｌに設けられるハーフミラーは、外界の光を透過させて左眼に導くと共に、光学部材によって導光された光を反射することにより、左眼に入射させる。胴部９５には、右眼用の表示パネルと、右眼用の表示パネルから射出された光をレンズ４１Ｒに導光する光学部材とが設けられる。レンズ４１Ｒに設けられるハーフミラーは、外界の光を透過させて右眼に導くと共に、光学部材によって導光された光を反射することにより、右眼に入射させる。 Each of the lenses 41L and 41R has a half mirror. The body portion 94 is provided with a liquid crystal panel or an organic EL panel for the left eye (hereinafter collectively referred to as a display panel) and an optical member for guiding light emitted from the display panel for the left eye to the lens 41L. . The half mirror provided in the lens 41L transmits external light and guides it to the left eye, and reflects the light guided by the optical member to enter the left eye. The body portion 95 is provided with a right-eye display panel and an optical member that guides light emitted from the right-eye display panel to the lens 41R. The half mirror provided in the lens 41R transmits external light and guides it to the right eye, and reflects the light guided by the optical member to enter the right eye.
　後述するディスプレイ２９は、レンズ４１Ｌ、左眼用の表示パネル、及び左眼用の光学部材、並びにレンズ４１Ｒ、右眼用の表示パネル、及び右眼用の光学部材を含む。 The display 29, which will be described later, includes a lens 41L, a left-eye display panel, a left-eye optical member, and a lens 41R, a right-eye display panel, and a right-eye optical member.
　以上の構成において、ユーザは表示パネルによる画像を、外界の様子と重ね合わせた状態で観察できる。また、ＡＲグラス２０においては、視差を伴う両眼画像のうち、左眼用画像が左眼用の表示パネルに表示され、右眼用画像が右眼用の表示パネルに表示される。両眼視差を利用することで、ユーザＵ１は、表示された画像があたかも奥行き、及び立体感を持つかのように知覚することが可能となる。 With the above configuration, the user can observe the image on the display panel superimposed on the state of the outside world. In addition, in the AR glasses 20, of the binocular images with parallax, the image for the left eye is displayed on the display panel for the left eye, and the image for the right eye is displayed on the display panel for the right eye. By using binocular parallax, the user U1 can perceive the displayed image as if it had depth and stereoscopic effect.
　図３及び図４は、ＡＲグラス２０を用いることによりユーザＵ１に提供される仮想空間ＶＳの模式図である。図３に示されるように、当該仮想空間ＶＳには、例えばブラウザ、クラウドサービス、画像、及び動画等の各種コンテンツを示す仮想オブジェクトＶＯ１～ＶＯ５が配置される。ユーザＵ１は、当該仮想空間ＶＳに配置される仮想オブジェクトＶＯ１～ＶＯ５が表示されたＡＲグラス２０を装着した状態で公共の空間を行き来することにより、公共の空間において、プライベートな空間としての当該仮想空間ＶＳを体験することが可能となる。延いては、当該ユーザＵ１が、当該仮想空間ＶＳに配置された仮想オブジェクトＶＯ１～ＶＯ５のもたらす便益を受けながら、公共の空間において行動することが可能となる。 3 and 4 are schematic diagrams of the virtual space VS provided to the user U1 by using the AR glasses 20. FIG. As shown in FIG. 3, in the virtual space VS, virtual objects VO1 to VO5 representing various contents such as browsers, cloud services, images, and moving images are arranged. The user U1 walks around the public space while wearing the AR glasses 20 on which the virtual objects VO1 to VO5 arranged in the virtual space VS are displayed. It becomes possible to experience the space VS. Ultimately, the user U1 can act in the public space while receiving benefits brought about by the virtual objects VO1 to VO5 placed in the virtual space VS.
　また、図４に示されるように、複数のユーザＵ１～Ｕ３で仮想空間ＶＳを共有することも可能である。複数のユーザＵ１～Ｕ３で仮想空間ＶＳを共有することにより、当該複数のユーザＵ１～Ｕ３で、１又は複数の仮想オブジェクトＶＯを共用すると共に、共用した仮想オブジェクトＶＯを介して、ユーザＵ１～Ｕ３は互いにコミュニケーションを行うことが可能となる。 Also, as shown in FIG. 4, it is possible for a plurality of users U1 to U3 to share the virtual space VS. By sharing the virtual space VS with a plurality of users U1-U3, the plurality of users U1-U3 share one or a plurality of virtual objects VO, and the users U1-U3 can communicate with each other.
　図５は、ＡＲグラス２０の構成例を示すブロック図である。ＡＲグラス２０は、処理装置２１、記憶装置２２、視線検出装置２３、収音装置２４、ＧＰＳ装置２５、動き検出装置２６、撮像装置２７、通信装置２８、及びディスプレイ２９を備える。ＡＲグラス２０の各要素は、情報を通信するための単体又は複数のバスで相互に接続される。 FIG. 5 is a block diagram showing a configuration example of the AR glasses 20. As shown in FIG. The AR glasses 20 include a processing device 21 , a storage device 22 , a line-of-sight detection device 23 , a sound collection device 24 , a GPS device 25 , a motion detection device 26 , an imaging device 27 , a communication device 28 and a display 29 . Each element of the AR glasses 20 is interconnected by one or more buses for communicating information.
　処理装置２１は、ＡＲグラス２０の全体を制御するプロセッサであり、例えば、単数又は複数のチップを用いて構成される。処理装置２１は、例えば、周辺装置とのインタフェース、演算装置及びレジスタ等を含む中央処理装置（ＣＰＵ：Central Processing Unit）を用いて構成される。なお、処理装置２１の機能の一部又は全部を、ＤＳＰ（Digital Signal Processor）、ＡＳＩＣ（Application Specific Integrated Circuit）、ＰＬＤ（Programmable Logic Device）、ＦＰＧＡ（Field Programmable Gate Array）等のハードウェアによって実現してもよい。処理装置２１は、各種の処理を並列的又は逐次的に実行する。 The processing device 21 is a processor that controls the entire AR glasses 20, and is configured using, for example, one or more chips. The processing device 21 is configured using, for example, a central processing unit (CPU) including an interface with peripheral devices, an arithmetic device, registers, and the like. Some or all of the functions of the processing device 21 are realized by hardware such as DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), PLD (Programmable Logic Device), FPGA (Field Programmable Gate Array), and the like. may The processing device 21 executes various processes in parallel or sequentially.
　記憶装置２２は、処理装置２１による読取及び書込が可能な記録媒体であり、処理装置２１が実行する制御プログラムＰＲ１を含む複数のプログラムを記憶する。 The storage device 22 is a recording medium readable and writable by the processing device 21, and stores a plurality of programs including the control program PR1 executed by the processing device 21.
　視線検出装置２３は、ユーザＵ１の視線を検出し、検出結果に基づいてユーザＵ１の視線の方向を示す視線データを、後述の処理装置２１に出力する。視線検出装置２３による視線の検出は、どのような方法を用いてもよいが、例えば、目頭の位置と虹彩の位置に基づいて視線データを検出してもよい。 The line-of-sight detection device 23 detects the line of sight of the user U1, and outputs line-of-sight data indicating the direction of the line of sight of the user U1 to the processing device 21, which will be described later. The line-of-sight detection device 23 may detect the line-of-sight by any method. For example, line-of-sight data may be detected based on the position of the inner corner of the eye and the position of the iris.
　収音装置２４は、音声を収音し、収音した音声に基づく音声データを、後述の処理装置２１に出力する。 The sound collection device 24 collects sound and outputs sound data based on the collected sound to the processing device 21, which will be described later.
　ＧＰＳ装置２５は、複数の衛星からの電波を受信し、受信した電波から位置データを生成する。位置データは、ＡＲグラス２０の位置を示す。位置データは、位置を特定できるのであれば、どのような形式であってもよい。位置データは、例えば、ＡＲグラス２０の緯度と経度とを示す。一例として、位置データはＧＰＳ装置２５から得られる。しかし、ＡＲグラス２０は、どのような方法で位置データを取得してもよい。取得された位置データは、処理装置２１に出力される。 The GPS device 25 receives radio waves from multiple satellites and generates position data from the received radio waves. The position data indicates the position of the AR glasses 20. FIG. The location data may be in any format as long as the location can be specified. The position data indicates the latitude and longitude of the AR glasses 20, for example. As an example, position data is obtained from GPS device 25 . However, the AR glasses 20 may acquire position data by any method. The acquired position data is output to the processing device 21 .
　動き検出装置２６は、ＡＲグラス２０の動きを検出して、動きデータを処理装置２１に出力する。動き検出装置２６としては、加速度を検出する加速度センサ及び角加速度を検出するジャイロセンサなどの慣性センサが例示される。加速度センサは、直交するＸ軸、Ｙ軸、及びＺ軸の加速度を検出する。ジャイロセンサは、Ｘ軸、Ｙ軸、及びＺ軸を回転の中心軸とする角加速度を検出する。動き検出装置２６は、ジャイロセンサの出力情報に基づいて、ＡＲグラス２０の姿勢を示す姿勢情報を生成できる。動きデータは、３軸の加速度を各々示す加速度データ及び３軸の角加速度を各々示す角加速度データを含む。 The motion detection device 26 detects motion of the AR glasses 20 and outputs motion data to the processing device 21 . Examples of the motion detection device 26 include inertial sensors such as an acceleration sensor that detects acceleration and a gyro sensor that detects angular acceleration. The acceleration sensor detects acceleration in orthogonal X-, Y-, and Z-axes. The gyro sensor detects angular acceleration around the X-, Y-, and Z-axes. The motion detection device 26 can generate orientation information indicating the orientation of the AR glasses 20 based on the output information of the gyro sensor. The motion data includes acceleration data respectively indicating three-axis accelerations and angular acceleration data respectively indicating three-axis angular accelerations.
　撮像装置２７は、外界を撮像することにより得られた撮像データを出力する。撮像装置２７は、例えば、レンズ、撮像素子、増幅器、及びＡＤ変換器を備える。レンズを介して集光された光は、撮像素子によってアナログ信号である撮像信号に変換される。増幅器は撮像信号を増幅した上でＡＤ変換器に出力する。ＡＤ変換器はアナログ信号である増幅された撮像信号をデジタル信号である撮像データに変換する。変換された撮像データは、処理装置２１に出力される。処理装置２１に出力された撮像データは、通信装置２８を介して、上記の端末装置１０に出力される。端末装置１０は、撮像データに基づいて、ユーザＵ１の各種のジェスチャーを認識し、認識されたジェスチャーに応じて端末装置１０を制御する。即ち、撮像装置２７は、ポインティングデバイス及びタッチパネルのように、ユーザＵ１の指示を入力するための入力装置として機能する。 The imaging device 27 outputs imaging data obtained by imaging the outside world. The imaging device 27 includes, for example, a lens, an imaging element, an amplifier, and an AD converter. The light condensed through the lens is converted into an image pickup signal, which is an analog signal, by the image pickup device. The amplifier amplifies the imaging signal and outputs it to the AD converter. The AD converter converts the amplified imaging signal, which is an analog signal, into imaging data, which is a digital signal. The converted imaging data is output to the processing device 21 . The imaging data output to the processing device 21 is output to the terminal device 10 via the communication device 28 . The terminal device 10 recognizes various gestures of the user U1 based on the imaging data, and controls the terminal device 10 according to the recognized gestures. That is, the imaging device 27 functions as an input device for inputting instructions from the user U1, like a pointing device and a touch panel.
　通信装置２８は、他の装置と通信を行うための、送受信デバイスとしてのハードウェアである。通信装置２８は、例えば、ネットワークデバイス、ネットワークコントローラ、ネットワークカード、通信モジュール等とも呼ばれる。通信装置２８は、有線接続用のコネクターを備え、上記コネクターに対応するインタフェース回路を備えていてもよい。また、通信装置２８は、無線通信インタフェースを備えていてもよい。有線接続用のコネクター及びインタフェース回路としては有線ＬＡＮ、ＩＥＥＥ１３９４、又はＵＳＢに準拠した製品が挙げられる。また、無線通信インタフェースとしては無線ＬＡＮ及びＢｌｕｅｔｏｏｔｈ（登録商標）等に準拠した製品が挙げられる。 The communication device 28 is hardware as a transmission/reception device for communicating with other devices. The communication device 28 is also called, for example, a network device, a network controller, a network card, a communication module, or the like. The communication device 28 may include a connector for wired connection and an interface circuit corresponding to the connector. The communication device 28 may also have a wireless communication interface. Products conforming to wired LAN, IEEE1394, or USB can be used as connectors and interface circuits for wired connection. Also, as a wireless communication interface, there are products conforming to wireless LAN, Bluetooth (registered trademark), and the like.
　ディスプレイ２９は、画像を表示するデバイスである。ディスプレイ２９は、処理装置２１による制御のもとで各種の画像を表示する。ディスプレイ２９は、上記のように、レンズ４１Ｌ、左眼用の表示パネル、及び左眼用の光学部材、並びにレンズ４１Ｒ、右眼用の表示パネル、及び右眼用の光学部材を含む。表示パネルとしては、例えば、液晶表示パネル及び有機ＥＬ表示パネル等の各種の表示パネルが好適に利用される。 The display 29 is a device that displays images. The display 29 displays various images under the control of the processing device 21 . The display 29 includes the lens 41L, the left-eye display panel, the left-eye optical member, and the lens 41R, the right-eye display panel, and the right-eye optical member, as described above. Various display panels such as a liquid crystal display panel and an organic EL display panel are preferably used as the display panel.
　処理装置２１は、例えば、記憶装置２２から制御プログラムＰＲ１を読み出して実行することによって、取得部２１１、及び表示制御部２１２として機能する。 The processing device 21 functions as an acquisition unit 211 and a display control unit 212, for example, by reading the control program PR1 from the storage device 22 and executing it.
　取得部２１１は、端末装置１０からの制御信号を取得する。より詳細には、取得部２１１は、端末装置１０に備わる、後述の表示制御部１１４によって生成される、ＡＲグラス２０での表示を制御するための制御信号を取得する。 The acquisition unit 211 acquires the control signal from the terminal device 10 . More specifically, the acquisition unit 211 acquires a control signal for controlling display on the AR glasses 20 generated by a display control unit 114 provided in the terminal device 10 and described later.
　また、取得部２１１は、視線検出装置２３から入力される視線データ、収音装置２４から入力される音声データ、ＧＰＳ装置２５から入力される位置データ、動き検出装置２６から入力される動きデータ、及び撮像装置２７から入力される撮像データを取得する。その上で、取得部２１１は、取得した視線データ、音声データ、位置データ、動きデータ、及び撮像データを、通信装置２８に出力する。 The acquisition unit 211 also receives line-of-sight data input from the line-of-sight detection device 23, audio data input from the sound collection device 24, position data input from the GPS device 25, motion data input from the motion detection device 26, and acquires imaging data input from the imaging device 27 . After that, the acquisition unit 211 outputs the acquired line-of-sight data, audio data, position data, motion data, and imaging data to the communication device 28 .
　表示制御部２１２は、取得部２１１によって取得された端末装置１０からの制御信号に基づいて、ディスプレイ２９における表示を制御する。 The display control unit 212 controls display on the display 29 based on the control signal from the terminal device 10 acquired by the acquisition unit 211 .
１．１．３：端末装置の構成
　図６は、端末装置１０の構成例を示すブロック図である。端末装置１０は、処理装置１１、記憶装置１２、通信装置１３、ディスプレイ１４、入力装置１５、及び慣性センサ１６を備える。端末装置１０の各要素は、情報を通信するための単体又は複数のバスで相互に接続される。なお、本明細書における「装置」という用語は、回路、デバイス又はユニット等の他の用語に読替えてもよい。 1.1.3: Configuration of Terminal Device FIG. 6 is a block diagram showing a configuration example of the terminal device 10. As shown in FIG. The terminal device 10 includes a processing device 11 , a storage device 12 , a communication device 13 , a display 14 , an input device 15 and an inertial sensor 16 . The elements of terminal 10 are interconnected by a bus or buses for communicating information. Note that the term "apparatus" in this specification may be replaced with another term such as a circuit, a device, or a unit.
　処理装置１１は、端末装置１０の全体を制御するプロセッサであり、例えば、単数又は複数のチップを用いて構成される。処理装置１１は、例えば、周辺装置とのインタフェース、演算装置及びレジスタ等を含む中央処理装置（ＣＰＵ）を用いて構成される。なお、処理装置１１が有する機能の一部又は全部を、ＤＳＰ、ＡＳＩＣ、ＰＬＤ、ＦＰＧＡ等のハードウェアによって実現してもよい。処理装置１１は、各種の処理を並列的又は逐次的に実行する。 The processing device 11 is a processor that controls the entire terminal device 10, and is configured using, for example, one or more chips. The processing unit 11 is configured using, for example, a central processing unit (CPU) including interfaces with peripheral devices, arithmetic units, registers, and the like. A part or all of the functions of the processing device 11 may be realized by hardware such as DSP, ASIC, PLD, and FPGA. The processing device 11 executes various processes in parallel or sequentially.
　記憶装置１２は、処理装置１１が読取及び書込が可能な記録媒体であり、処理装置１１が実行する制御プログラムＰＲ２を含む複数のプログラム、第１情報ＩＦ１、及び第２情報ＩＦ２を記憶する。 The storage device 12 is a recording medium readable and writable by the processing device 11, and stores a plurality of programs including a control program PR2 executed by the processing device 11, first information IF1, and second information IF2.
　図７は、第１情報ＩＦ１の例を示す図である。図７に示される例において、第１情報ＩＦ１は表形式の情報である。第１情報ＩＦ１は、仮想オブジェクトＶＯを一意に識別する識別情報、仮想オブジェクトＶＯに対応するタグＴＧ、天球型の仮想空間ＶＳにおける仮想オブジェクトＶＯの位置を示す位置情報、及び各仮想オブジェクトＶＯの画像情報を対応付ける。識別情報は以下ＩＤと称する。位置情報は仮想空間ＶＳにおける３次元座標である。なお、各仮想オブジェクトＶＯが、自身に対応するタグＴＧを有さない場合、タグＴＧの欄は空欄となる。すなわち、第１情報ＩＦ１においては、仮想空間ＶＳに配置される複数の仮想オブジェクトＶＯの一部又は全部が、複数のタグＴＧと１対１に対応付けられる。なお、タグＴＧとは、仮想オブジェクトＶＯの呼び名の一例である。また、仮想空間ＶＳに配置される複数の仮想オブジェクトＶＯは、ユーザＵ１が姿勢を変更することによって、視認可能な複数の仮想オブジェクトＶＯである。仮想オブジェクトＶＯは、ユーザＵ１の指示に基づいて、仮想空間ＶＳに配置可能である。また、仮想オブジェクトＶＯは、ユーザＵ１の指示が無くても、所定の条件が充足された場合に、仮想空間ＶＳに配置されることがある。例えば、Ｊ個の仮想オブジェクトＶＯのうち、Ｋ個の仮想オブジェクトＶＯが仮想空間ＶＳに配置される。そして、仮想空間ＶＳに配置されたＫ個の仮想オブジェクトＶＯのうち、Ｌ個の仮想オブジェクトＶＯにタグＴＧが割り当てられている。Ｊ、Ｋ、及びＬは整数であり、Ｊ≧Ｋ≧Ｌである。第１情報ＩＦ１は、Ｋ個の仮想オブジェクトＶＯに関する情報である。なお、第１情報ＩＦ１は、通信装置１３を介してサーバ３０から取得されてもよい。 FIG. 7 is a diagram showing an example of the first information IF1. In the example shown in FIG. 7, the first information IF1 is tabular information. The first information IF1 includes identification information that uniquely identifies the virtual object VO, a tag TG corresponding to the virtual object VO, position information indicating the position of the virtual object VO in the celestial virtual space VS, and an image of each virtual object VO. Match information. The identification information is hereinafter referred to as ID. The position information is three-dimensional coordinates in the virtual space VS. If each virtual object VO does not have a tag TG corresponding to itself, the tag TG column is blank. That is, in the first information IF1, some or all of the plurality of virtual objects VO arranged in the virtual space VS are associated one-to-one with the plurality of tags TG. Note that the tag TG is an example of the name of the virtual object VO. Also, the plurality of virtual objects VO arranged in the virtual space VS are the plurality of virtual objects VO that can be visually recognized by the user U1 changing the posture. The virtual object VO can be placed in the virtual space VS based on an instruction from the user U1. Also, the virtual object VO may be arranged in the virtual space VS when a predetermined condition is satisfied without an instruction from the user U1. For example, K virtual objects VO out of J virtual objects VO are arranged in the virtual space VS. Of the K virtual objects VO placed in the virtual space VS, L virtual objects VO are assigned tags TG. J, K, and L are integers, and J≧K≧L. The first information IF1 is information about K virtual objects VO. Note that the first information IF<b>1 may be acquired from the server 30 via the communication device 13 .
　図８は、第２情報ＩＦ２の例を示す図である。図８に示される例において、第２情報ＩＦ２は表形式の情報である。第２情報ＩＦ２は、仮想空間ＶＳに配置される仮想オブジェクトＶＯのＩＤと、仮想オブジェクトＶＯの属性とを対応付ける。第２情報ＩＦ２は、Ｋ個の仮想オブジェクトＶＯに関する情報である。なお、第２情報ＩＦ２は、通信装置１３を介してサーバ３０から取得されてもよい。ここで、「属性」とは、各仮想オブジェクトＶＯの内容を特徴又は性質によって分類した項目である。仮想オブジェクトＶＯには、静止画像、動画、３ＤＣＧモデル、ＨＴＭＬファイル、及びテキストファイル等のデータ、並びにアプリケーションが含まれる。この例では、仮想空間ＶＳに配置される複数の仮想オブジェクトＶＯの各々に属性が割り当てられるが、複数の仮想オブジェクトＶＯの一部に属性が割り当てられてもよい。属性は２以上の仮想オブジェクトＶＯに共通する属性が割り当てられる場合がある。一方、２以上の仮想オブジェクトＶＯに共通するタグＴＧは存在しない。したがって、属性を用いることによって、特徴又は性質が同じ仮想オブジェクトＶＯをグループ化できる。図８に示される例においては、例えば「動物」という属性を用いることにより、ＩＤ＝６、ＩＤ＝７、及びＩＤ＝８の仮想オブジェクトＶＯがグループ化される。一方、タグＴＧを用いることによって、一つの仮想オブジェクトＶＯを特定できる。図７に示される例においては、例えば「＃ＨＯＲＳＥ」というタグＴＧを用いることにより、ＩＤ＝８の仮想オブジェクトＶＯが特定される。 FIG. 8 is a diagram showing an example of the second information IF2. In the example shown in FIG. 8, the second information IF2 is tabular information. The second information IF2 associates the ID of the virtual object VO arranged in the virtual space VS with the attributes of the virtual object VO. The second information IF2 is information about K virtual objects VO. Note that the second information IF2 may be acquired from the server 30 via the communication device 13 . Here, the "attribute" is an item that classifies the content of each virtual object VO according to its features or properties. The virtual object VO includes data such as still images, moving images, 3DCG models, HTML files and text files, and applications. In this example, attributes are assigned to each of the plurality of virtual objects VO arranged in the virtual space VS, but attributes may be assigned to some of the plurality of virtual objects VO. Attributes may be assigned attributes common to two or more virtual objects VO. On the other hand, there is no tag TG common to two or more virtual objects VO. Therefore, attributes can be used to group virtual objects VO that have the same features or properties. In the example shown in FIG. 8, virtual objects VO with ID=6, ID=7, and ID=8 are grouped by using the attribute "animal", for example. On the other hand, one virtual object VO can be identified by using the tag TG. In the example shown in FIG. 7, the virtual object VO with ID=8 is identified by using the tag TG "#HORSE", for example.
　図６に説明を戻すと、通信装置１３は、他の装置と通信を行うための、送受信デバイスとしてのハードウェアである。通信装置１３は、例えば、ネットワークデバイス、ネットワークコントローラ、ネットワークカード、又は通信モジュール等とも呼ばれる。通信装置１３は、有線接続用のコネクターを備え、上記コネクターに対応するインタフェース回路を備えていてもよい。また、通信装置１３は、無線通信インタフェースを備えていてもよい。有線接続用のコネクター及びインタフェース回路としては有線ＬＡＮ、ＩＥＥＥ１３９４、又はＵＳＢに準拠した製品が挙げられる。また、無線通信インタフェースとしては無線ＬＡＮ及びＢｌｕｅｔｏｏｔｈ（登録商標）等に準拠した製品が挙げられる。 Returning to FIG. 6, the communication device 13 is hardware as a transmission/reception device for communicating with other devices. The communication device 13 is also called, for example, a network device, a network controller, a network card, or a communication module. The communication device 13 may include a connector for wired connection and an interface circuit corresponding to the connector. Further, the communication device 13 may have a wireless communication interface. Products conforming to wired LAN, IEEE1394, or USB can be used as connectors and interface circuits for wired connection. Also, as a wireless communication interface, there are products conforming to wireless LAN, Bluetooth (registered trademark), and the like.
　ディスプレイ１４は、画像及び文字情報を表示するデバイスである。ディスプレイ１４は、処理装置１１の制御のもとで各種の画像を表示する。例えば、液晶表示パネル及び有機ＥＬ（Electro　Luminescence）表示パネル等の各種の表示パネルがディスプレイ１４として好適に利用される。 The display 14 is a device that displays images and character information. The display 14 displays various images under the control of the processing device 11 . For example, various display panels such as a liquid crystal display panel and an organic EL (Electro Luminescence) display panel are preferably used as the display 14 .
　入力装置１５は、ＡＲグラス２０を頭部に装着したユーザＵ１からの操作を受け付ける。例えば、入力装置１５は、キーボード、タッチパッド、タッチパネル又はマウス等のポインティングデバイスを含んで構成される。ここで、入力装置１５は、タッチパネルを含んで構成される場合、ディスプレイ１４を兼ねてもよい。 The input device 15 accepts operations from the user U1 wearing the AR glasses 20 on his head. For example, the input device 15 includes a pointing device such as a keyboard, touch pad, touch panel, or mouse. Here, when the input device 15 includes a touch panel, the input device 15 may also serve as the display 14 .
　慣性センサ１６は、慣性力を検出するセンサである。慣性センサ１６は、例えば、加速度センサ、角速度センサ、及びジャイロセンサのうち、１以上のセンサを含む。処理装置１１は、慣性センサ１６の出力情報に基づいて、端末装置１０の姿勢を検出する。更に、処理装置１１は、端末装置１０の姿勢に基づいて、天球型の仮想空間ＶＳにおいて、仮想オブジェクトＶＯの選択、文字の入力、及び指示の入力を受け付ける。例えば、ユーザＵ１が端末装置１０の中心軸を仮想空間ＶＳの所定領域に向けた状態で、入力装置１５を操作することによって、所定領域に配置される仮想オブジェクトＶＯが選択される。入力装置１５に対するユーザＵ１の操作は、例えば、ダブルタップである。このようにユーザＵ１は端末装置１０を操作することによって、端末装置１０の入力装置１５を見なくても仮想オブジェクトＶＯを選択できる。 The inertial sensor 16 is a sensor that detects inertial force. The inertial sensor 16 includes, for example, one or more of an acceleration sensor, an angular velocity sensor, and a gyro sensor. The processing device 11 detects the orientation of the terminal device 10 based on the output information from the inertial sensor 16 . Further, the processing device 11 receives selection of the virtual object VO, input of characters, and input of instructions in the celestial sphere virtual space VS based on the orientation of the terminal device 10 . For example, the user U1 directs the central axis of the terminal device 10 toward a predetermined area of the virtual space VS, and operates the input device 15 to select the virtual object VO arranged in the predetermined area. The user U1's operation on the input device 15 is, for example, a double tap. By operating the terminal device 10 in this way, the user U1 can select the virtual object VO without looking at the input device 15 of the terminal device 10 .
　仮想空間ＶＳに仮想キーボードが配置されている場合、ユーザＵ１が入力したいキーに端末装置１０の中心軸を向けた状態で、入力装置１５を操作することによって、文字が入力される。また、例えば、ユーザＵ１が入力装置１５を押下した状態で、端末装置１０を左右に動かすことによって、所定の指示が入力される。 When a virtual keyboard is arranged in the virtual space VS, characters are input by operating the input device 15 with the central axis of the terminal device 10 facing the key that the user U1 wants to input. Further, for example, when the user U1 presses down the input device 15 and moves the terminal device 10 left or right, a predetermined instruction is input.
　この結果、端末装置１０は、仮想空間ＶＳを制御する携帯型のコントローラとして機能する。 As a result, the terminal device 10 functions as a portable controller that controls the virtual space VS.
　処理装置１１は、記憶装置１２から制御プログラムＰＲ２を読み出して実行することによって、取得部１１１、動作認識部１１２、特定部１１３、表示制御部１１４、判定部１１５、及び音声認識部１１６として機能する。 The processing device 11 functions as an acquisition unit 111, an action recognition unit 112, an identification unit 113, a display control unit 114, a determination unit 115, and a voice recognition unit 116 by reading and executing the control program PR2 from the storage device 12. .
　取得部１１１は、ＡＲグラス２０が頭部に装着されるユーザＵ１の動作に応じた指示情報を取得する。指示情報は、特定の仮想オブジェクトＶＯを指定する情報である。
ここで、ユーザＵ１の動作とは、例として、ユーザＵ１が入力装置１５を用いて端末装置１０へ入力することである。より詳細には、ユーザＵ１の動作は、端末装置１０に設けられた、入力装置１５としての特定部位の押下であってよい。あるいは、ユーザＵ１の動作は、端末装置１０を携帯型のコントローラとして用いた操作であってよい。 The acquisition unit 111 acquires instruction information according to the motion of the user U1 wearing the AR glasses 20 on the head. The instruction information is information that designates a specific virtual object VO.
Here, the operation of the user U1 is, for example, inputting to the terminal device 10 by the user U1 using the input device 15 . More specifically, the action of the user U1 may be pressing of a specific part as the input device 15 provided in the terminal device 10 . Alternatively, the user U1's action may be an operation using the terminal device 10 as a portable controller.
　又は、ユーザＵ１の動作は、ＡＲグラス２０に対するユーザＵ１の目視であってもよい。ユーザＵ１の動作が目視である場合、指示情報は、ＡＲグラス２０におけるユーザＵ１の視点である。この場合、指示情報は、ＡＲグラス２０から端末装置１０へ送信される。 Alternatively, the action of the user U1 may be visual observation of the AR glasses 20 by the user U1. If the action of the user U1 is visual observation, the instruction information is the viewpoint of the user U1 on the AR glasses 20 . In this case, the instruction information is transmitted from the AR glasses 20 to the terminal device 10 .
　あるいは、ユーザＵ１の動作は、ユーザＵ１のジェスチャーであってもよい。後述のように、動作認識部１１２は、ユーザＵ１の各種のジェスチャーを認識する。取得部１１１は、ユーザＵ１の各種ジェスチャーに応じた指示情報を取得してもよい。 Alternatively, the action of user U1 may be a gesture of user U1. As will be described later, the action recognition unit 112 recognizes various gestures of the user U1. The acquisition unit 111 may acquire instruction information according to various gestures of the user U1.
　また、取得部１１１は、通信装置１３を用いることにより、サーバ３０から第１情報ＩＦ１及び第２情報ＩＦ２を取得する。取得部１１１は、取得した第１情報ＩＦ１及び第２情報ＩＦ２を記憶装置１２に格納する。更に、取得部１１１は、記憶装置１２から、第１情報ＩＦ１及び第２情報ＩＦ２を取得する。 Also, the acquisition unit 111 acquires the first information IF1 and the second information IF2 from the server 30 by using the communication device 13 . The acquisition unit 111 stores the acquired first information IF1 and second information IF2 in the storage device 12 . Furthermore, the acquisition unit 111 acquires the first information IF1 and the second information IF2 from the storage device 12 .
　動作認識部１１２は、ＡＲグラス２０から得た撮像データに基づいて、ユーザＵ１の各種のジェスチャーを認識する。より詳細には、上記のように、ＡＲグラス２０に備わる撮像装置２７は、外界を撮像して得た撮像データを出力する。当該撮像データに、ＡＲグラス２０を頭部に装着したユーザＵ１の身体の一部が含まれる場合には、動作認識部１１２は、ＡＲグラス２０から取得した撮像データに基づいて、ユーザＵ１の各種のジェスチャーを認識する。 The motion recognition unit 112 recognizes various gestures of the user U1 based on the imaging data obtained from the AR glasses 20. More specifically, as described above, the imaging device 27 provided in the AR glasses 20 outputs imaging data obtained by imaging the outside world. When the imaging data includes a part of the body of the user U1 wearing the AR glasses 20 on the head, the action recognition unit 112 performs various actions of the user U1 based on the imaging data acquired from the AR glasses 20. gestures.
　特定部１１３は、図９に示されるように、仮想オブジェクト特定部１１３－１と、呼び名特定部１１３－２とを備える。仮想オブジェクト特定部１１３－１は、取得部１１１によって取得された指示情報に基づいて、仮想空間ＶＳに配置された複数の仮想オブジェクトＶＯのうち、１つの仮想オブジェクトＶＯを特定する。以下、仮想オブジェクト特定部１１３－１によって特定された１つの仮想オブジェクトＶＯを第１の仮想オブジェクトＶＯと称する。また、取得部１１１によって取得された第１情報ＩＦ１に、特定された仮想オブジェクトＶＯに対応する１つのタグＴＧが含まれる場合に、呼び名特定部１１３－２は、当該１つのタグＴＧを特定する。１つのタグＴＧは、第１の呼び名の一例である。 The identification unit 113, as shown in FIG. 9, includes a virtual object identification unit 113-1 and a nickname identification unit 113-2. Based on the instruction information acquired by the acquisition unit 111, the virtual object specifying unit 113-1 specifies one virtual object VO among a plurality of virtual objects VO arranged in the virtual space VS. One virtual object VO specified by the virtual object specifying unit 113-1 is hereinafter referred to as a first virtual object VO. Further, when the first information IF1 acquired by the acquisition unit 111 includes one tag TG corresponding to the specified virtual object VO, the nickname specifying unit 113-2 specifies the one tag TG. . One tag TG is an example of a first nickname.
　呼び名特定部１１３－２は、第１情報ＩＦ１を参照することによって、第１の仮想オブジェクトＶＯに対応するタグＴＧを特定する。換言すれば、第１の仮想オブジェクトＶＯに対応するタグＴＧが記憶装置１２に記憶されている場合に、呼び名特定部１１３－２は、当該対応するタグＴＧを、第１の呼び名として特定する。 The nickname identifying unit 113-2 identifies the tag TG corresponding to the first virtual object VO by referring to the first information IF1. In other words, when the tag TG corresponding to the first virtual object VO is stored in the storage device 12, the nickname identifying unit 113-2 identifies the corresponding tag TG as the first nickname.
　図７に示される第１情報ＩＦ１の例を用いて説明すると、仮想オブジェクト特定部１１３－１が、上記の指示情報に基づいて、ＩＤ＝７の仮想オブジェクトＶＯを特定したとする。この場合、第１情報ＩＦ１において、ＩＤ＝７の仮想オブジェクトＶＯに対応するタグＴＧが“＃ＦＯＸ”であるため、呼び名特定部１１３－２は“＃ＦＯＸ”のタグＴＧを特定する。一方で、仮想オブジェクト特定部１１３－１が、上記の指示情報に基づいて、ＩＤ＝６の仮想オブジェクトＶＯを特定したとする。この場合、第１情報ＩＦ１には、ＩＤ＝６の仮想オブジェクトＶＯに対応するタグＴＧが存在しないため、呼び名特定部１１３－２はタグＴＧを特定しない。 Using the example of the first information IF1 shown in FIG. 7, assume that the virtual object specifying unit 113-1 has specified the virtual object VO with ID=7 based on the above instruction information. In this case, in the first information IF1, the tag TG corresponding to the virtual object VO with ID=7 is "#FOX", so the nickname identifying unit 113-2 identifies the tag TG of "#FOX". On the other hand, assume that the virtual object specifying unit 113-1 has specified the virtual object VO with ID=6 based on the above instruction information. In this case, since the tag TG corresponding to the virtual object VO with ID=6 does not exist in the first information IF1, the nickname specifying unit 113-2 does not specify the tag TG.
　図１０Ａ～図１０Ｃは、タグＴＧの利用方法についての説明図である。例として、図１０Ａに示されるように、ユーザＵ１が知覚する仮想空間ＶＳには、各々が鹿、キツネ、及び馬を示す仮想オブジェクトＶＯ６～ＶＯ８が配置されているとする。また、キツネの仮想オブジェクトＶＯ７には、タグＴＧ７として“＃ＦＯＸ”の文字列が登録されているとする。ユーザＵ１は、例として、図１０Ｂに示されるように、端末装置１０に備わる入力装置１５としての特定部位を押下しながら、各コンテンツに対応するタグＴＧによって示される文字列を発声する。ここでは例として、ユーザＵ１は、キツネに対応するタグＴＧ７である“＃ＦＯＸ”の文字列を発声したとする。ユーザＵ１によるタグＴＧ７の発生により、図１０Ｃに示されるようにユーザＵ１は、視線の方向に、発声した文字列に対応する仮想オブジェクトＶＯ７を呼び出すことが可能となる。 　Figs. 10A to 10C are explanatory diagrams of how to use the tag TG. As an example, as shown in FIG. 10A, assume that virtual objects VO6 to VO8 representing a deer, a fox, and a horse are arranged in a virtual space VS perceived by the user U1. It is also assumed that the character string "#FOX" is registered as the tag TG7 in the fox virtual object VO7. For example, as shown in FIG. 10B, the user U1 utters a character string indicated by a tag TG corresponding to each content while pressing a specific portion of the input device 15 provided in the terminal device 10. Here, as an example, it is assumed that user U1 has uttered the character string "#FOX", which is the tag TG7 corresponding to the fox. Generation of the tag TG7 by the user U1 enables the user U1 to call the virtual object VO7 corresponding to the uttered character string in the direction of the line of sight, as shown in FIG. 10C.
　表示制御部１１４は、表示装置としてのＡＲグラス２０に、仮想空間ＶＳに配置される複数の仮想オブジェクトＶＯを表示させる。また、表示制御部１１４は、ＡＲグラス２０に、呼び名特定部１１３－２によって特定されたタグＴＧを表示させる。より具体的には、表示制御部１１４は、ＡＲグラス２０に表示させる画像データを生成し、生成した画像データを、通信装置１３を介してＡＲグラス２０へ送信する。 The display control unit 114 causes the AR glasses 20 as a display device to display a plurality of virtual objects VO placed in the virtual space VS. The display control unit 114 also causes the AR glasses 20 to display the tag TG specified by the name specifying unit 113-2. More specifically, the display control unit 114 generates image data to be displayed on the AR glasses 20 and transmits the generated image data to the AR glasses 20 via the communication device 13 .
　図１１Ａ及び図１１Ｂは、表示制御部１１４の第１の動作例についての説明図である。最初に、取得部１１１により、ユーザＵ１による入力装置１５に対する操作によって生成される操作信号がトリガとして取得される。すると図１１Ａに示されるように、表示制御部１１４は、天球型の仮想空間ＶＳを、当該天球の緯線と経線に対応する複数の直線によって、複数の領域Ｒ１～Ｒ１７に分割する。その上で、表示制御部１１４は、分割した複数の領域Ｒ１～Ｒ１７の各々を平面化する。更に、表示制御部１１４は、図１１Ｂに示されるように、ＡＲグラス２０が有する上記のディスプレイ２９において、平面化された複数の領域Ｒ１～Ｒ１７を平面展開した２次元画像ＳＩを表示させる。具体的には、表示制御部１１４は、天球型の空間の中心に位置するユーザＵ１から見て、天頂部の領域Ｒ１７を中心として右方に位置する領域Ｒ５及びＲ６を、２次元画像ＳＩにおいて領域Ｒ１７の右方に配置する。また、表示制御部１１４は、天球型の空間の中心に位置するユーザＵ１から見て、天頂部の領域Ｒ１７を中心として左方に位置する領域Ｒ１３及びＲ１４を、２次元画像ＳＩにおいて領域Ｒ１７の左方に配置する。また、表示制御部１１４は、天球型の空間の中心に位置するユーザＵ１から見て、天頂部の領域Ｒ１７を中心として前方に位置する領域Ｒ９及びＲ１０を、２次元画像ＳＩにおいて領域Ｒ１７の下方に配置する。また、表示制御部１１４は、天球型の空間の中心に位置するユーザＵ１から見て、天頂部の領域Ｒ１７を中心として後方に位置する領域Ｒ１及びＲ２を、２次元画像ＳＩにおいて領域Ｒ１７の上方に配置する。また、表示制御部１１４は、天球型の空間の中心に位置するユーザＵ１から見て、天頂部の領域Ｒ１７を中心として右前方に位置する領域Ｒ７及びＲ８を、２次元画像ＳＩにおいて領域Ｒ１７の右下方に配置する。また、表示制御部１１４は、天球型の空間の中心に位置するユーザＵ１から見て、天頂部の領域Ｒ１７を中心として左前方に位置する領域Ｒ１１及びＲ１２を、２次元画像ＳＩにおいて領域Ｒ１７の左下方に配置する。また、表示制御部１１４は、天球型の空間の中心に位置するユーザＵ１から見て、天頂部の領域Ｒ１７を中心として右後方に位置する領域Ｒ３及びＲ４を、２次元画像ＳＩにおいて領域Ｒ１７の右上方に配置する。また、表示制御部１１４は、天球型の空間の中心に位置するユーザＵ１から見て、天頂部の領域Ｒ１７を中心として左後方に位置する領域Ｒ１５及びＲ１６を、２次元画像ＳＩにおいて領域Ｒ１７の左上方に配置する。更に、図１１Ｂに示されるように、領域Ｒ１～Ｒ１７の各々の上下左右方向は、天球型の空間の中心に位置するユーザＵ１が当該領域Ｒ１～Ｒ１７に対峙した場合に、ユーザＵ１から見た上下左右方向と一致する。 11A and 11B are explanatory diagrams of a first operation example of the display control unit 114. FIG. First, the acquisition unit 111 acquires an operation signal generated by an operation on the input device 15 by the user U1 as a trigger. Then, as shown in FIG. 11A, the display control unit 114 divides the celestial sphere-shaped virtual space VS into a plurality of regions R1 to R17 by a plurality of straight lines corresponding to the latitude and longitude lines of the celestial sphere. After that, the display control unit 114 planarizes each of the plurality of divided regions R1 to R17. Furthermore, as shown in FIG. 11B, the display control unit 114 causes the display 29 of the AR glasses 20 to display a two-dimensional image SI obtained by planarizing the plurality of flattened regions R1 to R17. Specifically, the display control unit 114 displays regions R5 and R6 located on the right side of the zenith region R17 as viewed from the user U1 located in the center of the celestial space in the two-dimensional image SI. It is arranged to the right of the region R17. In addition, the display control unit 114 displays regions R13 and R14 positioned to the left of the zenith region R17 as viewed from the user U1 who is positioned in the center of the celestial sphere space in the two-dimensional image SI. Place on the left. In addition, the display control unit 114 displays the regions R9 and R10 positioned in front of the zenith region R17 as viewed from the user U1 positioned in the center of the celestial sphere as follows in the two-dimensional image SI below the region R17. to be placed. In addition, the display control unit 114 causes the regions R1 and R2 positioned behind the zenith region R17 as viewed from the user U1 positioned at the center of the celestial space to be displayed above the region R17 in the two-dimensional image SI. to be placed. Further, the display control unit 114 displays regions R7 and R8, which are positioned to the right and front of the zenith region R17 as viewed from the user U1 who is positioned at the center of the celestial space, in the two-dimensional image SI. Place it in the lower right corner. In addition, the display control unit 114 causes the regions R11 and R12, which are positioned in front left of the zenith region R17 as viewed from the user U1 positioned in the center of the celestial space, to be displayed in the two-dimensional image SI as the region R17. Place it in the lower left corner. In addition, the display control unit 114 causes the regions R3 and R4 positioned to the right rear of the zenith region R17 as viewed from the user U1 positioned in the center of the celestial space to be displayed as the regions R17 in the two-dimensional image SI. Place it in the upper right corner. In addition, the display control unit 114 displays regions R15 and R16 positioned to the left rear of the zenith region R17 as viewed from the user U1 positioned at the center of the celestial sphere space in the two-dimensional image SI. Place it on the upper left. Furthermore, as shown in FIG. 11B, the vertical and horizontal directions of each of the regions R1 to R17 are the same as those viewed from the user U1 when the user U1, who is positioned in the center of the celestial space, faces the regions R1 to R17. Matches up, down, left, and right.
　その上で、表示制御部１１４は、ＡＲグラス２０が有するディスプレイ２９において、仮想オブジェクト特定部１１３－１によって特定された仮想オブジェクトＶＯが含まれる領域Ｒに隣接させて、呼び名特定部１１３－２によって特定されたタグＴＧを表示させる。図１１Ｂに示される例においては、表示制御部１１４は、仮想オブジェクト特定部１１３－１によって特定された仮想オブジェクトＶＯ７が含まれる領域Ｒ３に隣接させて、呼び名特定部１１３－２によって特定されたタグＴＧ７を表示させる。 Then, the display control unit 114 causes the display 29 of the AR glasses 20 to be adjacent to the region R containing the virtual object VO specified by the virtual object specifying unit 113-1, and the virtual object VO specified by the name specifying unit 113-2. Display the specified tag TG. In the example shown in FIG. 11B, the display control unit 114 causes the tag specified by the nickname specifying unit 113-2 to be adjacent to the region R3 containing the virtual object VO7 specified by the virtual object specifying unit 113-1. Display TG7.
　なお、上記のように、表示制御部１１４は、ユーザＵ１による入力装置１５に対する操作に応じた操作信号をトリガとして、複数の領域Ｒ１～Ｒ１７を平面展開する。しかし、平面展開のトリガとなる操作信号は、入力装置１５に対する操作に応じて生成される場合に限定されない。例えば、上記のように、処理装置１１は、動作認識部１１２として機能する。処理装置１１が動作認識部１１２として機能することによって検出されたユーザＵ１のジェスチャーに応じて、上記の操作信号が生成されてもよい。あるいは、端末装置１０の姿勢に応じて、上記の操作信号が生成されてもよい。 Note that, as described above, the display control unit 114 planarly develops the plurality of regions R1 to R17 using an operation signal corresponding to the operation of the input device 15 by the user U1 as a trigger. However, the operation signal that triggers planar development is not limited to being generated in response to an operation on the input device 15 . For example, as described above, the processing device 11 functions as the motion recognition unit 112 . The operation signal may be generated according to the gesture of the user U1 detected by the processing device 11 functioning as the motion recognition unit 112 . Alternatively, the above operation signal may be generated according to the attitude of the terminal device 10 .
　図１２Ａ及び図１２Ｂは、表示制御部１１４の第２の動作例についての説明図である。図１２Ａに示されるように、通常時において、表示制御部１１４は、天底にユーザＵ１が位置するような天球型の仮想空間ＶＳを、ユーザＵ１の頭部に装着されるＡＲグラス２０が有するディスプレイ２９に表示させる。更に、表示制御部１１４は、ユーザＵ１による入力装置１５に対する操作によって生成される操作信号に基づいて、図１２Ｂに示されるように、ＡＲグラス２０が有するディスプレイ２９に対して、上記の仮想空間ＶＳを縮小した３次元画像ＴＩを表示させる。その上で、表示制御部１１４は、当該３次元画像ＴＩ上で、仮想オブジェクト特定部１１３－１によって特定された仮想オブジェクトＶＯと当該仮想オブジェクトＶＯに対応するタグＴＧとを隣接させて表示させる。図１２Ｂに示される例においては、表示制御部１１４は、当該３次元画像ＴＩ上で、仮想オブジェクト特定部１１３－１によって特定された仮想オブジェクトＶＯ７と当該仮想オブジェクトＶＯ７に対応するタグＴＧ７とを隣接させて表示させる。 12A and 12B are explanatory diagrams of a second operation example of the display control unit 114. FIG. As shown in FIG. 12A, during normal operation, the display control unit 114 causes the AR glasses 20 worn on the head of the user U1 to have a celestial virtual space VS in which the user U1 is positioned at the nadir. display on the display 29. 12B, the display control unit 114 controls the display 29 of the AR glasses 20 based on the operation signal generated by the operation of the input device 15 by the user U1. is displayed as a reduced three-dimensional image TI. Then, the display control unit 114 causes the virtual object VO specified by the virtual object specifying unit 113-1 and the tag TG corresponding to the virtual object VO to be displayed adjacent to each other on the three-dimensional image TI. In the example shown in FIG. 12B, the display control unit 114 causes the virtual object VO7 specified by the virtual object specifying unit 113-1 and the tag TG7 corresponding to the virtual object VO7 to be adjacent to each other on the three-dimensional image TI. display.
　なお、上記のように、表示制御部１１４は、ユーザＵ１による入力装置１５に対する操作によって生成される操作信号に基づいて、ディスプレイ２９に対して３次元画像ＴＩを表示させる。しかし、３次元画像ＴＩの表示のトリガとなる操作信号は、入力装置１５に対する操作に応じて生成される場合に限定されない。例えば、上記と同様に、処理装置１１が動作認識部１１２として機能することによって検出されたユーザＵ１のジェスチャーに応じて、上記の操作信号が生成されてもよい。あるいは、端末装置１０の姿勢に応じて、上記の操作信号が生成されてもよい。 Note that, as described above, the display control unit 114 causes the display 29 to display the three-dimensional image TI based on the operation signal generated by the operation of the input device 15 by the user U1. However, the operation signal that triggers the display of the three-dimensional image TI is not limited to being generated in response to the operation on the input device 15 . For example, similar to the above, the above operation signal may be generated according to a gesture of the user U1 detected by the processing device 11 functioning as the action recognition unit 112 . Alternatively, the above operation signal may be generated according to the attitude of the terminal device 10 .
　図６に説明を戻すと、判定部１１５は、表示装置としてのＡＲグラス２０に表示される特定の仮想オブジェクトＶＯ内に、ユーザＵ１の視点が所定時間以上位置するか否かを判定する。 Returning to FIG. 6, the determination unit 115 determines whether or not the viewpoint of the user U1 is positioned within a specific virtual object VO displayed on the AR glasses 20 as a display device for a predetermined time or longer.
　上記のように、ＡＲグラス２０は視線検出装置２３を備える。また、視線検出装置２３は、例えば、ユーザＵ１の目頭の位置と虹彩の位置に基づいて視線データを検出する。判定部１１５は、当該視線データによって示される視線の方向が、仮想空間ＶＳとしての天球の球面と衝突する点の座標を、ユーザＵ１の視点の位置とする。図７に例示されるように、第１情報ＩＦ１には各第１の仮想オブジェクトＶＯの位置情報が含まれる。当該位置情報によって示される位置に第１の仮想オブジェクトＶＯが配置される。判定部１１５は、第１の仮想オブジェクトＶＯが配置される領域内に、ユーザＵ１の視点が所定時間以上位置するか否かを判定する。 As described above, the AR glasses 20 are equipped with the line-of-sight detection device 23. Also, the line-of-sight detection device 23 detects line-of-sight data based on, for example, the position of the inner corner of the eye and the position of the iris of the user U1. The determination unit 115 determines the coordinates of the point where the line-of-sight direction indicated by the line-of-sight data collides with the spherical surface of the celestial sphere as the virtual space VS as the viewpoint position of the user U1. As illustrated in FIG. 7, the first information IF1 includes position information of each first virtual object VO. The first virtual object VO is arranged at the position indicated by the position information. The determination unit 115 determines whether or not the viewpoint of the user U1 is located within the area where the first virtual object VO is arranged for a predetermined time or longer.
　図１３Ａ及び図１３Ｂは、とりわけ判定部１１５と連携した場合の、表示制御部１１４の第３の動作例についての説明図である。図１３Ａに示されるように、仮想空間ＶＳにおいて、特定の仮想オブジェクトＶＯ内に、ユーザＵ１の視点が所定時間以上位置すると判定された場合に、表示制御部１１４は、図１３Ｂに示されるように、当該仮想オブジェクトＶＯの近傍に、当該仮想オブジェクトＶＯに対応するタグＴＧを表示させる。ここで、当該仮想オブジェクトＶＯの「近傍」とは、具体的には、当該仮想オブジェクトＶＯから所定距離以内の範囲のことである。図１３Ａ及び図１３Ｂに示される例においては、特定の仮想オブジェクトＶＯ７内に、ユーザＵ１の視点が所定時間以上位置すると判定された場合に、表示制御部１１４は、当該仮想オブジェクトＶＯ７の近傍に、当該仮想オブジェクトＶＯ７に対応するタグＴＧ７である“＃ＦＯＸ”を表示させる。 FIGS. 13A and 13B are explanatory diagrams of a third operation example of the display control unit 114 especially in cooperation with the determination unit 115. FIG. As shown in FIG. 13A, in the virtual space VS, when it is determined that the viewpoint of the user U1 is positioned within the specific virtual object VO for a predetermined time or longer, the display control unit 114 performs the following operations as shown in FIG. 13B. , the tag TG corresponding to the virtual object VO is displayed near the virtual object VO. Here, the “neighborhood” of the virtual object VO specifically means a range within a predetermined distance from the virtual object VO. In the example shown in FIGS. 13A and 13B, when it is determined that the user U1's viewpoint is positioned within a specific virtual object VO7 for a predetermined time or longer, the display control unit 114 displays a "#FOX", which is the tag TG7 corresponding to the virtual object VO7, is displayed.
　図６に説明を戻すと、音声認識部１１６は、ユーザＵ１が発話する音声を認識する。 Returning to FIG. 6, the speech recognition unit 116 recognizes the speech uttered by the user U1.
　上記のように、ＡＲグラス２０は収音装置２４を備える。ＡＲグラス２０を頭部に装着したユーザＵ１によって発話された音声は、収音装置２４によって収音され、音声データに変換される。音声データはＡＲグラス２０から端末装置１０に出力される。音声認識部１１６は、ＡＲグラス２０から取得した音声データに基づいて、発話の内容を認識する。より具体的には、音声認識部１１６は、音声データをテキストデータに変換する。 As described above, the AR glasses 20 are equipped with the sound pickup device 24. A sound uttered by the user U1 wearing the AR glasses 20 on the head is picked up by the sound pickup device 24 and converted into sound data. Audio data is output from the AR glasses 20 to the terminal device 10 . The voice recognition unit 116 recognizes the contents of the utterance based on the voice data acquired from the AR glasses 20 . More specifically, speech recognition unit 116 converts speech data into text data.
　図１４Ａ及び図１４Ｂは、とりわけ音声認識部１１６と連携した場合の、表示制御部１１４の第４の動作例についての説明図である。図１４Ａに示されるように、ＡＲグラス２０を頭部に装着したユーザＵ１によって、仮想オブジェクトＶＯの属性が発話されると、音声認識部１１６は、ユーザＵ１が発話した音声を示す音声データを認識する。音声認識部１１６による認識結果が、第２情報ＩＦ２に含まれる１つの属性を示す場合に、仮想オブジェクト特定部１１３－１は、第２情報ＩＦ２に基づいて、当該１つの属性に対応する１以上の仮想オブジェクトＶＯを特定する。 FIGS. 14A and 14B are explanatory diagrams of a fourth operation example of the display control unit 114 especially in cooperation with the speech recognition unit 116. FIG. As shown in FIG. 14A, when the user U1 wearing the AR glasses 20 on his or her head utters the attribute of the virtual object VO, the voice recognition unit 116 recognizes the voice data representing the voice uttered by the user U1. do. When the recognition result by the speech recognition unit 116 indicates one attribute included in the second information IF2, the virtual object specifying unit 113-1 generates one or more attributes corresponding to the one attribute based on the second information IF2. to identify the virtual object VO.
　図１４Ａに示す例において、ユーザＵ１は、上記の属性として「動物」という言葉を発話する。音声認識部１１６は、ユーザＵ１が発話した音声を「動物」という文字列として認識する。「動物」という認識結果に対応する仮想オブジェクトＶＯは、鹿の３Ｄモデルである仮想オブジェクトＶＯ６、キツネの３Ｄモデルである仮想オブジェクトＶＯ７、及び馬の３Ｄモデルである仮想オブジェクトＶＯ８である。このため、仮想オブジェクト特定部１１３－１は、複数の仮想オブジェクトＶＯのうち、「動物」という属性に対応する仮想オブジェクトＶＯとして、仮想オブジェクトＶＯ６～ＶＯ８を特定する。 In the example shown in FIG. 14A, user U1 utters the word "animal" as the above attribute. The voice recognition unit 116 recognizes the voice uttered by the user U1 as the character string "animal". The virtual objects VO corresponding to the recognition result "animal" are a virtual object VO6 that is a 3D model of a deer, a virtual object VO7 that is a 3D model of a fox, and a virtual object VO8 that is a 3D model of a horse. Therefore, the virtual object specifying unit 113-1 specifies virtual objects VO6 to VO8 as virtual objects VO corresponding to the attribute "animal" among the plurality of virtual objects VO.
　また、呼び名特定部１１３－２は、第１情報ＩＦ１に基づいて、仮想オブジェクト特定部１１３－１によって特定された１以上の仮想オブジェクトＶＯの一部又は全部の仮想オブジェクトＶＯの各々に対応する１以上のタグＴＧを特定する。図１４Ａに示す例において、仮想オブジェクト特定部１１３－１によって特定された１以上の仮想オブジェクトＶＯの例としては、鹿の３Ｄモデルである仮想オブジェクトＶＯ６、キツネの３Ｄモデルである仮想オブジェクトＶＯ７、及び馬の３Ｄモデルである仮想オブジェクトＶＯ８が挙げられる。また、鹿の３Ｄモデルである仮想オブジェクトＶＯ６に対応するタグＴＧ６は存在しないとする。キツネの３Ｄモデルである仮想オブジェクトＶＯ７に対応するタグＴＧ７は、“＃ＦＯＸ”であるとする。馬の３Ｄモデルである仮想オブジェクトＶＯ８に対応するタグＴＧ８は、“＃ＨＯＲＳＥ”であるとする。よって、特定された１以上の仮想オブジェクトＶＯの一部は、キツネの３Ｄモデルである仮想オブジェクトＶＯ７、及び馬の３Ｄモデルである仮想オブジェクトＶＯ８となる。更に、特定された１以上の仮想オブジェクトＶＯの一部の仮想オブジェクトＶＯの各々に対応する１以上のタグＴＧは、“＃ＦＯＸ”のタグＴＧ７と“＃ＨＯＲＳＥ”のタグＴＧ８の２つのタグＴＧとなる。したがって、呼び名特定部１１３－２は、“＃ＦＯＸ”のタグＴＧ７と“＃ＨＯＲＳＥ”のタグＴＧ８の２つのタグＴＧを特定する。一方で、仮に、鹿の３Ｄモデルである仮想オブジェクトＶＯ６に対応するタグＴＧ６として、“＃ＤＥＥＲ”が割り当てられている場合、呼び名特定部１１３－２は、仮想オブジェクト特定部１１３－１によって特定された１以上の仮想オブジェクトＶＯの全部の仮想オブジェクトＶＯの各々に対応する１以上のタグＴＧを特定することとなる。 In addition, based on the first information IF1, the call name specifying unit 113-2 provides one virtual object VO corresponding to each of a part or all of the one or more virtual objects VO specified by the virtual object specifying unit 113-1. Identify the above tag TG. In the example shown in FIG. 14A, examples of one or more virtual objects VO identified by the virtual object identifying unit 113-1 include a virtual object VO6 that is a 3D model of a deer, a virtual object VO7 that is a 3D model of a fox, and a virtual object VO7 that is a 3D model of a fox. A virtual object VO8, which is a 3D model of a horse, can be mentioned. Also, assume that there is no tag TG6 corresponding to the virtual object VO6, which is a 3D model of a deer. It is assumed that the tag TG7 corresponding to the virtual object VO7, which is a 3D model of a fox, is "#FOX". It is assumed that the tag TG8 corresponding to the virtual object VO8, which is a 3D model of a horse, is "#HORSE". Therefore, some of the identified one or more virtual objects VO become a virtual object VO7 that is a 3D model of a fox and a virtual object VO8 that is a 3D model of a horse. Furthermore, the one or more tags TG corresponding to each of the partial virtual objects VO of the identified one or more virtual objects VO are divided into two tags TG, a tag TG7 of "#FOX" and a tag TG8 of "#HORSE". becomes. Therefore, the calling name identifying unit 113-2 identifies two tags TG, ie, the tag TG7 of "#FOX" and the tag TG8 of "#HORSE". On the other hand, if "#DEER" is assigned as the tag TG6 corresponding to the virtual object VO6, which is a 3D model of a deer, the calling name identifying unit 113-2 is identified by the virtual object identifying unit 113-1. In addition, one or more tags TG corresponding to each of all virtual objects VO of one or more virtual objects VO are specified.
　表示制御部１１４は、表示装置としてのＡＲグラス２０に対して、図１４Ｂに示されるように、仮想オブジェクト特定部１１３－１によって第２情報ＩＦ２に基づいて特定された１以上の仮想オブジェクトＶＯを表示させる。また、表示制御部１１４は、当該仮想オブジェクトＶＯに対応するタグＴＧが存在する場合には、当該タグＴＧを、対応する仮想オブジェクトＶＯに関連付けて表示させる。 The display control unit 114 displays one or more virtual objects VO specified by the virtual object specifying unit 113-1 based on the second information IF2, as shown in FIG. 14B, on the AR glasses 20 as the display device. display. Further, when a tag TG corresponding to the virtual object VO exists, the display control unit 114 displays the tag TG in association with the corresponding virtual object VO.
　図１４Ｂに示される例においては、仮想オブジェクト特定部１１３－１によって特定された仮想オブジェクトＶＯ６～ＶＯ８の各々を示すアイコンＩＣ６～ＩＣ８が、ポップアップＰ１内に表示される。更に、当該ポップアップＰ１内において、アイコンＩＣ７に“＃ＦＯＸ”というタグＴＧ７が、アイコンＩＣ８に“＃ＨＯＲＳＥ”というタグＴＧ８が付記される。 In the example shown in FIG. 14B, icons IC6 to IC8 indicating virtual objects VO6 to VO8 identified by the virtual object identifying unit 113-1 are displayed in the popup P1. Further, in the popup P1, the tag TG7 of "#FOX" is added to the icon IC7, and the tag TG8 of "#HORSE" is added to the icon IC8.
　また、音声認識部１１６による認識結果が、第１情報ＩＦ１に含まれるタグＴＧと一致した場合に、表示制御部１１４は、当該タグＴＧに対応する第１の仮想オブジェクトＶＯに関する表示内容を変更させる。例として、図１０Ａ～図１０Ｃを参照することにより説明したように、表示制御部１１４は、当該タグＴＧに対応する仮想オブジェクトＶＯを、ユーザＵ１の視線の方向に移動させてもよい。あるいは、当該タグＴＧに対応する仮想オブジェクトＶＯがアプリケーションである場合に、表示制御部１１４は、当該アプリケーションを起動するか否かを選択する画面を表示してもよい。 Further, when the recognition result by the voice recognition unit 116 matches the tag TG included in the first information IF1, the display control unit 114 changes the display content regarding the first virtual object VO corresponding to the tag TG. . As an example, as described with reference to FIGS. 10A to 10C, the display control unit 114 may move the virtual object VO corresponding to the tag TG in the line-of-sight direction of the user U1. Alternatively, when the virtual object VO corresponding to the tag TG is an application, the display control unit 114 may display a screen for selecting whether to start the application.
　なお、図６には示さないが、端末装置１０は、ＡＲグラス２０に備わるＧＰＳ装置２５と同様のＧＰＳ装置を備えてもよい。この場合、ＡＲグラス２０は、ＧＰＳ装置２５を備えなくてもよい。 Although not shown in FIG. 6, the terminal device 10 may include a GPS device similar to the GPS device 25 provided in the AR glasses 20. In this case, the AR glasses 20 do not have to be equipped with the GPS device 25 .
１．１．４：サーバの構成
　図１５は、サーバ３０の構成例を示すブロック図である。サーバ３０は、処理装置３１、記憶装置３２、通信装置３３、ディスプレイ３４、及び入力装置３５を備える。サーバ３０の各要素は、情報を通信するための単体又は複数のバスで相互に接続される。 1.1.4: Server Configuration FIG. 15 is a block diagram showing a configuration example of the server 30. As shown in FIG. The server 30 comprises a processing device 31 , a storage device 32 , a communication device 33 , a display 34 and an input device 35 . Each element of server 30 is interconnected by a bus or buses for communicating information.
　処理装置３１は、サーバ３０の全体を制御するプロセッサであり、例えば、単数又は複数のチップを用いて構成される。処理装置３１は、例えば、周辺装置とのインタフェース、演算装置及びレジスタ等を含む中央処理装置（ＣＰＵ）を用いて構成される。なお、処理装置３１の機能の一部又は全部を、ＤＳＰ、ＡＳＩＣ、ＰＬＤ、又はＦＰＧＡ等のハードウェアによって実現してもよい。処理装置３１は、各種の処理を並列的又は逐次的に実行する。 The processing device 31 is a processor that controls the entire server 30, and is configured using, for example, one or more chips. The processing unit 31 is configured using, for example, a central processing unit (CPU) including interfaces with peripheral devices, arithmetic units, registers, and the like. A part or all of the functions of the processing device 31 may be realized by hardware such as DSP, ASIC, PLD, or FPGA. The processing device 31 executes various processes in parallel or sequentially.
　記憶装置３２は、処理装置３１が読取及び書込が可能な記録媒体であり、処理装置３１が実行する制御プログラムＰＲ３を含む複数のプログラム、第１情報ＩＦ１、及び第２情報ＩＦ２を記憶する。 The storage device 32 is a recording medium readable and writable by the processing device 31, and stores a plurality of programs including the control program PR3 executed by the processing device 31, first information IF1, and second information IF2.
　通信装置３３は、他の装置と通信を行うための、送受信デバイスとしてのハードウェアである。通信装置３３は、例えば、ネットワークデバイス、ネットワークコントローラ、ネットワークカード、又は通信モジュール等とも呼ばれる。通信装置３３は、有線接続用のコネクターを備え、上記コネクターに対応するインタフェース回路を備えていてもよい。また、通信装置３３は、無線通信インタフェースを備えていてもよい。有線接続用のコネクター及びインタフェース回路としては有線ＬＡＮ、ＩＥＥＥ１３９４、ＵＳＢに準拠した製品が挙げられる。また、無線通信インタフェースとしては無線ＬＡＮ及びＢｌｕｅｔｏｏｔｈ（登録商標）等に準拠した製品が挙げられる。 The communication device 33 is hardware as a transmission/reception device for communicating with other devices. The communication device 33 is also called, for example, a network device, a network controller, a network card, or a communication module. The communication device 33 may include a connector for wired connection and an interface circuit corresponding to the connector. Further, the communication device 33 may have a wireless communication interface. Products conforming to wired LAN, IEEE1394, and USB are examples of connectors and interface circuits for wired connection. Also, as a wireless communication interface, there are products conforming to wireless LAN, Bluetooth (registered trademark), and the like.
　ディスプレイ３４は、画像及び文字情報を表示するデバイスである。ディスプレイ３４は、処理装置３１による制御のもとで各種の画像を表示する。例えば、液晶表示パネル及び有機ＥＬ表示パネル等の各種の表示パネルがディスプレイ３４として好適に利用される。 The display 34 is a device that displays images and character information. The display 34 displays various images under the control of the processing device 31 . For example, various display panels such as a liquid crystal display panel and an organic EL display panel are preferably used as the display 34 .
　入力装置３５は、情報処理システム１の管理者による操作を受け付ける機器である。例えば、入力装置３５は、キーボード、タッチパッド、タッチパネル又はマウス等のポインティングデバイスを含んで構成される。ここで、入力装置３５は、タッチパネルを含んで構成される場合、ディスプレイ３４を兼ねてもよい。とりわけ、情報処理システム１の管理者は、入力装置３５を用いることにより、第１情報ＩＦ１及び第２情報ＩＦ２を入力したり、編集したりすることが可能である。 The input device 35 is a device that accepts operations by the administrator of the information processing system 1 . For example, the input device 35 includes a pointing device such as a keyboard, touch pad, touch panel, or mouse. Here, when the input device 35 includes a touch panel, the input device 35 may also serve as the display 34 . In particular, the administrator of the information processing system 1 can use the input device 35 to input and edit the first information IF1 and the second information IF2.
　処理装置３１は、例えば、記憶装置３２から制御プログラムＰＲ３を読み出して実行することによって、出力部３１１、及び取得部３１２として機能する。 The processing device 31 functions as an output unit 311 and an acquisition unit 312 by reading and executing the control program PR3 from the storage device 32, for example.
　出力部３１１は、記憶装置３２に格納される第１情報ＩＦ１及び第２情報ＩＦ２を、通信装置３３を用いることにより、端末装置１０に出力する。また、出力部３１１は、ＡＲグラス２０を頭部に装着したユーザＵ１に対して、端末装置１０が仮想空間ＶＳを提供する上で必要となるデータを、端末装置１０に出力する。当該データには、仮想オブジェクトＶＯ自体に係るデータ、及び、クラウドサービスを利用するための図示しないアプリケーションに係るデータが含まれる。 The output unit 311 outputs the first information IF1 and the second information IF2 stored in the storage device 32 to the terminal device 10 by using the communication device 33 . In addition, the output unit 311 outputs to the terminal device 10 data necessary for the terminal device 10 to provide the virtual space VS to the user U1 wearing the AR glasses 20 on the head. The data includes data related to the virtual object VO itself and data related to an application (not shown) for using the cloud service.
　取得部３１２は、通信装置３３を用いることにより、端末装置１０から各種のデータを取得する。当該データには、例として、ＡＲグラス２０を頭部に装着したユーザＵ１によって端末装置１０に入力される、仮想オブジェクトＶＯに対する操作内容を示すデータが含まれる。また、当該ユーザＵ１が、上記のクラウドサービスを利用する場合、当該データには、上記のアプリケーションへの入力データが含まれる。 The acquisition unit 312 acquires various data from the terminal device 10 by using the communication device 33 . The data includes, for example, data indicating the operation content for the virtual object VO, which is input to the terminal device 10 by the user U1 wearing the AR glasses 20 on the head. Further, when the user U1 uses the above cloud service, the data includes input data to the above application.
２：第１実施形態の動作
　図１６～図１９は、第１実施形態に係る情報処理システム１の動作を示すフローチャートである。以下、図１６～図１９を参照することにより、情報処理システム１の動作について説明する。 2: Operation of First Embodiment FIGS. 16 to 19 are flowcharts showing the operation of the information processing system 1 according to the first embodiment. The operation of the information processing system 1 will be described below with reference to FIGS. 16 to 19. FIG.
１．２．１：第１の動作
　図１６は、情報処理システム１の第１の動作について説明するフローチャートである。 1.2.1: First Operation FIG. 16 is a flow chart explaining the first operation of the information processing system 1 .
　ステップＳ１において、処理装置１１は、取得部１１１として機能することにより、操作信号を取得する。当該操作信号は、図１１Ａ及び図１１Ｂを用いて例示したように、ＡＲグラス２０に備わるディスプレイ２９に対して、天球型の仮想空間ＶＳを平面展開した、２次元画像ＳＩを表示させるためのトリガである。 In step S<b>1 , the processing device 11 acquires an operation signal by functioning as the acquisition unit 111 . 11A and 11B, the operation signal is a trigger for causing the display 29 provided in the AR glasses 20 to display a two-dimensional image SI obtained by planarly developing the celestial virtual space VS. is.
　ステップＳ２において、処理装置１１は、表示制御部１１４として機能することにより、ＡＲグラス２０に備わるディスプレイ２９に対して、天球型の仮想空間ＶＳを平面展開した、２次元画像ＳＩを表示させる。 In step S2, the processing device 11 functions as the display control unit 114 to cause the display 29 provided in the AR glasses 20 to display a two-dimensional image SI obtained by planarly developing the celestial virtual space VS.
　ステップＳ３において、処理装置１１は、取得部１１１として機能することにより、指示情報を取得する。当該指示情報は、仮想空間ＶＳに配置される複数の仮想オブジェクトＶＯのうち、第１の仮想オブジェクトＶＯを特定するための情報である。 In step S3, the processing device 11 functions as the acquisition unit 111 to acquire instruction information. The instruction information is information for specifying the first virtual object VO among the plurality of virtual objects VO arranged in the virtual space VS.
　ステップＳ４において、処理装置１１は、仮想オブジェクト特定部１１３－１として機能することにより、指示情報に基づいて、複数の仮想オブジェクトＶＯのうち、第１の仮想オブジェクトＶＯを特定する。また、処理装置１１が、仮想オブジェクト特定部１１３－１として機能することにより、第１の仮想オブジェクトＶＯを特定した結果として、当該第１の仮想オブジェクトＶＯに対応するＩＤが出力される。 In step S4, the processing device 11, by functioning as the virtual object specifying unit 113-1, specifies the first virtual object VO among the plurality of virtual objects VO based on the instruction information. Further, the processing device 11 functions as the virtual object specifying unit 113-1, and as a result of specifying the first virtual object VO, an ID corresponding to the first virtual object VO is output.
　ステップＳ５において、処理装置１１は、呼び名特定部１１３－２として機能することにより、記憶装置１２に記憶される第１情報ＩＦ１を参照する。その上で、処理装置１１は、呼び名特定部１１３－２として機能することにより、仮想オブジェクト特定部１１３－１として機能することによって特定された第１の仮想オブジェクトＶＯに対応するタグＴＧが、第１情報ＩＦ１に含まれるか否かを判定する。具体的には、処理装置１１は、呼び名特定部１１３－２として機能することにより、ステップＳ４において出力されたＩＤに対応するタグＴＧが、第１情報ＩＦ１に含まれるか否かを判定する。タグＴＧが第１情報ＩＦ１に含まれる場合、すなわちステップＳ５の判定結果がＹＥＳの場合、処理装置１１は、呼び名特定部１１３－２として機能することにより、当該タグＴＧを第１の呼び名として特定した上で、ステップＳ６の処理を実行する。タグＴＧが第１情報ＩＦ１に含まれない場合、すなわちステップＳ５の判定結果がＮＯの場合、処理装置１１は、全ての処理を終了する。 In step S5, the processing device 11 refers to the first information IF1 stored in the storage device 12 by functioning as the calling name identification unit 113-2. After that, the processing device 11 functions as the call name identifying unit 113-2 so that the tag TG corresponding to the first virtual object VO identified by functioning as the virtual object identifying unit 113-1 is changed to the first virtual object VO. 1 information IF1. Specifically, the processing device 11 functions as the calling name identification unit 113-2 to determine whether the tag TG corresponding to the ID output in step S4 is included in the first information IF1. If the tag TG is included in the first information IF1, that is, if the determination result in step S5 is YES, the processing device 11 functions as the nickname specifying unit 113-2 to specify the tag TG as the first nickname. After that, the process of step S6 is executed. If the tag TG is not included in the first information IF1, that is, if the determination result in step S5 is NO, the processing device 11 ends all the processes.
　ステップＳ６において、処理装置１１は、表示制御部１１４として機能することにより、呼び名特定部１１３－２によって特定されたタグＴＧを、ＡＲグラス２０に備わるディスプレイ２９に表示させる。より詳細には、処理装置１１は、表示制御部１１４として機能することにより、ディスプレイ２９に表示された２次元画像ＳＩにおいて、特定されたタグＴＧを表示させる。なお、ステップＳ５において、タグＴＧが特定されなかった場合には、処理装置１１はステップＳ６の処理を省略する。 In step S6, the processing device 11 functions as the display control unit 114 to display the tag TG specified by the name specifying unit 113-2 on the display 29 of the AR glasses 20. More specifically, the processing device 11 functions as the display control unit 114 to display the specified tag TG in the two-dimensional image SI displayed on the display 29 . Note that if the tag TG is not specified in step S5, the processing device 11 omits the processing of step S6.
１．２．２：第２の動作
　図１７は、情報処理システム１の第２の動作について説明するフローチャートである。 1.2.2: Second Operation FIG. 17 is a flow chart explaining the second operation of the information processing system 1 .
　ステップＳ１１において、処理装置１１は、取得部１１１として機能することにより、操作信号を取得する。当該操作信号は、図１２Ａ及び図１２Ｂを用いて例示したように、ＡＲグラス２０に備わるディスプレイ２９に対して、天球型の仮想空間ＶＳを縮小した３次元画像ＴＩを表示させるためのトリガである。 In step S<b>11 , the processing device 11 acquires the operation signal by functioning as the acquisition unit 111 . 12A and 12B, the operation signal is a trigger for causing the display 29 provided in the AR glasses 20 to display a three-dimensional image TI obtained by reducing the celestial sphere virtual space VS. .
　ステップＳ１２において、処理装置１１は、表示制御部１１４として機能することにより、ＡＲグラス２０に備わるディスプレイ２９に対して、天球型の仮想空間ＶＳを縮小した３次元画像ＴＩを表示させる。 In step S12, the processing device 11 functions as the display control unit 114 to cause the display 29 provided in the AR glasses 20 to display a three-dimensional image TI obtained by reducing the celestial sphere-shaped virtual space VS.
　ステップＳ１３において、処理装置１１は、取得部１１１として機能することにより、指示情報を取得する。当該指示情報は、仮想空間ＶＳに配置される複数の仮想オブジェクトＶＯのうち、第１の仮想オブジェクトＶＯの特定するための情報である。 In step S<b>13 , the processing device 11 acquires the instruction information by functioning as the acquisition unit 111 . The instruction information is information for specifying the first virtual object VO among the plurality of virtual objects VO arranged in the virtual space VS.
　ステップＳ１４において、処理装置１１は、仮想オブジェクト特定部１１３－１として機能することにより、指示情報に基づいて、複数の仮想オブジェクトＶＯのうち、第１の仮想オブジェクトＶＯを特定する。また、処理装置１１が、仮想オブジェクト特定部１１３－１として機能することにより、第１の仮想オブジェクトＶＯを特定した結果として、当該第１の仮想オブジェクトＶＯに対応するＩＤが出力される。 In step S14, the processing device 11, by functioning as the virtual object specifying unit 113-1, specifies the first virtual object VO among the plurality of virtual objects VO based on the instruction information. Further, the processing device 11 functions as the virtual object specifying unit 113-1, and as a result of specifying the first virtual object VO, an ID corresponding to the first virtual object VO is output.
　ステップＳ１５において、処理装置１１は、呼び名特定部１１３－２として機能することにより、記憶装置１２に記憶される第１情報ＩＦ１を参照する。その上で、処理装置１１は、呼び名特定部１１３－２として機能することにより、仮想オブジェクト特定部１１３－１として機能することによって特定された第１の仮想オブジェクトＶＯに対応するタグＴＧが、第１情報ＩＦ１に含まれるか否かを判定する。具体的には、処理装置１１は、呼び名特定部１１３－２として機能することにより、ステップＳ１４において出力されたＩＤに対応するタグＴＧが、第１情報ＩＦ１に含まれるか否かを判定する。タグＴＧが第１情報ＩＦ１に含まれる場合、すなわちステップＳ１５の判定結果がＹＥＳの場合、処理装置１１は、呼び名特定部１１３－２として機能することにより、当該タグＴＧを第１の呼び名として特定した上で、ステップＳ１６の処理を実行する。タグＴＧが第１情報ＩＦ１に含まれない場合、すなわちステップＳ１５の判定結果がＮＯの場合、処理装置１１は、全ての処理を終了する。 In step S15, the processing device 11 refers to the first information IF1 stored in the storage device 12 by functioning as the calling name identification unit 113-2. After that, the processing device 11 functions as the call name identifying unit 113-2 so that the tag TG corresponding to the first virtual object VO identified by functioning as the virtual object identifying unit 113-1 is changed to the first virtual object VO. 1 information IF1. Specifically, the processing device 11 functions as the nickname identification unit 113-2 to determine whether the tag TG corresponding to the ID output in step S14 is included in the first information IF1. If the tag TG is included in the first information IF1, that is, if the determination result in step S15 is YES, the processing device 11 functions as the nickname identification unit 113-2 to identify the tag TG as the first nickname. After that, the process of step S16 is executed. If the tag TG is not included in the first information IF1, that is, if the determination result in step S15 is NO, the processing device 11 ends all the processes.
　ステップＳ１６において、処理装置１１は、表示制御部１１４として機能することにより、呼び名特定部１１３－２によって特定されたタグＴＧを、ＡＲグラス２０に備わるディスプレイ２９に表示させる。より詳細には、処理装置１１は、表示制御部１１４として機能することにより、ディスプレイ２９に表示された３次元画像ＴＩにおいて、特定されたタグＴＧを表示させる。なお、ステップＳ１５において、タグＴＧが特定されなかった場合には、処理装置１１はステップＳ１６の処理を省略する。 In step S16, the processing device 11 functions as the display control unit 114 to display the tag TG specified by the name specifying unit 113-2 on the display 29 of the AR glasses 20. More specifically, the processing device 11 functions as the display control unit 114 to display the specified tag TG in the three-dimensional image TI displayed on the display 29 . Note that if the tag TG is not specified in step S15, the processing device 11 omits the process of step S16.
１．２．３：第３の動作
　図１８は、情報処理システム１の第３の動作について説明するフローチャートである。 1.2.3: Third Operation FIG. 18 is a flow chart explaining the third operation of the information processing system 1 .
　ステップＳ２１において、処理装置１１は、取得部１１１として機能することにより、ＡＲグラス２０におけるユーザＵ１の視線に係る視線データを取得する。より詳細には、ＡＲグラス２０の処理装置２１は、取得部２１１として機能することにより、取得した視線データを、通信装置２８に出力する。通信装置２８は、処理装置２１から取得した視線データを、端末装置１０に出力する。端末装置１０の処理装置１１は、取得部１１１として機能することにより、通信装置１３を用いて、ＡＲグラス２０から、視線データを取得する。 In step S<b>21 , the processing device 11 functions as the acquisition unit 111 to acquire line-of-sight data related to the line of sight of the user U<b>1 on the AR glasses 20 . More specifically, the processing device 21 of the AR glasses 20 outputs the acquired line-of-sight data to the communication device 28 by functioning as the acquisition unit 211 . The communication device 28 outputs the line-of-sight data acquired from the processing device 21 to the terminal device 10 . The processing device 11 of the terminal device 10 functions as the acquisition unit 111 to acquire line-of-sight data from the AR glasses 20 using the communication device 13 .
　ステップＳ２２において、処理装置１１は、判定部１１５として機能することにより、ＡＲグラス２０に表示される第１の仮想オブジェクトＶＯ内に、ユーザＵ１の視点が所定時間以上位置するか否かを判定する。より詳細には、処理装置１１は、判定部１１５として機能することにより、ステップＳ２１において取得された視線データに基づいて、指示情報としての、ユーザＵ１の視点の位置を取得する。その上で、処理装置１１は、判定部１１５として、特定の仮想オブジェクトＶＯに、ユーザＵ１の視点が所定時間以上位置したか否かを判定する。判定結果が真である場合、すなわちステップＳ２２の判定結果が“ＹＥＳ”である場合には、処理装置１１は、ステップＳ２３の処理を実行する。判定結果が偽である場合、すなわちステップＳ２２の判定結果が“ＮＯ”である場合には、処理装置１１は、ステップＳ２１の処理を実行する。 In step S22, the processing device 11 functions as the determination unit 115 to determine whether or not the viewpoint of the user U1 is positioned within the first virtual object VO displayed on the AR glasses 20 for a predetermined time or longer. . More specifically, the processing device 11 functions as the determination unit 115 to acquire the viewpoint position of the user U1 as instruction information based on the line-of-sight data acquired in step S21. After that, the processing device 11, as the determination unit 115, determines whether or not the viewpoint of the user U1 has been positioned at the specific virtual object VO for a predetermined time or longer. When the determination result is true, that is, when the determination result of step S22 is "YES", the processing device 11 executes the process of step S23. When the determination result is false, that is, when the determination result of step S22 is "NO", the processing device 11 executes the process of step S21.
　ステップＳ２３において、処理装置１１は、仮想オブジェクト特定部１１３－１として機能することにより、ステップＳ２２における判定結果に基づいて、複数の仮想オブジェクトＶＯのうち、第１の仮想オブジェクトＶＯを特定する。より詳細には、処理装置１１は、仮想オブジェクト特定部１１３－１として機能することにより、ステップＳ２２において、ユーザＵ１の視点が所定時間以上位置したと判定された仮想オブジェクトＶＯを、第１の仮想オブジェクトＶＯとして特定する。また、処理装置１１が、仮想オブジェクト特定部１１３－１として機能することにより、第１の仮想オブジェクトＶＯを特定した結果として、当該第１の仮想オブジェクトＶＯに対応するＩＤが出力される。 In step S23, the processing device 11 functions as the virtual object specifying unit 113-1 to specify the first virtual object VO among the plurality of virtual objects VO based on the determination result in step S22. More specifically, the processing device 11, by functioning as the virtual object specifying unit 113-1, determines in step S22 that the virtual object VO for which the viewpoint of the user U1 has been positioned for a predetermined period of time or more is assigned to the first virtual object VO. Identify as object VO. Further, the processing device 11 functions as the virtual object specifying unit 113-1, and as a result of specifying the first virtual object VO, an ID corresponding to the first virtual object VO is output.
　ステップＳ２４において、処理装置１１は、呼び名特定部１１３－２として機能することにより、記憶装置１２に記憶される第１情報ＩＦ１を参照する。その上で、処理装置１１は、呼び名特定部１１３－２として機能することにより、仮想オブジェクト特定部１１３－１として機能することによって特定された第１の仮想オブジェクトＶＯに対応するタグＴＧが、第１情報ＩＦ１に含まれるか否かを判定する。具体的には、処理装置１１は、呼び名特定部１１３－２として機能することにより、ステップＳ２３において出力されたＩＤに対応するタグＴＧが、第１情報ＩＦ１に含まれるか否かを判定する。タグＴＧが第１情報ＩＦ１に含まれる場合、すなわちステップＳ２４の判定結果がＹＥＳの場合、処理装置１１は、呼び名特定部１１３－２として機能することにより、当該タグＴＧを第１の呼び名として特定した上で、ステップＳ２５の処理を実行する。タグＴＧが第１情報ＩＦ１に含まれない場合、すなわちステップＳ２４の判定結果がＮＯの場合、処理装置１１は、全ての処理を終了する。 In step S24, the processing device 11 refers to the first information IF1 stored in the storage device 12 by functioning as the calling name identification unit 113-2. After that, the processing device 11 functions as the call name identifying unit 113-2 so that the tag TG corresponding to the first virtual object VO identified by functioning as the virtual object identifying unit 113-1 is changed to the first virtual object VO. 1 information IF1. Specifically, the processing device 11 functions as the calling name identification unit 113-2 to determine whether the tag TG corresponding to the ID output in step S23 is included in the first information IF1. If the tag TG is included in the first information IF1, that is, if the determination result in step S24 is YES, the processing device 11 functions as the nickname identification unit 113-2 to identify the tag TG as the first nickname. After that, the process of step S25 is executed. If the tag TG is not included in the first information IF1, that is, if the determination result in step S24 is NO, the processing device 11 terminates all the processes.
　ステップＳ２５において、処理装置１１は、表示制御部１１４として機能することにより、第１の呼び名として特定されたタグＴＧを、ＡＲグラス２０に備わるディスプレイ２９に表示させる。より詳細には、処理装置１１は、表示制御部１１４として機能することにより、ディスプレイ２９に対して、第１の仮想オブジェクトＶＯの近傍に、特定されたタグＴＧを表示させる。ここで、第１の仮想オブジェクトＶＯの「近傍」とは、具体的には、第１の仮想オブジェクトＶＯから所定距離以内の範囲のことである。なお、ステップＳ２５において、タグＴＧが特定されなかった場合には、処理装置１１はステップＳ２６の処理を省略する。 In step S<b>25 , the processing device 11 functions as the display control unit 114 to display the tag TG specified as the first nickname on the display 29 provided in the AR glasses 20 . More specifically, the processing device 11 functions as the display control unit 114 to cause the display 29 to display the specified tag TG in the vicinity of the first virtual object VO. Here, the “neighborhood” of the first virtual object VO specifically means a range within a predetermined distance from the first virtual object VO. Note that if the tag TG is not identified in step S25, the processing device 11 omits the processing of step S26.
１．２．４：第４の動作
　図１９は、情報処理システム１の第４の動作について説明するフローチャートである。 1.2.4: Fourth Operation FIG. 19 is a flow chart explaining the fourth operation of the information processing system 1 .
　ステップＳ３１において、処理装置１１は、音声認識部１１６として機能することにより、ユーザＵ１が発話した音声を認識する。より詳細には、ＡＲグラス２０の処理装置２１は、取得部２１１として機能することにより、収音装置２４から、ユーザＵ１の音声を示す音声データを取得する。また、ＡＲグラス２０の処理装置２１は、取得部２１１として機能することにより、取得した音声データを、通信装置２８に出力する。通信装置２８は、処理装置２１から取得した音声データを、端末装置１０に出力する。端末装置１０の処理装置１１は、取得部１１１として機能することにより、通信装置１３を用いて、ＡＲグラス２０から、音声データを取得する。また、端末装置１０の処理装置１１は、音声認識部１１６として機能することにより、音声データを音声認識する。音声認識結果としての文字列は、上記の第１の動作～第３の動作における指示情報に相当する。なお、本動作例において、音声認識結果としての文字列は、属性を示す文字列であるとする。 In step S31, the processing device 11 functions as the speech recognition unit 116 to recognize the speech uttered by the user U1. More specifically, the processing device 21 of the AR glasses 20 acquires voice data representing the voice of the user U1 from the sound pickup device 24 by functioning as the acquisition unit 211 . Also, the processing device 21 of the AR glasses 20 outputs the acquired audio data to the communication device 28 by functioning as the acquisition unit 211 . The communication device 28 outputs the audio data acquired from the processing device 21 to the terminal device 10 . The processing device 11 of the terminal device 10 functions as the acquisition unit 111 to acquire audio data from the AR glasses 20 using the communication device 13 . Further, the processing device 11 of the terminal device 10 performs voice recognition on voice data by functioning as a voice recognition unit 116 . A character string as a speech recognition result corresponds to instruction information in the first to third operations described above. In this operation example, it is assumed that a character string as a speech recognition result is a character string indicating an attribute.
　ステップＳ３２において、処理装置１１は、仮想オブジェクト特定部１１３－１として機能することにより、仮想オブジェクトＶＯを特定する。より詳細には、処理装置１１は、仮想オブジェクト特定部１１３－１として機能することにより、記憶装置１２に記憶される第２情報ＩＦ２を参照する。また、処理装置１１は、仮想オブジェクト特定部１１３－１として機能することにより、音声認識結果としての文字列が第２情報ＩＦ２に含まれる属性に対応する、１又は複数の仮想オブジェクトＶＯを特定する。また、処理装置１１が、仮想オブジェクト特定部１１３－１として機能することにより、１又は複数の仮想オブジェクトＶＯを特定した結果として、当該１又は複数の仮想オブジェクトＶＯに対応するＩＤが出力される。 In step S32, the processing device 11 identifies the virtual object VO by functioning as the virtual object identification unit 113-1. More specifically, the processing device 11 refers to the second information IF2 stored in the storage device 12 by functioning as the virtual object identification unit 113-1. Further, the processing device 11, by functioning as the virtual object specifying unit 113-1, specifies one or more virtual objects VO corresponding to the attribute whose character string as the speech recognition result is included in the second information IF2. . Further, the processing device 11 functions as the virtual object specifying unit 113-1, and as a result of specifying one or more virtual objects VO, an ID corresponding to the one or more virtual objects VO is output.
　ステップＳ３３において、処理装置１１は、呼び名特定部１１３－２として機能することにより、記憶装置１２に記憶される第１情報ＩＦ１を参照する。その上で、処理装置１１は、呼び名特定部１１３－２として機能することにより、仮想オブジェクト特定部１１３－１として機能することによって特定された第１の仮想オブジェクトＶＯに対応するタグＴＧが、第１情報ＩＦ１に含まれるか否かを判定する。具体的には、処理装置１１は、呼び名特定部１１３－２として機能することにより、ステップＳ３２において出力されたＩＤに対応するタグＴＧが、第１情報ＩＦ１に含まれるか否かを判定する。タグＴＧが第１情報ＩＦ１に含まれる場合、すなわちステップＳ３３の判定結果がＹＥＳの場合、処理装置１１は、呼び名特定部１１３－２として機能することにより、当該タグＴＧを第１の呼び名として特定した上で、ステップＳ３４の処理を実行する。タグＴＧが第１情報ＩＦ１に含まれない場合、すなわちステップＳ３３の判定結果がＮＯの場合、処理装置１１は、全ての処理を終了する。 In step S33, the processing device 11 refers to the first information IF1 stored in the storage device 12 by functioning as the nickname identification unit 113-2. After that, the processing device 11 functions as the call name identifying unit 113-2 so that the tag TG corresponding to the first virtual object VO identified by functioning as the virtual object identifying unit 113-1 is changed to the first virtual object VO. 1 information IF1. Specifically, the processing device 11 functions as the nickname identification unit 113-2 to determine whether the tag TG corresponding to the ID output in step S32 is included in the first information IF1. If the tag TG is included in the first information IF1, that is, if the determination result in step S33 is YES, the processing device 11 functions as the nickname specifying unit 113-2 to specify the tag TG as the first nickname. After that, the process of step S34 is executed. If the tag TG is not included in the first information IF1, that is, if the determination result in step S33 is NO, the processing device 11 ends all the processes.
　ステップＳ３４において、処理装置１１は、表示制御部１１４として機能することにより、第１の呼び名として特定されたタグＴＧを、ＡＲグラス２０に備わるディスプレイ２９に表示させる。一例として、処理装置１１は、表示制御部１１４として機能することにより、ディスプレイ２９に対して、ポップアップＰ１内において仮想オブジェクトＶＯに付記する形で、特定されたタグＴＧを表示させる。なお、ステップＳ３３において、タグＴＧが特定されなかった場合には、処理装置１１はステップＳ３４の処理を省略する。 In step S<b>34 , the processing device 11 functions as the display control unit 114 to display the tag TG specified as the first nickname on the display 29 provided in the AR glasses 20 . As an example, the processing device 11 functions as the display control unit 114 to cause the display 29 to display the specified tag TG in a form attached to the virtual object VO within the popup P1. Note that if the tag TG is not specified in step S33, the processing device 11 omits the processing of step S34.
１．３：第１実施形態が奏する効果
　以上の説明によれば、情報処理装置としての端末装置１０は、表示制御部１１４と、仮想オブジェクト特定部１１３－１と、呼び名特定部１１３－２とを備える。表示制御部１１４は、ユーザＵ１の頭部に装着される表示装置としてのＡＲグラス２０に、仮想空間ＶＳに配置される複数の仮想オブジェクトＶＯを表示させる。仮想オブジェクト特定部１１３－１は、ユーザＵ１の動作に応じて生成される指示情報に基づいて、複数の仮想オブジェクトＶＯのうち第１の仮想オブジェクトＶＯを特定する。呼び名特定部１１３－２は、仮想オブジェクト特定部１１３－１が特定した第１の仮想オブジェクトＶＯに対応するタグＴＧが、記憶装置１２に記憶されている場合、当該対応するタグＴＧを、第１の呼び名として特定する。表示制御部１１４は、表示装置としてのＡＲグラス２０に第１の呼び名としてのタグＴＧを表示させる。 1.3: Effects of the First Embodiment According to the above description, the terminal device 10 as an information processing device includes the display control unit 114, the virtual object identification unit 113-1, and the name identification unit 113-2. Prepare. The display control unit 114 causes the AR glasses 20 as a display device worn on the head of the user U1 to display a plurality of virtual objects VO arranged in the virtual space VS. The virtual object specifying unit 113-1 specifies the first virtual object VO among the plurality of virtual objects VO based on instruction information generated according to the user U1's action. If the tag TG corresponding to the first virtual object VO identified by the virtual object identifying unit 113-1 is stored in the storage device 12, the call name identifying unit 113-2 identifies the corresponding tag TG as the first virtual object VO. identified as a nickname for The display control unit 114 causes the AR glasses 20 as the display device to display the tag TG as the first name.
　情報処理装置としての端末装置１０は、上記の構成を用いることにより、ＡＲグラス２０を頭部に装着したユーザＵ１に対して、仮想空間ＶＳに配置される第１の仮想オブジェクトＶＯを特定するための第１の呼び名としてのタグＴＧを、簡便に想起させることが可能となる。具体的には、ユーザＵ１にとって各仮想オブジェクトＶＯと各タグＴＧとの対応関係を把握すること、及び当該タグＴＧを記憶することが困難な場合がある。このような場合に、ユーザＵ１が頭部に装着する、表示装置としてのＡＲグラス２０に対して、ユーザＵ１の動作に応じて生成される指示情報に基づいて特定される仮想オブジェクトＶＯと関連付ける形で、当該仮想オブジェクトＶＯに対応するタグＴＧを表示させる。ユーザＵ１は、ＡＲグラス２０において、表示されたタグＴＧを視認することにより、当該仮想オブジェクトＶＯに対応するタグＴＧを想起することが可能となる。 By using the above configuration, the terminal device 10 as an information processing device identifies the first virtual object VO arranged in the virtual space VS for the user U1 wearing the AR glasses 20 on the head. It is possible to easily recall the tag TG as the first name of the. Specifically, it may be difficult for the user U1 to grasp the correspondence between each virtual object VO and each tag TG and to store the tag TG. In such a case, the AR glasses 20 as a display device that the user U1 wears on the head are associated with the virtual object VO specified based on the instruction information generated according to the user U1's motion. to display the tag TG corresponding to the virtual object VO. By visually recognizing the displayed tag TG on the AR glasses 20, the user U1 can recall the tag TG corresponding to the virtual object VO.
　また、表示制御部１１４は、表示装置としてのＡＲグラス２０に、仮想空間ＶＳを平面展開した２次元画像ＳＩを表示させる。更に、表示制御部１１４は、当該２次元画像ＳＩ内において、第１の仮想オブジェクトＶＯに関連付けて、第１の呼び名としてのタグＴＧを表示させる。 Also, the display control unit 114 causes the AR glasses 20 as a display device to display a two-dimensional image SI obtained by planarly developing the virtual space VS. Further, the display control unit 114 displays the tag TG as the first name in association with the first virtual object VO within the two-dimensional image SI.
　情報処理装置としての端末装置１０が上記の構成を用いることにより、ユーザＵ１は、仮想空間ＶＳにおいて、自身が呼び出したい仮想オブジェクトＶＯがどの位置にあるかを把握した上で、当該仮想オブジェクトＶＯに対応するタグＴＧを想起することが可能となる。 By using the above-described configuration of the terminal device 10 as an information processing device, the user U1 can grasp the position of the virtual object VO that the user wants to call in the virtual space VS, and then call the virtual object VO. It becomes possible to recall the corresponding tag TG.
　また、表示制御部１１４は、表示装置としてのＡＲグラス２０に、仮想空間ＶＳを縮小した３次元画像ＴＩを表示させる。更に、表示制御部１１４は、当該３次元画像ＴＩ内において、第１の仮想オブジェクトＶＯに関連付けて、第１の呼び名としてのタグＴＧを表示させる。 Also, the display control unit 114 causes the AR glasses 20 as a display device to display a three-dimensional image TI obtained by reducing the virtual space VS. Further, the display control unit 114 displays the tag TG as the first name in association with the first virtual object VO within the three-dimensional image TI.
　情報処理装置としての端末装置１０が上記の構成を用いることにより、ユーザＵ１は、仮想空間ＶＳにおいて、自身が呼び出したい仮想オブジェクトＶＯがどの位置にあるかを把握した上で、当該仮想オブジェクトＶＯに対応するタグＴＧを想起することが可能となる。 By using the above-described configuration of the terminal device 10 as an information processing device, the user U1 can grasp the position of the virtual object VO that the user wants to call in the virtual space VS, and then call the virtual object VO. It becomes possible to recall the corresponding tag TG.
　また、上記のユーザＵ１の動作は表示装置としてのＡＲグラス２０に対する目視である。上記の指示情報は、表示装置としてのＡＲグラス２０におけるユーザＵ１の視点を示す。情報処理装置としての端末装置１０は、判定部１１５を更に備える。判定部１１５は、表示装置としてのＡＲグラス２０に表示される第１の仮想オブジェクトＶＯ内に、ユーザＵ１の視点が所定時間以上位置するか否かを判定する。第１の仮想オブジェクトＶＯ内にユーザＵ１の視点が所定時間以上位置すると判定部１１５によって判定された場合に、表示制御部１１４は、表示装置としてのＡＲグラス２０に、当該第１の仮想オブジェクトＶＯに対応する第１の呼び名としてのタグＴＧを表示させる。 Also, the above operation of the user U1 is viewing the AR glasses 20 as a display device. The above instruction information indicates the viewpoint of the user U1 on the AR glasses 20 as a display device. The terminal device 10 as an information processing device further includes a determination unit 115 . The determination unit 115 determines whether or not the viewpoint of the user U1 is positioned within the first virtual object VO displayed on the AR glasses 20 as a display device for a predetermined time or longer. When the determination unit 115 determines that the viewpoint of the user U1 is positioned within the first virtual object VO for a predetermined time or longer, the display control unit 114 displays the first virtual object VO on the AR glasses 20 as a display device. to display the tag TG as the first nickname corresponding to .
　情報処理装置としての端末装置１０が上記の構成を用いることにより、ユーザＵ１は、表示装置としてのＡＲグラス２０に対する視線に係る動作以外の動作の必要なく、第１の仮想オブジェクトＶＯに対応する第１の呼び名としてのタグＴＧを想起することが可能となる。 By using the above-described configuration of the terminal device 10 as the information processing device, the user U1 does not need to perform any motion other than the motion related to the line of sight with respect to the AR glasses 20 as the display device. It becomes possible to recall the tag TG as a name of 1.
　また、表示制御部１１４は、ユーザＵ１が発話する音声の認識結果と第１の呼び名としてのタグＴＧとが一致した場合に、第１の仮想オブジェクトＶＯに関する表示内容を変更させる。 Also, the display control unit 114 changes the display content regarding the first virtual object VO when the recognition result of the voice uttered by the user U1 matches the tag TG as the first nickname.
　情報処理装置としての端末装置１０が上記の構成を用いることにより、ユーザＵ１は、一例として、音声の認識結果に一致したタグＴＧに対応する第１の仮想オブジェクトＶＯを、当該ユーザＵ１の視線の方向に移動させることが可能となる。あるいは、ユーザＵ１は、一例として、音声の認識結果に一致したタグＴＧに対応する第１の仮想オブジェクトＶＯがアプリケーションである場合に、当該アプリケーションを起動することが可能となる。 By using the above-described configuration of the terminal device 10 as an information processing device, the user U1 can, for example, display the first virtual object VO corresponding to the tag TG that matches the speech recognition result in the line of sight of the user U1. It is possible to move in the direction Alternatively, for example, if the first virtual object VO corresponding to the tag TG that matches the speech recognition result is an application, the user U1 can start the application.
　また以上の説明によれば、情報処理装置としての端末装置１０において、仮想オブジェクト特定部１１３－１は、ユーザＵ１が発話する音声であって、複数の仮想オブジェクトＶＯの少なくともいずれかの属性を表す音声の認識結果に基づいて、１以上の仮想オブジェクトＶＯを特定する。呼び名特定部１１３－２は、仮想オブジェクト特定部１１３－１によって特定された１以上の仮想オブジェクトＶＯの一部又は全部の仮想オブジェクトＶＯの各々に対応する、呼び名としてのタグＴＧを特定する。表示制御部１１４は、一部又は全部の仮想オブジェクトＶＯの各々に対応関連付けて、呼び名特定部１１３－２により特定された対応する呼び名としてのタグＴＧを、表示装置としてのＡＲグラス２０に表示させる。 Further, according to the above description, in the terminal device 10 as the information processing device, the virtual object specifying unit 113-1 is the voice uttered by the user U1, which represents at least one attribute of the plurality of virtual objects VO. One or more virtual objects VO are specified based on the speech recognition result. The nickname specifying unit 113-2 specifies a tag TG as a nickname corresponding to each of the one or more virtual objects VO specified by the virtual object specifying unit 113-1. The display control unit 114 causes the AR glasses 20 as the display device to display the tag TG as the corresponding nickname specified by the nickname specifying unit 113-2 in association with each of a part or all of the virtual objects VO. .
　情報処理装置としての端末装置１０が上記の構成を用いることにより、ユーザＵ１は、仮想空間ＶＳに配置される複数の仮想オブジェクトＶＯを、同一の属性に対応する１以上の仮想オブジェクトＶＯに絞り込む。絞り込まれた仮想オブジェクトＶＯに対応するタグＴＧの中に、ユーザＵ１が呼び出したい仮想オブジェクトＶＯに対応するタグＴＧが含まれる場合には、ユーザＵ１は、当該タグＴＧを想起することが容易になる。 By using the above configuration of the terminal device 10 as an information processing device, the user U1 narrows down a plurality of virtual objects VO arranged in the virtual space VS to one or more virtual objects VO corresponding to the same attribute. If the tags TG corresponding to the narrowed-down virtual objects VO include the tag TG corresponding to the virtual object VO that the user U1 wants to call, the user U1 can easily remember the tag TG. .
２：第２実施形態
　以下、図２０～図２９を参照することにより、本発明の第２実施形態に係る情報処理装置を含む情報処理システム１Ａの構成について説明する。なお、以下の説明では、説明の簡略化のため、第１実施形態と同一の構成要素に対しては、同一の符号を用いると共に、その機能の説明を省略することがある。また、以下の説明では、説明の簡略化のため、主として、第２実施形態が、第１実施形態に比較して相違する点について説明する。 2: Second Embodiment A configuration of an information processing system 1A including an information processing apparatus according to a second embodiment of the present invention will be described below with reference to FIGS. 20 to 29. FIG. In the following description, for simplification of description, the same symbols are used for the same components as in the first embodiment, and the description of their functions may be omitted. Also, in the following description, for the sake of simplification of description, mainly the differences between the second embodiment and the first embodiment will be described.
２．１：第１実施形態の構成
２．１．１：全体構成
　図２０は、本発明の第２実施形態に係る情報処理システム１Ａの全体構成を示す図である。情報処理システム１Ａは、第１実施形態に係る情報処理システム１に比較して、端末装置１０の代わりに端末装置１０Ａを備える点で異なる。 2.1: Configuration of First Embodiment 2.1.1: Overall Configuration FIG. 20 is a diagram showing the overall configuration of an information processing system 1A according to the second embodiment of the present invention. The information processing system 1A differs from the information processing system 1 according to the first embodiment in that it includes a terminal device 10A instead of the terminal device 10. FIG.
２．１．２：端末装置の構成
　図２１は、端末装置１０Ａの構成例を示すブロック図である。端末装置１０Ａは、第１実施形態に係る端末装置１０に比較して、処理装置１１の代わりに処理装置１１Ａを、記憶装置１２の代わりに記憶装置１２Ａを備える点で異なる。 2.1.2: Configuration of Terminal Device FIG. 21 is a block diagram showing a configuration example of the terminal device 10A. The terminal device 10A differs from the terminal device 10 according to the first embodiment in that it includes a processing device 11A instead of the processing device 11 and a storage device 12A instead of the storage device 12 .
　記憶装置１２Ａは、第１実施形態に係る記憶装置１２とは異なり、第２情報ＩＦ２を記憶することが必須ではない点、及び、学習モデルＬＭ１を記憶する点で異なる。 The storage device 12A differs from the storage device 12 according to the first embodiment in that it is not essential to store the second information IF2 and that it stores the learning model LM1.
　学習モデルＬＭ１は、後述の呼び名特定部１１３－２Ａが用いるための学習モデルである。具体的には、学習モデルＬＭ１は、第１の単語と第２の単語との類似度を算出するための学習モデルである。一例として、学習モデルＬＭ１は、単語の意味を数値ベクトル化し、第１の単語のベクトルの向きと、第２の単語のベクトルの向きとが、どの程度同じ方向を向いているかに基づいて、第１の単語と第２の単語の類似度を算出する。しかし、上記の類似度の算出方法は一例であって、これには限定されない。学習モデルＬＭ１は、第１の単語と第２の単語との類似度を算出することができれば、他の方法を用いてもよい。 The learning model LM1 is a learning model for use by the later-described calling name identification unit 113-2A. Specifically, the learning model LM1 is a learning model for calculating the degree of similarity between the first word and the second word. As an example, the learning model LM1 converts the meaning of a word into a numerical vector, and based on how much the direction of the vector of the first word and the direction of the vector of the second word are in the same direction, the Calculate the similarity between the first word and the second word. However, the similarity calculation method described above is merely an example, and the present invention is not limited to this. The learning model LM1 may use another method as long as it can calculate the degree of similarity between the first word and the second word.
　学習モデルＬＭ１は、学習フェーズにおいて、教師データを学習することによって生成される。学習モデルＬＭ１を生成するために用いられる教師データは、第１の単語と第２の単語のセットと、類似度を示す数値との組を複数有する。 The learning model LM1 is generated by learning teacher data in the learning phase. The teacher data used to generate the learning model LM1 has a plurality of pairs of sets of first words and second words and numerical values indicating degrees of similarity.
　また、学習モデルＬＭ１は、端末装置１０Ａの外部で生成される。とりわけ学習モデルＬＭ１は、サーバ３０において生成されることが好適である。この場合、端末装置１０Ａは、通信網ＮＥＴを介してサーバ３０から学習モデルＬＭ１を取得する。 Also, the learning model LM1 is generated outside the terminal device 10A. In particular, learning model LM1 is preferably generated in server 30 . In this case, the terminal device 10A acquires the learning model LM1 from the server 30 via the communication network NET.
　処理装置１１Ａは、記憶装置１２Ａから制御プログラムＰＲ４を読み出して実行することによって、取得部１１１、動作認識部１１２、特定部１１３Ａ、表示制御部１１４Ａ、音声認識部１１６、及び更新部１１７として機能する。なお、取得部１１１、動作認識部１１２、及び音声認識部１１６は、第１実施形態に係る処理装置１１の機能としての、取得部１１１、動作認識部１１２、及び音声認識部１１６と同一であるため、その説明を省略する。 The processing device 11A functions as an acquisition unit 111, an action recognition unit 112, a specification unit 113A, a display control unit 114A, a voice recognition unit 116, and an update unit 117 by reading and executing the control program PR4 from the storage device 12A. . Note that the acquisition unit 111, the action recognition unit 112, and the speech recognition unit 116 are the same as the acquisition unit 111, the action recognition unit 112, and the speech recognition unit 116 as functions of the processing device 11 according to the first embodiment. Therefore, its description is omitted.
　図２２は、特定部１１３Ａの構成を示す機能ブロック図である。特定部１１３Ａは、第１実施形態に係る特定部１１３に比較して、呼び名特定部１１３－２の代わりに呼び名特定部１１３－２Ａを備える点で異なる。 FIG. 22 is a functional block diagram showing the configuration of the identification unit 113A. The identification unit 113A differs from the identification unit 113 according to the first embodiment in that it includes a nickname identification unit 113-2A instead of the nickname identification unit 113-2.
　呼び名特定部１１３－２Ａは、第１の動作として、呼び名特定部１１３－２と同様の動作を実行する。具体的には、呼び名特定部１１３－２Ａは、仮想オブジェクト特定部１１３－１によって特定された、第１の仮想オブジェクトＶＯに対応するタグＴＧが、記憶装置１２に記憶される第１情報ＩＦ１に含まれる場合に、当該対応するタグＴＧを、第１の呼び名として特定する。 The nickname identification unit 113-2A performs the same operation as the nickname identification unit 113-2 as the first operation. Specifically, the call name specifying unit 113-2A adds the tag TG corresponding to the first virtual object VO specified by the virtual object specifying unit 113-1 to the first information IF1 stored in the storage device 12. When included, the corresponding tag TG is identified as the first nickname.
　また、呼び名特定部１１３－２Ａは、第２の動作として、仮想空間ＶＳに配置される複数の仮想オブジェクトＶＯの一部又は全部に対応する複数のタグＴＧを、第１の呼び名として特定する。より詳細には、呼び名特定部１１３－２Ａは、仮想空間ＶＳに配置されている複数の仮想オブジェクトＶＯのうち、第１情報ＩＦ１にタグＴＧが含まれている仮想オブジェクトＶＯの、当該タグＴＧを特定する。 In addition, as a second operation, the nickname identifying unit 113-2A identifies, as a first nickname, a plurality of tags TG corresponding to some or all of the plurality of virtual objects VO arranged in the virtual space VS. More specifically, the call name specifying unit 113-2A identifies the tag TG of the virtual object VO whose first information IF1 includes the tag TG among the plurality of virtual objects VO placed in the virtual space VS. Identify.
　更に、呼び名特定部１１３－２Ａは、第３の動作として、ユーザＵ１が発話した第１の音声の認識結果が、第１情報ＩＦ１に含まれる複数のタグＴＧの、いずれにも一致しない第２の呼び名である場合に、第１情報ＩＦ１に含まれる複数のタグＴＧのうち、第２の呼び名に最も類似するタグＴＧを、第１の呼び名として特定する。 Furthermore, as a third operation, the nickname identification unit 113-2A performs a second operation in which the recognition result of the first speech uttered by the user U1 does not match any of the plurality of tags TG included in the first information IF1. , the tag TG that is most similar to the second nickname among the plurality of tags TG included in the first information IF1 is specified as the first nickname.
　より詳細には、呼び名特定部１１３－２Ａは、ユーザＵ１が発話した第１の音声の認識結果としての文字列と、第１情報ＩＦ１に含まれる複数のタグＴＧのうち１つのタグＴＧとを、上記の学習モデルＬＭ１に入力する。学習モデルＬＭ１は、第１の音声の認識結果としての文字列と、上記の１つのタグＴＧとの類似度を出力する。呼び名特定部１１３－２Ａは、第１情報ＩＦ１に記載される全てのタグＴＧについて、同様の動作を実行し、第１の音声の認識結果としての文字列と、全てのタグＴＧとの類似度を取得する。更に、呼び名特定部１１３－２Ａは、全てのタグＴＧのうち、最も類似度の数値の高かったタグＴＧを特定する。 More specifically, the nickname identification unit 113-2A identifies the character string as the recognition result of the first voice uttered by the user U1 and one tag TG among the plurality of tags TG included in the first information IF1. , into the learning model LM1 described above. The learning model LM1 outputs the degree of similarity between the character string as the recognition result of the first speech and the one tag TG. The nickname specifying unit 113-2A performs the same operation for all the tags TG described in the first information IF1, and determines the degree of similarity between the character string as the recognition result of the first speech and all the tags TG. to get Furthermore, the nickname identifying unit 113-2A identifies the tag TG with the highest similarity value among all the tag TGs.
　図２１に説明を戻すと、表示制御部１１４Ａは、表示装置としてのＡＲグラス２０に、呼び名特定部１１３－２Ａによって特定されたタグＴＧを表示させる。 Returning to FIG. 21, the display control unit 114A causes the AR glasses 20 as a display device to display the tag TG specified by the name specifying unit 113-2A.
　図２３は、表示制御部１１４Ａの第１の動作例についての説明図である。図２３に示されるように、ユーザＵ１が、呼び出したい仮想オブジェクトＶＯに対応するタグＴＧとして“ＤＯＧ”を発話したものの、発話された第１の音声の認識結果である“ＤＯＧ”という文字列が、第１情報ＩＦ１に含まれていない第２の呼び名であるとする。この場合、呼び名特定部１１３－２Ａは、第１情報ＩＦ１に含まれている複数のタグＴＧのうち、“ＤＯＧ”という第２の呼び名に最も類似するタグＴＧとして、“ＨＯＲＳＥ”というタグＴＧを特定する。その上で、表示制御部１１４Ａは、仮想空間ＶＳにポップアップＰ２を表示させる。更に、当該表示制御部１１４Ａは、当該ポップアップＰ２内において、ユーザＵ１に対し、当該ユーザＵ１が本来発話しようとしていたタグＴＧが“ＨＯＲＳＥ”ではないか、確認するメッセージを表示させる。 FIG. 23 is an explanatory diagram of a first operation example of the display control unit 114A. As shown in FIG. 23, the user U1 uttered "DOG" as the tag TG corresponding to the virtual object VO to be called. , is a second nickname that is not included in the first information IF1. In this case, the nickname identification unit 113-2A selects the tag TG "HORSE" as the tag TG most similar to the second nickname "DOG" among the plurality of tags TG contained in the first information IF1. Identify. After that, the display control unit 114A displays the popup P2 in the virtual space VS. Furthermore, the display control unit 114A causes the user U1 to display a message in the popup P2 to confirm whether the tag TG that the user U1 was originally trying to utter is "HORSE".
　図２４は、表示制御部１１４Ａの第２の動作例についての説明図である。図２４に示されるように、ユーザＵ１が呼び出したい仮想オブジェクトＶＯに対応するタグＴＧとして、何らかの文字列を発話したものの、発話された第１の音声の認識結果である文字列が、第１情報ＩＦ１に含まれていない第２の呼び名であるとする。この場合、呼び名特定部１１３－２Ａは、第１情報ＩＦ１に基づいて、仮想空間ＶＳに配置されている複数の仮想オブジェクトＶＯの一部又は全部に対応する複数のタグＴＧを特定する。その上で、表示制御部１１４Ａは、仮想空間ＶＳにポップアップＰ３を表示させる。更に、表示制御部１１４Ａは、当該ポップアップＰ３内において、特定された複数のタグＴＧに対応する複数の仮想オブジェクトＶＯのアイコン、すなわち縮小表示と、当該特定された複数のタグＴＧとを関連付けて、一覧表示させる。 FIG. 24 is an explanatory diagram of a second operation example of the display control unit 114A. As shown in FIG. 24, the user U1 uttered some character string as the tag TG corresponding to the virtual object VO that the user U1 wants to call, but the character string that is the recognition result of the uttered first voice is not the first information. Assume that it is a second nickname that is not included in IF1. In this case, the call name identification unit 113-2A identifies a plurality of tags TG corresponding to some or all of the plurality of virtual objects VO placed in the virtual space VS based on the first information IF1. Then, the display control unit 114A causes the popup P3 to be displayed in the virtual space VS. Further, the display control unit 114A associates, in the popup P3, the icons of the multiple virtual objects VO corresponding to the specified multiple tags TG, that is, the reduced display, with the specified multiple tags TG, display the list.
　図２５は、表示制御部１１４Ａの第３の動作例についての説明図である。図２５に示されるように、ユーザＵ１が呼び出したい仮想オブジェクトＶＯに対応するタグＴＧとして、何らかの文字列を発話したものの、発話された第１の音声の認識結果である文字列が第１情報ＩＦ１に含まれていない第２の呼び名であるとする。この場合、呼び名特定部１１３－２Ａは、第１情報ＩＦ１に基づいて、仮想空間ＶＳに配置されている複数の仮想オブジェクトＶＯの一部又は全部に対応する複数のタグＴＧを特定する。即ち、呼び名特定部１１３－２Ａは、第１情報ＩＦ１に含まれる全てのタグＴＧを特定する。その上で、表示制御部１１４Ａは、仮想空間ＶＳにおいて、当該複数のタグＴＧに対応する複数の仮想オブジェクトＶＯの一部又は全部の近傍に、当該複数のタグＴＧを表示させる。ここで、複数の仮想オブジェクトＶＯの一部又は全部の「近傍」とは、具体的には、各々の仮想オブジェクトＶＯから所定距離以内の範囲のことである。 FIG. 25 is an explanatory diagram of a third operation example of the display control unit 114A. As shown in FIG. 25, although the user U1 uttered some character string as the tag TG corresponding to the virtual object VO that the user U1 wants to call, the character string that is the recognition result of the uttered first voice is the first information IF1. Suppose that it is a second nickname that is not included in . In this case, the call name identification unit 113-2A identifies a plurality of tags TG corresponding to some or all of the plurality of virtual objects VO placed in the virtual space VS based on the first information IF1. That is, the call name identification unit 113-2A identifies all tags TG included in the first information IF1. Then, the display control unit 114A displays the plurality of tags TG in the vicinity of some or all of the plurality of virtual objects VO corresponding to the plurality of tags TG in the virtual space VS. Here, the “neighborhood” of some or all of the plurality of virtual objects VO specifically means a range within a predetermined distance from each virtual object VO.
　表示制御部１１４Ａが、上記の第１の動作例に示される動作を実行した後、ユーザＵ１は、呼び名特定部１１３－２Ａによって特定されたタグＴＧを発話したとする。あるいは、表示制御部１１４Ａが、上記の第２の動作例又は第３の動作例に示される動作を実行した後、ユーザＵ１は、呼び名特定部１１３－２Ａによって特定された１又は複数のタグＴＧの中から、いずれかのタグＴＧを発話したとする。これら第１の動作例～第３の動作例に示される動作において、ユーザＵ１が、呼び出したい仮想オブジェクトＶＯに対するタグＴＧとして、何らかの文字列を発話したものの、当該文字列が第１情報ＩＦ１に含まれていなかった場合、当該発話した音声は第１の音声である。その後、ユーザＵ１が、呼び名特定部１１３－２Ａによって特定された１又は複数のタグＴＧの中から、いずれかのタグＴＧを発話した場合、当該発話した音声は第２の音声である。ユーザＵ１が発話した第２の音声の認識結果が、呼び名特定部１１３－２Ａによって特定されたタグＴＧに一致する場合、表示制御部１１４Ａは、一致したタグＴＧに対応する仮想オブジェクトＶＯに関する表示を変更させる。具体的には、表示制御部１１４Ａは、一致したタグＴＧに対応する仮想オブジェクトＶＯを、ユーザＵ１の視線の方向に移動させてもよい。あるいは、一致したタグＴＧに対応する仮想オブジェクトＶＯがアプリケーションである場合に、表示制御部１１４は、当該アプリケーションを起動するか否かを選択する画面を表示してもよい。 After the display control unit 114A has executed the operation shown in the first operation example, the user U1 utters the tag TG specified by the nickname specifying unit 113-2A. Alternatively, after display control unit 114A performs the operation shown in the second operation example or the third operation example, user U1 may select one or more tags TG specified by nickname specifying unit 113-2A. Suppose that one of the tags TG is uttered. In the operations shown in the first to third operation examples, the user U1 uttered some character string as the tag TG for the virtual object VO to be called, but the character string was not included in the first information IF1. If not, the uttered voice is the first voice. Thereafter, when user U1 utters any tag TG from among the one or more tag TGs specified by the nickname specifying unit 113-2A, the uttered voice is the second voice. When the recognition result of the second voice uttered by user U1 matches the tag TG specified by the nickname specifying unit 113-2A, the display control unit 114A displays the virtual object VO corresponding to the matching tag TG. change. Specifically, the display control unit 114A may move the virtual object VO corresponding to the matching tag TG in the line-of-sight direction of the user U1. Alternatively, if the virtual object VO corresponding to the matched tag TG is an application, the display control unit 114 may display a screen for selecting whether to activate the application.
　図２１に説明を戻すと、更新部１１７は、上記のように、ユーザＵ１が発話した第１の音声の認識結果が第１情報ＩＦ１に含まれず、第２の音声の認識結果が第１情報ＩＦ１に含まれる特定のタグＴＧに一致した回数が所定回数に達した場合、当該特定のタグＴＧに対応する仮想オブジェクトＶＯを、第１の音声の認識結果である第２の呼び名としてのタグＴＧに対応付ける。より詳細には、上記の第１の動作例～第３の動作例の説明中で述べたように、ユーザＵ１が、呼び出したい仮想オブジェクトＶＯに対するタグＴＧとして、何らかの文字列を発話したものの、当該発話による第１の音声の認識結果が第１情報ＩＦ１に含まれていなかったとする。次に、ユーザＵ１が、呼び名特定部１１３－２Ａによって特定された１又は複数のタグＴＧの中から、いずれかのタグＴＧとしての文字列を発話し、当該発話による第２の音声の認識結果が第１情報ＩＦ１に含まれていたとする。このように、第１の音声の認識結果が第１情報ＩＦ１に含まれなかった後に、第２の音声の認識結果が第１情報ＩＦ１に含まれていたことが、所定回数に達したとする。この場合、更新部１１７は、第１の音声の認識結果としての文字列を、新たなタグＴＧとした上で、第２の音声の認識結果としてのタグＴＧに対応する仮想オブジェクトＶＯを、当該新たなタグＴＧに対応付ける。 Returning to FIG. 21, as described above, the updating unit 117 determines that the recognition result of the first voice uttered by the user U1 is not included in the first information IF1, and the recognition result of the second voice is the first information IF1. When the number of matches with a specific tag TG included in IF1 reaches a predetermined number of times, the virtual object VO corresponding to the specific tag TG is treated as a tag TG as a second name, which is the recognition result of the first voice. correspond to More specifically, as described in the explanations of the first to third operation examples above, although the user U1 uttered some character string as the tag TG for the virtual object VO to be called, Suppose that the first information IF1 does not include the recognition result of the first speech by utterance. Next, the user U1 utters a character string as one of the tag TG from among the one or a plurality of tags TG specified by the nickname specifying unit 113-2A, and the recognition result of the second voice resulting from the utterance is obtained. is included in the first information IF1. Assume that the first information IF1 includes the second speech recognition result after the first speech recognition result has not been included in the first information IF1 for a predetermined number of times. . In this case, the updating unit 117 sets the character string as the first speech recognition result to a new tag TG, and then replaces the virtual object VO corresponding to the tag TG as the second speech recognition result with the new tag TG. Associate with a new tag TG.
　ここで、記憶装置１２Ａは、当該仮想オブジェクトＶＯと、第１の音声の認識結果と、当該第１の音声が発話された後、第２の音声の認識結果が第１情報ＩＦ１に含まれていた回数とを表形式で記憶しておく。処理装置１１Ａが当該表を参照することにより、更新部１１７は、第１の音声の認識結果が第１情報ＩＦ１に含まれておらず、第２の音声の認識結果が第１情報ＩＦ１に含まれていた回数が所定回数に達したか否かを判定する。その上で、当該回数が所定回数に達した場合に、更新部１１７は、第１の音声の認識結果としての文字列を、新たなタグＴＧとした上で、第２の音声の認識結果としてのタグＴＧに対応する仮想オブジェクトＶＯを、当該新たなタグＴＧに対応付ける。 Here, the storage device 12A stores the virtual object VO, the recognition result of the first voice, and the recognition result of the second voice after the first voice is uttered in the first information IF1. and the number of times the test was performed are stored in a tabular form. By referencing the table by the processing device 11A, the update unit 117 determines that the first information IF1 does not include the first speech recognition result and the second speech recognition result is included in the first information IF1. It is determined whether or not the number of times of being held has reached a predetermined number of times. After that, when the number of times reaches a predetermined number of times, the update unit 117 sets the character string as the first speech recognition result to a new tag TG, and then sets the character string as the second speech recognition result. is associated with the new tag TG.
　図２６Ａ及び図２６Ｂは、更新部１１７の機能を示す説明図である。図２６Ａに示されるように、ＩＤ＝１の仮想オブジェクトＶＯに対応するタグＴＧが“＃ＷＥＢ”であり、ＩＤ＝８の仮想オブジェクトＶＯに対応するタグＴＧが“＃ＨＯＲＳＥ”であるとする。また、図２３～図２５を参照することにより説明した第１の動作例～第３の動作例において、第１の音声の認識結果が、第２の呼び名としての“ＤＯＧ”であり、第２の音声の認識結果が、第１情報ＩＦ１に含まれる“ＨＯＲＳＥ”であったとする。この場合、更新部１１７は、図２６Ｂに示されるように、“＃ＨＯＲＳＥ”というタグＴＧに対応するＩＤ＝８の仮想オブジェクトＶＯを、“＃ＤＯＧ”というタグＴＧに対応付ける。すなわち、ＩＤ＝８の仮想オブジェクトＶＯに対応するタグＴＧは、“＃ＨＯＲＳＥ”と“＃ＤＯＧ”の２つのタグＴＧとなる。 26A and 26B are explanatory diagrams showing the function of the updating unit 117. FIG. As shown in FIG. 26A, it is assumed that the tag TG corresponding to the virtual object VO with ID=1 is "#WEB" and the tag TG corresponding to the virtual object VO with ID=8 is "#HORSE". Also, in the first to third operation examples described with reference to FIGS. is "HORSE" included in the first information IF1. In this case, the update unit 117 associates the virtual object VO with ID=8 corresponding to the tag TG "#HORSE" with the tag TG "#DOG", as shown in FIG. 26B. That is, the tags TG corresponding to the virtual object VO with ID=8 are two tags TG, "#HORSE" and "#DOG".
２．２：第２実施形態の動作
　図２７～図２９は、第２実施形態に係る情報処理システム１Ａの動作を示すフローチャートである。以下、図２７～図２９を参照することにより、情報処理システム１Ａの動作について説明する。 2.2: Operation of Second Embodiment FIGS. 27 to 29 are flowcharts showing the operation of the information processing system 1A according to the second embodiment. The operation of the information processing system 1A will be described below with reference to FIGS. 27 to 29. FIG.
２．２．１：第１の動作
　図２７は、情報処理システム１Ａの第１の動作について説明するフローチャートである。 2.2.1: First Operation FIG. 27 is a flowchart describing the first operation of the information processing system 1A.
　ステップＳ４１において、処理装置１１Ａは、音声認識部１１６として機能することにより、ユーザＵ１が発話した音声を認識する。より詳細には、ＡＲグラス２０の処理装置２１は、取得部２１１として機能することにより、収音装置２４から、ユーザＵ１の音声を示す音声データを取得する。また、ＡＲグラス２０の処理装置２１は、取得部２１１として機能することにより、取得した音声データを、通信装置２８に出力する。通信装置２８は、処理装置２１から取得した音声データを、端末装置１０Ａに出力する。端末装置１０Ａの処理装置１１Ａは、取得部１１１として機能することにより、通信装置１３を用いて、ＡＲグラス２０から、音声データを取得する。また、端末装置１０Ａの処理装置１１Ａは、音声認識部１１６として機能することにより、音声データを音声認識する。音声認識結果としての文字列は、第１実施形態における指示情報に対応する。 In step S41, the processing device 11A recognizes the voice uttered by the user U1 by functioning as the voice recognition unit 116. More specifically, the processing device 21 of the AR glasses 20 acquires voice data representing the voice of the user U1 from the sound pickup device 24 by functioning as the acquisition unit 211 . Also, the processing device 21 of the AR glasses 20 outputs the acquired audio data to the communication device 28 by functioning as the acquisition unit 211 . The communication device 28 outputs the audio data acquired from the processing device 21 to the terminal device 10A. The processing device 11A of the terminal device 10A acquires audio data from the AR glasses 20 using the communication device 13 by functioning as the acquisition unit 111 . Further, the processing device 11A of the terminal device 10A functions as a voice recognition unit 116 to perform voice recognition on voice data. A character string as a speech recognition result corresponds to the instruction information in the first embodiment.
　ステップＳ４２において、処理装置１１Ａは、呼び名特定部１１３－２Ａとして機能することにより、ユーザＵ１が発話した音声の認識結果が、第１情報ＩＦ１に含まれる、複数の仮想オブジェクトＶＯに対応する複数のタグＴＧのいずれかに対応するか否かを判定する。判定結果が真である場合、すなわちステップＳ４２の判定結果が“ＹＥＳ”である場合には、処理装置１１Ａは、ステップＳ４５の処理を実行する。なお、この場合、ユーザＵ１が発話した音声の認識結果は、上記の第１の呼び名の一例である。判定結果が偽である場合、すなわちステップＳ４２の判定結果が“ＮＯ”である場合には、処理装置１１Ａは、ステップＳ４３の処理を実行する。なお、この場合、ユーザＵ１が発話した音声の認識結果は、上記の第２の呼び名の一例である。 In step S42, the processing device 11A functions as the nickname identification unit 113-2A, so that the recognition result of the voice uttered by the user U1 is included in the first information IF1 and corresponds to the plurality of virtual objects VO. It is determined whether or not it corresponds to any of the tags TG. When the determination result is true, that is, when the determination result of step S42 is "YES", the processing device 11A executes the process of step S45. In this case, the recognition result of the voice uttered by the user U1 is an example of the above first name. When the determination result is false, that is, when the determination result of step S42 is "NO", the processing device 11A executes the process of step S43. In this case, the recognition result of the voice uttered by the user U1 is an example of the above second name.
　ステップＳ４３において、処理装置１１Ａは、呼び名特定部１１３－２Ａとして機能することにより、第１情報ＩＦ１に含まれている複数のタグＴＧ、すなわち複数の第１の呼び名のうち、ユーザＵ１の音声の認識結果としての第２の呼び名に最も類似するタグＴＧを、第１の呼び名として特定する。 In step S43, the processing device 11A functions as the nickname specifying unit 113-2A, thereby identifying the voice of the user U1 among the plurality of tags TG included in the first information IF1, that is, the plurality of first nicknames. A tag TG that is most similar to the second nickname as a recognition result is specified as the first nickname.
　ステップＳ４４において、処理装置１１Ａは、表示制御部１１４Ａとして機能することにより、ステップＳ４３において特定された最も類似するタグＴＧを、仮想空間ＶＳに表示させる。例えば、処理装置１１Ａは、表示制御部１１４Ａとして機能することにより、仮想空間ＶＳにポップアップＰ２を表示させる。更に、処理装置１１Ａは、表示制御部１１４Ａとして機能することにより、当該ポップアップＰ２内において、ユーザＵ１に対し、当該ユーザＵ１が本来発話しようとしていたタグＴＧを確認するメッセージを表示させる。その後、処理装置１１Ａは、ステップＳ４１の処理を実行する。 In step S44, the processing device 11A functions as the display control unit 114A to display the most similar tag TG identified in step S43 in the virtual space VS. For example, the processing device 11A displays the pop-up P2 in the virtual space VS by functioning as the display control unit 114A. Further, the processing device 11A functions as the display control unit 114A to display a message for the user U1 to confirm the tag TG that the user U1 was originally trying to utter in the popup P2. After that, the processing device 11A executes the process of step S41.
　ステップＳ４５において、処理装置１１Ａは、表示制御部１１４Ａとして機能することにより、ステップＳ４２における認識結果である、第１の呼び名としてのタグＴＧに対応する仮想オブジェクトＶＯの表示を変更させる。 In step S45, the processing device 11A functions as the display control unit 114A to change the display of the virtual object VO corresponding to the tag TG as the first nickname, which is the recognition result in step S42.
　なお、ユーザＵ１が発話した第１の音声の認識結果が第１情報ＩＦ１に含まれず、第２の音声の認識結果が第１情報ＩＦ１に含まれる特定のタグＴＧに一致した回数が所定回数に達した場合、ステップＳ４５の後に、更新部１１７は、当該特定のタグＴＧに対応する仮想オブジェクトＶＯを、第１の音声の認識結果である第２の呼び名としてのタグＴＧに対応付けてもよい。 Note that the number of times the recognition result of the first voice uttered by the user U1 is not included in the first information IF1 and the recognition result of the second voice matches the specific tag TG included in the first information IF1 reaches the predetermined number of times. If it has reached, after step S45, the update unit 117 may associate the virtual object VO corresponding to the specific tag TG with the tag TG as the second nickname that is the recognition result of the first voice. .
２．２．２：第２の動作
　図２８は、情報処理システム１Ａの第２の動作について説明するフローチャートである。 2.2.2: Second Operation FIG. 28 is a flowchart illustrating the second operation of the information processing system 1A.
　ステップＳ５１において、処理装置１１Ａは、音声認識部１１６として機能することにより、ユーザＵ１が発話した音声を認識する。なお、動作の詳細については、第１の動作におけるステップＳ４１と同一であるため、その説明を省略する。 In step S51, the processing device 11A recognizes the voice uttered by the user U1 by functioning as the voice recognition unit 116. Note that the details of the operation are the same as in step S41 in the first operation, so description thereof will be omitted.
　ステップＳ５２において、処理装置１１Ａは、呼び名特定部１１３－２Ａとして機能することにより、ユーザＵ１が発話した音声の認識結果が、第１情報ＩＦ１に含まれる複数のタグＴＧのいずれかに対応するか否かを判定する。判定結果が真である場合、すなわちステップＳ５２の判定結果が“ＹＥＳ”である場合には、処理装置１１Ａは、ステップＳ５５の処理を実行する。なお、この場合、ユーザＵ１が発話した音声の認識結果は、上記の第１の呼び名の一例である。判定結果が偽である場合、すなわちステップＳ５２の判定結果が“ＮＯ”である場合には、処理装置１１Ａは、ステップＳ５３の処理を実行する。なお、この場合、ユーザＵ１が発話した音声の認識結果は、上記の第２の呼び名の一例である。 In step S52, the processing device 11A, by functioning as the nickname identification unit 113-2A, determines whether the recognition result of the voice uttered by the user U1 corresponds to any of the plurality of tags TG included in the first information IF1. determine whether or not When the determination result is true, that is, when the determination result of step S52 is "YES", the processing device 11A executes the process of step S55. In this case, the recognition result of the voice uttered by the user U1 is an example of the above first name. When the determination result is false, that is, when the determination result of step S52 is "NO", the processing device 11A executes the process of step S53. In this case, the recognition result of the voice uttered by the user U1 is an example of the above second name.
　ステップＳ５３において、処理装置１１Ａは、呼び名特定部１１３－２Ａとして機能することにより、第１情報ＩＦ１に基づいて、仮想空間ＶＳに配置される複数の仮想オブジェクトＶＯの一部又は全部に対応する複数のタグＴＧを特定する。 In step S53, the processing device 11A functions as the call name specifying unit 113-2A, and based on the first information IF1, the plurality of virtual objects VO corresponding to some or all of the plurality of virtual objects VO arranged in the virtual space VS. identifies the tag TG of
　ステップＳ５４において、処理装置１１Ａは、表示制御部１１４Ａとして機能することにより、仮想空間ＶＳにポップアップＰ３を表示させる。更に、処理装置１１Ａは、表示制御部１１４Ａとして機能することにより、当該ポップアップＰ３内において、ステップＳ５３で特定された複数のタグＴＧに対応する複数の仮想オブジェクトＶＯのアイコン、すなわち縮小表示と、当該特定された複数のタグＴＧとを関連付けて、一覧表示させる。その後、処理装置１１Ａは、ステップＳ５１の処理を実行する。 In step S54, the processing device 11A functions as the display control unit 114A to display the popup P3 in the virtual space VS. Furthermore, by functioning as the display control unit 114A, the processing device 11A, in the popup P3, displays the icons of the multiple virtual objects VO corresponding to the multiple tags TG identified in step S53, that is, reduced display, and A list is displayed in association with a plurality of specified tags TG. After that, the processing device 11A executes the process of step S51.
　ステップＳ５５において、処理装置１１Ａは、表示制御部１１４Ａとして機能することにより、ステップＳ５２における認識結果である、第１の呼び名としてのタグＴＧに対応する仮想オブジェクトＶＯの表示を変更させる。 In step S55, the processing device 11A functions as the display control unit 114A to change the display of the virtual object VO corresponding to the tag TG as the first nickname, which is the recognition result in step S52.
　なお、図２７に示される第１の動作と同様に、ユーザＵ１が発話した第１の音声の認識結果が第１情報ＩＦ１に含まれず、第２の音声の認識結果が第１情報ＩＦ１に含まれる特定のタグＴＧに一致した回数が所定回数に達した場合、ステップＳ５５の後に、更新部１１７は、当該特定のタグＴＧに対応する仮想オブジェクトＶＯを、第１の音声の認識結果である第２の呼び名としてのタグＴＧに対応付けてもよい。 As in the first operation shown in FIG. 27, the first information IF1 does not include the recognition result of the first voice uttered by the user U1, and the first information IF1 includes the recognition result of the second voice. If the number of matches with the specified tag TG reaches a predetermined number, after step S55, the update unit 117 updates the virtual object VO corresponding to the specified tag TG to the first voice recognition result, which is the first voice recognition result. 2 may be associated with a tag TG as a nickname.
２．２．３：第３の動作
　図２９は、情報処理システム１Ａの第３の動作について説明するフローチャートである。 2.2.3: Third Operation FIG. 29 is a flowchart illustrating the third operation of the information processing system 1A.
　ステップＳ６１において、処理装置１１Ａは、音声認識部１１６として機能することにより、ユーザＵ１が発話した音声を認識する。なお、動作の詳細については、第１の動作におけるステップＳ４１、及び第２の動作におけるステップＳ５１と同一であるため、その説明を省略する。 In step S61, the processing device 11A recognizes the voice uttered by the user U1 by functioning as the voice recognition unit 116. The details of the operation are the same as those in step S41 in the first operation and step S51 in the second operation, and thus description thereof is omitted.
　ステップＳ６２において、処理装置１１Ａは、呼び名特定部１１３－２Ａとして機能することにより、ユーザＵ１が発話した音声の認識結果が、第１情報ＩＦ１に含まれる複数のタグＴＧのいずれかに対応するか否かを判定する。判定結果が真である場合、すなわちステップＳ６２の判定結果が“ＹＥＳ”である場合には、処理装置１１Ａは、ステップＳ６５の処理を実行する。なお、この場合、ユーザＵ１が発話した音声の認識結果は、上記の第１の呼び名の一例である。判定結果が偽である場合、すなわちステップＳ６２の判定結果が“ＮＯ”である場合には、処理装置１１Ａは、ステップＳ６３の処理を実行する。なお、この場合、ユーザＵ１が発話した音声の認識結果は、上記の第２の呼び名の一例である。 In step S62, the processing device 11A, by functioning as the nickname identification unit 113-2A, determines whether the recognition result of the voice uttered by the user U1 corresponds to any of the plurality of tags TG included in the first information IF1. determine whether or not When the determination result is true, that is, when the determination result of step S62 is "YES", the processing device 11A executes the process of step S65. In this case, the recognition result of the voice uttered by the user U1 is an example of the above first name. When the determination result is false, that is, when the determination result of step S62 is "NO", the processing device 11A executes the process of step S63. In this case, the recognition result of the voice uttered by the user U1 is an example of the above second name.
　ステップＳ６３において、処理装置１１Ａは、呼び名特定部１１３－２Ａとして機能することにより、第１情報ＩＦ１に基づいて、仮想空間ＶＳに配置される複数の仮想オブジェクトＶＯの一部又は全部に対応する複数のタグＴＧを特定する。 In step S63, the processing device 11A functions as the call name specifying unit 113-2A, and based on the first information IF1, the plurality of virtual objects VO corresponding to some or all of the plurality of virtual objects VO arranged in the virtual space VS. identifies the tag TG of
　ステップＳ６４において、処理装置１１Ａは、表示制御部１１４Ａとして機能することにより、ステップＳ６３で特定された複数のタグＴＧに対応する複数の仮想オブジェクトＶＯの一部又は全部の近傍に、当該複数のタグＴＧを表示させる。ここで、複数の仮想オブジェクトＶＯの一部又は全部の「近傍」とは、具体的には、各々の仮想オブジェクトＶＯから所定距離以内の範囲のことである。その後、処理装置１１Ａは、ステップＳ６１の処理を実行する。 In step S64, the processing device 11A, by functioning as the display control unit 114A, displays the plurality of tags TG in the vicinity of some or all of the plurality of virtual objects VO corresponding to the plurality of tags TG identified in step S63. Display TG. Here, the “neighborhood” of some or all of the plurality of virtual objects VO specifically means a range within a predetermined distance from each virtual object VO. After that, the processing device 11A executes the process of step S61.
　ステップＳ６５において、処理装置１１Ａは、表示制御部１１４Ａとして機能することにより、ステップＳ６２における認識結果である、第１の呼び名としてのタグＴＧに対応する仮想オブジェクトＶＯの表示を変更させる。 In step S65, the processing device 11A functions as the display control unit 114A to change the display of the virtual object VO corresponding to the tag TG as the first nickname, which is the recognition result in step S62.
　なお、図２７に示される第１の動作、及び図２８に示される第２の動作と同様に、ユーザＵ１が発話した第１の音声の認識結果が第１情報ＩＦ１に含まれず、第２の音声の認識結果が第１情報ＩＦ１に含まれる特定のタグＴＧに一致した回数が所定回数に達した場合、ステップＳ６５の後に、更新部１１７は、当該特定のタグＴＧに対応する仮想オブジェクトＶＯを、第１の音声の認識結果である第２の呼び名としてのタグＴＧに対応付けてもよい。 As in the first operation shown in FIG. 27 and the second operation shown in FIG. 28, the first information IF1 does not include the recognition result of the first voice uttered by the user U1, When the number of times that the speech recognition result matches the specific tag TG included in the first information IF1 reaches a predetermined number of times, the updating unit 117 updates the virtual object VO corresponding to the specific tag TG after step S65. , may be associated with the tag TG as the second nickname that is the recognition result of the first voice.
２．３：第２実施形態が奏する効果
　以上の説明によれば、情報処理装置としての端末装置１０Ａは、表示制御部１１４Ａと、呼び名特定部１１３－２Ａを備える。表示制御部１１４Ａは、ユーザＵ１の頭部に装着される表示装置としてのＡＲグラス２０に、仮想空間ＶＳに配置される複数の仮想オブジェクトＶＯを表示させる。呼び名特定部１１３－２Ａは、ユーザＵ１が発話した第１の音声の認識結果が、複数の仮想オブジェクトＶＯに対応する複数の第１の呼び名のいずれにも一致しない第２の呼び名である場合に、複数の第１の呼び名のうち、第２の呼び名に最も類似する第１の呼び名を特定する。表示制御部１１４Ａは、呼び名特定部１１３－２Ａが特定した第１の呼び名を表示装置としてのＡＲグラス２０に表示させる。 2.3: Effects of the Second Embodiment According to the above description, the terminal device 10A as an information processing device includes the display control section 114A and the nickname specifying section 113-2A. The display control unit 114A causes the AR glasses 20 as a display device worn on the head of the user U1 to display a plurality of virtual objects VO arranged in the virtual space VS. If the recognition result of the first voice uttered by the user U1 is a second nickname that does not match any of the plurality of first nicknames corresponding to the plurality of virtual objects VO, the nickname specifying unit 113-2A , to identify a first nickname that is most similar to the second nickname among the plurality of first nicknames. The display control unit 114A causes the AR glasses 20 as a display device to display the first nickname specified by the nickname specifying unit 113-2A.
　情報処理装置としての端末装置１０Ａが上記の構成を用いることにより、ユーザＵ１の発話の音声認識結果が、第１情報ＩＦ１に含まれるタグＴＧには対応しない場合でも、第１情報ＩＦ１に含まれるタグＴＧを想起することが可能となる。とりわけ、ユーザＵ１は、自身が発話した音声の認識結果に最も類似する１つのタグＴＧを想起することが可能となる。 By using the above-described configuration of the terminal device 10A as an information processing device, even if the speech recognition result of the user U1's utterance does not correspond to the tag TG included in the first information IF1, it is included in the first information IF1. It becomes possible to recall the tag TG. In particular, user U1 can recall one tag TG that is most similar to the recognition result of the voice uttered by him/herself.
　また、呼び名特定部１１３－２Ａが特定した第１の呼び名が、表示装置としてのＡＲグラス２０に表示された後、ユーザＵ１が発話した第２の音声の認識結果が、特定した第１の呼び名に一致する場合、表示制御部１１４Ａは、複数の仮想オブジェクトＶＯのうち特定した第１の呼び名に対応する仮想オブジェクトＶＯに関する表示を変更させる。また、情報処理装置としての端末装置１０Ａは、第１の音声の認識結果が第２の呼び名となる回数が所定回数に達した場合、仮想オブジェクトＶＯを第２の呼び名に対応付ける更新部１１７を更に備える。 Further, after the first nickname specified by the nickname specifying unit 113-2A is displayed on the AR glasses 20 as the display device, the recognition result of the second voice uttered by the user U1 is the specified first nickname. , the display control unit 114A changes the display of the virtual object VO corresponding to the specified first nickname among the plurality of virtual objects VO. Further, the terminal device 10A as an information processing device further includes an updating unit 117 that associates the virtual object VO with the second nickname when the number of times the recognition result of the first voice becomes the second nickname reaches a predetermined number of times. Prepare.
　情報処理装置としての端末装置１０Ａは、上記の構成を用いることにより、ユーザＵ１が、ある仮想オブジェクトＶＯに対して、第１情報ＩＦ１には含まれない第２の呼び名を用いて呼び出そうとする回数が所定回数に達した場合、当該第２の呼び名を当該仮想オブジェクトＶＯに対応付けることが可能となる。 By using the above configuration, the terminal device 10A as an information processing device allows the user U1 to call a certain virtual object VO using a second name not included in the first information IF1. When the number of times of doing so reaches a predetermined number of times, it becomes possible to associate the second nickname with the virtual object VO.
　また以上の説明によれば、情報処理装置としての端末装置１０Ａは、表示制御部１１４Ａと、呼び名特定部１１３－２Ａを備える。表示制御部１１４Ａは、ユーザＵ１の頭部に装着される表示装置としてのＡＲグラス２０に、仮想空間ＶＳに配置される複数の仮想オブジェクトＶＯを表示させる。呼び名特定部１１３－２Ａは、複数の仮想オブジェクトＶＯの一部又は全部に対応する複数の第１の呼び名を特定する。表示制御部１１４Ａは、ユーザＵ１が発話する音声の認識結果が、複数の第１の呼び名のいずれにも一致しない第２の呼び名である場合に、呼び名特定部１１３－２Ａによって特定された複数の第１の呼び名の各々を、複数の仮想オブジェクトＶＯの一部又は全部のうち対応する仮想オブジェクトＶＯに関連付けて表示させる。 According to the above description, the terminal device 10A as an information processing device includes a display control section 114A and a call name specifying section 113-2A. The display control unit 114A causes the AR glasses 20 as a display device worn on the head of the user U1 to display a plurality of virtual objects VO arranged in the virtual space VS. The nickname identification unit 113-2A identifies a plurality of first nicknames corresponding to some or all of the plurality of virtual objects VO. When the recognition result of the voice uttered by user U1 is the second nickname that does not match any of the plurality of first nicknames, display control unit 114A controls the plurality of names specified by nickname specifying unit 113-2A. Each of the first nicknames is displayed in association with the corresponding virtual object VO out of some or all of the plurality of virtual objects VO.
　情報処理装置としての端末装置１０Ａが上記の構成を用いることにより、ユーザＵ１の発話の音声認識結果が、第１情報ＩＦ１に含まれるタグＴＧには対応しない場合でも、ユーザＵ１は、第１情報ＩＦ１に含まれるタグＴＧを想起することが可能となる。とりわけ、ユーザＵ１は、第１情報ＩＦ１に含まれるタグＴＧの全てを、仮想空間ＶＳ内において視認することが可能となる。 By using the above configuration of the terminal device 10A as an information processing device, even if the speech recognition result of the user U1's utterance does not correspond to the tag TG included in the first information IF1, the user U1 can obtain the first information It becomes possible to recall the tag TG contained in IF1. In particular, the user U1 can visually recognize all the tags TG included in the first information IF1 within the virtual space VS.
　また、呼び名特定部１１３－２Ａが特定した複数の第１の呼び名が、表示装置としてのＡＲグラス２０に表示された後、ユーザＵ１が発話した第２の音声の認識結果が、特定した複数の第１の呼び名のいずれかに一致する場合、表示制御部１１４Ａは、複数の仮想オブジェクトＶＯのうち一致した第１の呼び名に対応する仮想オブジェクトＶＯに関する表示を変更させる。また、情報処理装置としての端末装置１０Ａは、第１の音声の認識結果が第２の呼び名となる回数が所定回数に達した場合、仮想オブジェクトＶＯを第２の呼び名に対応付ける更新部１１７を更に備える。 Further, after the plurality of first nicknames specified by the nickname specifying unit 113-2A are displayed on the AR glasses 20 as the display device, the recognition result of the second voice uttered by the user U1 is the specified plurality of names. If the virtual object VO matches any of the first nicknames, the display control unit 114A changes the display of the virtual object VO corresponding to the matching first nickname among the plurality of virtual objects VO. Further, the terminal device 10A as an information processing device further includes an updating unit 117 that associates the virtual object VO with the second nickname when the number of times the recognition result of the first voice becomes the second nickname reaches a predetermined number of times. Prepare.
　情報処理装置としての端末装置１０Ａは、上記の構成を用いることにより、ユーザＵ１が、ある仮想オブジェクトＶＯに対して、第１情報ＩＦ１には含まれない第２の呼び名を用いて呼び出そうとする回数が所定回数に達した場合、当該第２の呼び名を当該仮想オブジェクトＶＯに対応付けることが可能となる。 By using the above configuration, the terminal device 10A as an information processing device allows the user U1 to call a certain virtual object VO using a second name not included in the first information IF1. When the number of times of doing so reaches a predetermined number of times, it becomes possible to associate the second nickname with the virtual object VO.
３：変形例
　本開示は、以上に例示した実施形態に限定されない。具体的な変形の態様を以下に例示する。以下の例示から任意に選択された２以上の態様を併合してもよい。 3: Modifications The present disclosure is not limited to the embodiments illustrated above. Specific modification modes are exemplified below. Two or more aspects arbitrarily selected from the following examples may be combined.
３．１：変形例１
　第１実施形態に係る端末装置１０は、処理装置１１の機能として、音声認識部１１６を備える。同様に、第２実施形態に係る端末装置１０Ａは、処理装置１１Ａの機能として、音声認識部１１６を備える。しかし、端末装置１０及び１０Ａは、音声認識部１１６を備えなくてもよい。具体的には、音声認識部１１６は、端末装置１０及び１０Ａの外部装置であって、当該端末装置１０及び１０Ａと通信可能に接続される形態であってもよい。この場合、音声認識部１１６に対応する音声認識装置は、クラウド上に存在し、通信網ＮＥＴを介して、端末装置１０及び１０Ａと通信可能に接続されてもよい。 3.1: Modification 1
The terminal device 10 according to the first embodiment includes a speech recognition unit 116 as a function of the processing device 11 . Similarly, the terminal device 10A according to the second embodiment includes a speech recognition unit 116 as a function of the processing device 11A. However, the terminal devices 10 and 10A do not have to include the speech recognition section 116 . Specifically, the voice recognition unit 116 may be an external device of the terminal devices 10 and 10A, and may be communicably connected to the terminal devices 10 and 10A. In this case, the speech recognition device corresponding to the speech recognition unit 116 may exist on the cloud and be communicably connected to the terminal devices 10 and 10A via the communication network NET.
３．２：変形例２
　第１実施形態に係る端末装置１０は、処理装置１１の機能として、取得部１１１を備える。当該取得部１１１は、記憶装置１２から第１情報ＩＦ１及び第２情報ＩＦ２を取得する。同様に、第２実施形態に係る端末装置１０Ａは、処理装置１１Ａの機能として、取得部１１１を備える。当該取得部１１１は、記憶装置１２Ａから第１情報ＩＦ１を取得する。しかし、取得部１１１の第１情報ＩＦ１及び第２情報ＩＦ２の取得元は、記憶装置１２又は１２Ａでなくてもよい。具体的には、取得部１１１は、サーバ３０から直接第１情報ＩＦ１及び第２情報ＩＦ２を取得してもよい。 3.2: Modification 2
The terminal device 10 according to the first embodiment includes an acquisition unit 111 as a function of the processing device 11 . The acquisition unit 111 acquires the first information IF1 and the second information IF2 from the storage device 12 . Similarly, the terminal device 10A according to the second embodiment includes an acquisition unit 111 as a function of the processing device 11A. The acquiring unit 111 acquires the first information IF1 from the storage device 12A. However, the acquisition source of the first information IF1 and the second information IF2 of the acquisition unit 111 may not be the storage device 12 or 12A. Specifically, the acquisition unit 111 may directly acquire the first information IF1 and the second information IF2 from the server 30 .
３．３：変形例３
　第１実施形態に係る端末装置１０は、処理装置１１の機能として、動作認識部１１２を備える。同様に、第２実施形態に係る端末装置１０Ａは、処理装置１１Ａの機能として、動作認識部１１２を備える。当該動作認識部１１２は、ユーザＵ１のジェスチャーを認識する。しかし、ユーザＵ１のジェスチャーの認識方法は、上記の方法に限定されない。例えば、ＡＲグラス２０が、動作認識部１１２と同様の動作認識部を備えることにより、ユーザＵ１のジェスチャーを認識してもよい。 3.3: Modification 3
The terminal device 10 according to the first embodiment includes a motion recognition unit 112 as a function of the processing device 11 . Similarly, the terminal device 10A according to the second embodiment includes a motion recognition unit 112 as a function of the processing device 11A. The action recognition unit 112 recognizes gestures of the user U1. However, the method of recognizing gestures of user U1 is not limited to the above method. For example, the AR glasses 20 may recognize gestures of the user U1 by including a motion recognition unit similar to the motion recognition unit 112 .
３．４：変形例４
　第１実施形態に係る端末装置１０は、処理装置１１の機能として、呼び名特定部１１３－２を備える。同様に、第２実施形態に係る端末装置１０Ａは、処理装置１１Ａの機能として、呼び名特定部１１３－２Ａを備える。呼び名特定部１１３－２及び１１３－２Ａは、仮想オブジェクト特定部１１３－１によって特定された仮想オブジェクトＶＯに対応する、呼び名としてのタグＴＧを特定する。一方で、呼び名特定部１１３－２及び１１３－２Ａは、仮想オブジェクト特定部１１３－１によって特定された仮想オブジェクトＶＯに対応するタグＴＧが存在しない場合には、タグＴＧを特定しない。この場合、処理装置１１及び１１Ａは、対応するタグＴＧが存在しない仮想オブジェクトＶＯに対して、新たなタグＴＧを設定する機能を備えてもよい。 3.4: Modification 4
The terminal device 10 according to the first embodiment includes, as a function of the processing device 11, a call name identifying section 113-2. Similarly, the terminal device 10A according to the second embodiment includes a call name identifying section 113-2A as a function of the processing device 11A. The nickname identifying units 113-2 and 113-2A identify a tag TG as a nickname corresponding to the virtual object VO identified by the virtual object identifying unit 113-1. On the other hand, the call name specifying units 113-2 and 113-2A do not specify the tag TG when there is no tag TG corresponding to the virtual object VO specified by the virtual object specifying unit 113-1. In this case, the processing devices 11 and 11A may have a function of setting a new tag TG for a virtual object VO that does not have a corresponding tag TG.
３．５：変形例５
　第１実施形態に係る情報処理システム１において、端末装置１０とＡＲグラス２０とは別体として実現されている。第２実施形態に係る情報処理システム１Ａにおいても同様に、端末装置１０ＡとＡＲグラス２０とは別体として実現されている。しかし、本発明の実施形態における、端末装置１０又は１０ＡとＡＲグラス２０の実現方法は、これには限定されない。例えば、ＡＲグラス２０が、端末装置１０又は１０Ａと同一の機能を備えることにより、端末装置１０又は１０ＡとＡＲグラス２０とが単一の筐体内で実現されてもよい。 3.5: Modification 5
In the information processing system 1 according to the first embodiment, the terminal device 10 and the AR glasses 20 are implemented separately. Similarly, in the information processing system 1A according to the second embodiment, the terminal device 10A and the AR glasses 20 are implemented separately. However, the method of implementing the terminal device 10 or 10A and the AR glasses 20 in the embodiment of the present invention is not limited to this. For example, the terminal device 10 or 10A and the AR glasses 20 may be realized within a single housing by providing the AR glasses 20 with the same functions as the terminal device 10 or 10A.
３．６：変形例６
　第２実施形態に係る端末装置１０Ａは、処理装置１１Ａの機能として、更新部１１７を備える。更新部１１７は、ユーザＵ１が発話した第１の音声の認識結果が第１情報ＩＦ１に含まれず、第２の音声の認識結果が第１情報ＩＦ１に含まれる特定のタグＴＧに一致した回数が所定回数に達した場合、当該特定のタグＴＧに対応する仮想オブジェクトＶＯを、第１の音声の認識結果である第２の呼び名としてのタグＴＧに対応付ける。すなわち、更新部１１７は、１つの仮想オブジェクトＶＯに対して、複数のタグＴＧを対応付ける。しかし、更新部１１７の動作は、これには限定されない。例えば、更新部１１７は、１つの仮想オブジェクトＶＯに対して複数のタグＴＧを対応付ける代わりに、当該１つの仮想オブジェクトＶＯに対して１つのタグＴＧを対応付けると共に、タグＴＧの種類の数だけ、当該１つの仮想オブジェクトＶＯと１つのタグＴＧとの組を設定してもよい。 3.6: Modification 6
10 A of terminal devices which concern on 2nd Embodiment are provided with the update part 117 as a function of 11 A of processing apparatuses. The update unit 117 updates the number of times the recognition result of the first voice uttered by the user U1 is not included in the first information IF1 and the recognition result of the second voice matches the specific tag TG included in the first information IF1. When the predetermined number of times has been reached, the virtual object VO corresponding to the specific tag TG is associated with the tag TG as the second name, which is the recognition result of the first voice. That is, the update unit 117 associates multiple tags TG with one virtual object VO. However, the operation of the updating unit 117 is not limited to this. For example, instead of associating a plurality of tags TG with one virtual object VO, the updating unit 117 associates one tag TG with one virtual object VO, A set of one virtual object VO and one tag TG may be set.
４：その他
（１）上述した実施形態では、記憶装置１２、記憶装置１２Ａ、記憶装置２２、及び記憶装置３２は、ＲＯＭ及びＲＡＭなどを例示したが、フレキシブルディスク、光磁気ディスク(例えば、コンパクトディスク、デジタル多用途ディスク、Ｂｌｕ－ｒａｙ（登録商標）ディスク)、スマートカード、フラッシュメモリデバイス(例えば、カード、スティック、キードライブ)、ＣＤ－ＲＯＭ（Compact　Disc－ＲＯＭ）、レジスタ、リムーバブルディスク、ハードディスク、フロッピー（登録商標）ディスク、磁気ストリップ、データベース、サーバその他の適切な記憶媒体である。また、プログラムは、電気通信回線を介してネットワークから送信されてもよい。また、プログラムは、電気通信回線を介して通信網ＮＥＴから送信されてもよい。 4: Others (1) In the above-described embodiment, the storage device 12, the storage device 12A, the storage device 22, and the storage device 32 are ROM and RAM, but flexible disks, magneto-optical disks (for example, compact disks) , Digital Versatile Discs, Blu-ray Discs), Smart Cards, Flash Memory Devices (e.g. Cards, Sticks, Key Drives), CD-ROMs (Compact Disc-ROMs), Registers, Removable Discs, Hard Disks, Floppy disk, magnetic strip, database, server or other suitable storage medium. Also, the program may be transmitted from a network via an electric communication line. Also, the program may be transmitted from the communication network NET via an electric communication line.
（２）上述した実施形態において、説明した情報、信号などは、様々な異なる技術のいずれかを使用して表されてもよい。例えば、上記の説明全体に渡って言及され得るデータ、命令、コマンド、情報、信号、ビット、シンボル、チップなどは、電圧、電流、電磁波、磁界若しくは磁性粒子、光場若しくは光子、又はこれらの任意の組み合わせによって表されてもよい。 (2) In the embodiments described above, the information, signals, etc. described may be represented using any of a variety of different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may refer to voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, light fields or photons, or any of these. may be represented by a combination of
（３）上述した実施形態において、入出力された情報等は特定の場所（例えば、メモリ）に保存されてもよいし、管理テーブルを用いて管理してもよい。入出力される情報等は、上書き、更新、又は追記され得る。出力された情報等は削除されてもよい。入力された情報等は他の装置へ送信されてもよい。 (3) In the above-described embodiments, input/output information and the like may be stored in a specific location (for example, memory), or may be managed using a management table. Input/output information and the like can be overwritten, updated, or appended. The output information and the like may be deleted. The entered information and the like may be transmitted to another device.
（４）上述した実施形態において、判定は、１ビットを用いて表される値（０か１か）によって行われてもよいし、真偽値（Boolean：true又はfalse）によって行われてもよいし、数値の比較（例えば、所定の値との比較）によって行われてもよい。 (4) In the above-described embodiment, the determination may be made by a value (0 or 1) represented using 1 bit, or by a true/false value (Boolean: true or false). Alternatively, it may be performed by numerical comparison (for example, comparison with a predetermined value).
（５）上述した実施形態において例示した処理手順、シーケンス、フローチャートなどは、矛盾の無い限り、順序を入れ替えてもよい。例えば、本開示において説明した方法については、例示的な順序を用いて様々なステップの要素を提示しており、提示した特定の順序に限定されない。 (5) The order of the processing procedures, sequences, flowcharts, etc. exemplified in the above embodiments may be changed as long as there is no contradiction. For example, the methods described in this disclosure present elements of the various steps using a sample order, and are not limited to the specific order presented.
（６）図１～図２９に例示された各機能は、ハードウェア及びソフトウェアの少なくとも一方の任意の組み合わせによって実現される。また、各機能ブロックの実現方法は特に限定されない。すなわち、各機能ブロックは、物理的又は論理的に結合した１つの装置を用いて実現されてもよいし、物理的又は論理的に分離した２つ以上の装置を直接的又は間接的に（例えば、有線、無線などを用いて）接続し、これら複数の装置を用いて実現されてもよい。機能ブロックは、上記１つの装置又は上記複数の装置にソフトウェアを組み合わせて実現されてもよい。 (6) Each function illustrated in FIGS. 1 to 29 is realized by any combination of at least one of hardware and software. Also, the method of realizing each functional block is not particularly limited. That is, each functional block may be implemented using one device that is physically or logically coupled, or directly or indirectly using two or more devices that are physically or logically separated (e.g. , wired, wireless, etc.) and may be implemented using these multiple devices. A functional block may be implemented by combining software in the one device or the plurality of devices.
（７）上述した実施形態において例示したプログラムは、ソフトウェア、ファームウェア、ミドルウェア、マイクロコード、ハードウェア記述言語と呼ばれるか、他の名称を用いて呼ばれるかを問わず、命令、命令セット、コード、コードセグメント、プログラムコード、プログラム、サブプログラム、ソフトウェアモジュール、アプリケーション、ソフトウェアアプリケーション、ソフトウェアパッケージ、ルーチン、サブルーチン、オブジェクト、実行可能ファイル、実行スレッド、手順、機能などを意味するよう広く解釈されるべきである。 (7) The programs illustrated in the above embodiments, whether referred to as software, firmware, middleware, microcode, hardware description language or by other names, instructions, instruction sets, code, code shall be interpreted broadly to mean segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executables, threads of execution, procedures, functions, and the like.
　また、ソフトウェア、命令、情報などは、伝送媒体を介して送受信されてもよい。例えば、ソフトウェアが、有線技術（同軸ケーブル、光ファイバケーブル、ツイストペア、デジタル加入者回線（ＤＳＬ：Digital　Subscriber　Line）など）及び無線技術（赤外線、マイクロ波など）の少なくとも一方を使用してウェブサイト、サーバ、又は他のリモートソースから送信される場合、これらの有線技術及び無線技術の少なくとも一方は、伝送媒体の定義内に含まれる。 In addition, software, instructions, information, etc. may be transmitted and received via a transmission medium. For example, the software uses at least one of wired technology (coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), etc.) and wireless technology (infrared, microwave, etc.) to website, Wired and/or wireless technologies are included within the definition of transmission medium when sent from a server or other remote source.
（８）前述の各形態において、「システム」及び「ネットワーク」という用語は、互換的に使用される。 (8) In each of the above aspects, the terms "system" and "network" are used interchangeably.
（９）本開示において説明した情報、パラメータなどは、絶対値を用いて表されてもよいし、所定の値からの相対値を用いて表されてもよいし、対応する別の情報を用いて表されてもよい。 (9) Information, parameters, etc. described in this disclosure may be expressed using absolute values, may be expressed using relative values from a predetermined value, or may be expressed using corresponding other information. may be represented as
（１０）上述した実施形態において、端末装置１０、端末装置１０Ａ、及びサーバ３０は、移動局（ＭＳ：Mobile　Station）である場合が含まれる。移動局は、当業者によって、加入者局、モバイルユニット、加入者ユニット、ワイヤレスユニット、リモートユニット、モバイルデバイス、ワイヤレスデバイス、ワイヤレス通信デバイス、リモートデバイス、モバイル加入者局、アクセス端末、モバイル端末、ワイヤレス端末、リモート端末、ハンドセット、ユーザエージェント、モバイルクライアント、クライアント、又はいくつかの他の適切な用語で呼ばれる場合もある。また、本開示においては、「移動局」、「ユーザ端末（user　terminal）」、「ユーザ装置（ＵＥ：User　Equipment）」、「端末」等の用語は、互換的に使用され得る。 (10) In the above-described embodiments, the terminal device 10, the terminal device 10A, and the server 30 may be mobile stations (MS). A mobile station is defined by those skilled in the art as a subscriber station, mobile unit, subscriber unit, wireless unit, remote unit, mobile device, wireless device, wireless communication device, remote device, mobile subscriber station, access terminal, mobile terminal, wireless It may also be called a terminal, remote terminal, handset, user agent, mobile client, client, or some other suitable term. Also, in the present disclosure, terms such as "mobile station", "user terminal", "user equipment (UE)", "terminal", etc. may be used interchangeably.
（１１）上述した実施形態において、「接続された(connected)」、「結合された(coupled)」という用語、又はこれらのあらゆる変形は、２又はそれ以上の要素間の直接的又は間接的なあらゆる接続又は結合を意味し、互いに「接続」又は「結合」された２つの要素間に１又はそれ以上の中間要素が存在することを含むことができる。要素間の結合又は接続は、物理的な結合又は接続であっても、論理的な結合又は接続であっても、或いはこれらの組み合わせであってもよい。例えば、「接続」は「アクセス」を用いて読み替えられてもよい。本開示において使用する場合、２つの要素は、１又はそれ以上の電線、ケーブル及びプリント電気接続の少なくとも一つを用いて、並びにいくつかの非限定的かつ非包括的な例として、無線周波数領域、マイクロ波領域及び光（可視及び不可視の両方）領域の波長を有する電磁エネルギーなどを用いて、互いに「接続」又は「結合」されると考えることができる。 (11) In the above-described embodiments, the terms "connected," "coupled," or any variation thereof refer to any direct or indirect connection between two or more elements. Any connection or coupling is meant and can include the presence of one or more intermediate elements between two elements that are "connected" or "coupled" to each other. Couplings or connections between elements may be physical couplings or connections, logical couplings or connections, or a combination thereof. For example, "connection" may be replaced with "access". As used in this disclosure, two elements are defined using at least one of one or more wires, cables, and printed electrical connections and, as some non-limiting and non-exhaustive examples, in the radio frequency domain. , electromagnetic energy having wavelengths in the microwave and optical (both visible and invisible) regions, and the like.
（１２）上述した実施形態において、「に基づいて」という記載は、別段に明記されていない限り、「のみに基づいて」を意味しない。言い換えれば、「に基づいて」という記載は、「のみに基づいて」と「に少なくとも基づいて」の両方を意味する。 (12) In the above-described embodiments, the phrase "based on" does not mean "based only on," unless expressly specified otherwise. In other words, the phrase "based on" means both "based only on" and "based at least on."
（１３）本開示において使用される「判断(determining)」、「決定(determining)」という用語は、多種多様な動作を包含する場合がある。「判断」、「決定」は、例えば、判定(judging)、計算(calculating)、算出(computing)、処理(processing)、導出(deriving)、調査(investigating)、探索(looking　up、search、inquiry)（例えば、テーブル、データベース又は別のデータ構造での探索）、確認(ascertaining)した事を「判断」「決定」したとみなす事などを含み得る。また、「判断」、「決定」は、受信(receiving)（例えば、情報を受信すること）、送信(transmitting)(例えば、情報を送信すること)、入力(input)、出力(output)、アクセス(accessing)（例えば、メモリ中のデータにアクセスすること）した事を「判断」「決定」したとみなす事などを含み得る。また、「判断」、「決定」は、解決(resolving)、選択(selecting)、選定(choosing)、確立(establishing)、比較(comparing)などした事を「判断」「決定」したとみなす事を含み得る。つまり、「判断」「決定」は、何らかの動作を「判断」「決定」したとみなす事を含み得る。また、「判断（決定）」は、「想定する（assuming）」、「期待する（expecting）」、「みなす（considering）」などで読み替えられてもよい。 (13) The terms "determining" and "determining" as used in this disclosure may encompass a wide variety of actions. "Judgement" and "determination" are, for example, judging, calculating, computing, processing, deriving, investigating, looking up, searching, inquiring (eg, lookup in a table, database, or other data structure), ascertaining as "judged" or "determined", and the like. Also, "judgment" and "determination" are used for receiving (e.g., receiving information), transmitting (e.g., transmitting information), input, output, access (accessing) (for example, accessing data in memory) may include deeming that a "judgment" or "decision" has been made. In addition, "judgment" and "decision" are considered to be "judgment" and "decision" by resolving, selecting, choosing, establishing, comparing, etc. can contain. In other words, "judgment" and "decision" may include considering that some action is "judgment" and "decision". Also, "judgment (decision)" may be read as "assuming", "expecting", "considering", or the like.
（１４）上述した実施形態において、「含む（include）」、「含んでいる（including）」及びそれらの変形が使用されている場合、これらの用語は、用語「備える（comprising）」と同様に、包括的であることが意図される。更に、本開示において使用されている用語「又は（or）」は、排他的論理和ではないことが意図される。 (14) In the above-described embodiments, where "include," "including," and variations thereof are used, these terms are synonymous with the term "comprising." , is intended to be inclusive. Furthermore, the term "or" as used in this disclosure is not intended to be an exclusive OR.
（１５）本開示において、例えば、英語でのa, an及びtheのように、翻訳により冠詞が追加された場合、本開示は、これらの冠詞の後に続く名詞が複数形であることを含んでもよい。 (15) In this disclosure, where articles have been added by translation, such as a, an, and the in English, the disclosure includes the plural nouns following these articles. good.
（１６）本開示において、「ＡとＢが異なる」という用語は、「ＡとＢが互いに異なる」ことを意味してもよい。なお、当該用語は、「ＡとＢがそれぞれＣと異なる」ことを意味してもよい。「離れる」、「結合される」等の用語も、「異なる」と同様に解釈されてもよい。 (16) In the present disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean that "A and B are different from C". Terms such as "separate," "coupled," etc. may also be interpreted in the same manner as "different."
（１７）本開示において説明した各態様／実施形態は単独で用いてもよいし、組み合わせて用いてもよいし、実行に伴って切り替えて用いてもよい。また、所定の情報の通知（例えば、「Ｘであること」の通知）は、明示的に行う通知に限られず、暗黙的（例えば、当該所定の情報の通知を行わない）ことによって行われてもよい。 (17) Each aspect/embodiment described in the present disclosure may be used alone, may be used in combination, or may be used by switching according to execution. In addition, notification of predetermined information (for example, notification of “being X”) is not limited to explicit notification, but is performed implicitly (for example, not notification of the predetermined information). good too.
　以上、本開示について詳細に説明したが、当業者にとっては、本開示が本開示中に説明した実施形態に限定されないということは明らかである。本開示は、請求の範囲の記載により定まる本開示の趣旨及び範囲を逸脱することなく修正及び変更態様として実施できる。したがって、本開示の記載は、例示説明を目的とし、本開示に対して何ら制限的な意味を有さない。 Although the present disclosure has been described in detail above, it is clear to those skilled in the art that the present disclosure is not limited to the embodiments described in this disclosure. The present disclosure can be practiced with modifications and variations without departing from the spirit and scope of the present disclosure as defined by the claims. Accordingly, the description of the present disclosure is for illustrative purposes and is not meant to be limiting in any way on the present disclosure.
１、１Ａ…情報処理システム、１０、１０Ａ…端末装置、１１、１１Ａ…処理装置、１２、１２Ａ…記憶装置、１３…通信装置、１４…ディスプレイ、１５…入力装置、２０…ＡＲグラス、２１…処理装置、２２…記憶装置、２３…視線取得装置、２４…収音装置、２５…ＧＰＳ装置、２６…動き検出装置、２７…撮像装置、２８…通信装置、２９…ディスプレイ、３０…サーバ、３１…処理装置、３２…記憶装置、３３…通信装置、３４…ディスプレイ、３５…入力装置、４１Ｌ、４１Ｒ…レンズ、９１、９２…テンプル、９３…ブリッジ、９４、９５…胴部、１１１…取得部、１１２…動作認識部、１１３、１１３Ａ…特定部、１１３－１…仮想オブジェクト特定部、１１３－２、１１３－２Ａ…呼び名特定部、１１４、１１４Ａ…表示制御部、１１５…判定部、１１６…音声認識部、１１７…更新部、２１１…取得部、２１２…表示制御部、３１１…出力部、３１２…取得部、ＩＦ１…第１情報、ＩＦ２…第２情報、ＬＭ１…学習モデル、Ｐ１、Ｐ２、Ｐ３…ポップアップ、ＰＲ１、ＰＲ２、ＰＲ３、ＰＲ４…制御プログラム、Ｒ…領域、ＴＧ…タグ、Ｕ１、Ｕ２、Ｕ３…ユーザ、ＶＯ…仮想オブジェクト Reference Signs List 1, 1A... Information processing system 10, 10A... Terminal device 11, 11A... Processing device 12, 12A... Storage device 13... Communication device 14... Display 15... Input device 20... AR glass 21... Processing device 22 Storage device 23 Line of sight acquisition device 24 Sound pickup device 25 GPS device 26 Motion detection device 27 Imaging device 28 Communication device 29 Display 30 Server 31 Processing device 32 Storage device 33 Communication device 34 Display 35 Input device 41L, 41R Lens 91, 92 Temple 93 Bridge 94, 95 Body 111 Acquisition unit , 112... Action recognition unit 113, 113A... Specification unit 113-1... Virtual object specification unit 113-2, 113-2A... Call name specification unit 114, 114A... Display control unit 115... Judgment unit 116... Voice recognition unit 117 Update unit 211 Acquisition unit 212 Display control unit 311 Output unit 312 Acquisition unit IF1 First information IF2 Second information LM1 Learning model P1, P2 , P3... popup, PR1, PR2, PR3, PR4... control program, R... area, TG... tag, U1, U2, U3... user, VO... virtual object

Claims (10)

  1. 　ユーザの頭部に装着される表示装置に、仮想空間に配置される複数の仮想オブジェクトを表示させる表示制御部と、
    　前記ユーザの動作に応じて生成される指示情報に基づいて前記複数の仮想オブジェクトのうち第１の仮想オブジェクトを特定する仮想オブジェクト特定部と、
    　前記仮想オブジェクト特定部が特定した第１の仮想オブジェクトに対応する呼び名が記憶装置に記憶されている場合、当該対応する呼び名を第１の呼び名として特定する呼び名特定部と
    　を備え、
    　前記表示制御部は、前記表示装置に前記第１の呼び名を表示させる、
    　情報処理装置。     a display control unit for displaying a plurality of virtual objects arranged in a virtual space on a display device worn on the user's head;
    a virtual object identifying unit that identifies a first virtual object among the plurality of virtual objects based on instruction information generated according to the user's action;
    a nickname identifying unit that identifies the corresponding nickname as a first nickname when a nickname corresponding to the first virtual object identified by the virtual object identifying unit is stored in a storage device;
    The display control unit causes the display device to display the first nickname.
    Information processing equipment.
  2. 　前記表示制御部は、前記表示装置に、前記仮想空間を平面展開した２次元画像を表示させ、前記２次元画像内において、前記第１の仮想オブジェクトに関連付けて前記第１の呼び名を表示させる、請求項１に記載の情報処理装置。 The display control unit causes the display device to display a two-dimensional image obtained by planarizing the virtual space, and causes the first name to be displayed in association with the first virtual object in the two-dimensional image. The information processing device according to claim 1 .
  3. 　前記表示制御部は、前記表示装置に、前記仮想空間を縮小した３次元画像を表示させ、前記３次元画像内において、前記第１の仮想オブジェクトに関連付けて前記第１の呼び名を表示させる、請求項１に記載の情報処理装置。 The display control unit causes the display device to display a three-dimensional image obtained by reducing the virtual space, and causes the first name to be displayed in the three-dimensional image in association with the first virtual object. Item 1. The information processing apparatus according to item 1.
  4. 　前記ユーザの動作は前記表示装置に対する目視であり、前記指示情報は前記表示装置における前記ユーザの視点を示し、
    　前記表示装置に表示される前記第１の仮想オブジェクト内に、前記視点が所定時間以上位置するか否かを判定する判定部を更に備え、
    　前記第１の仮想オブジェクト内に前記視点が前記所定時間以上位置すると前記判定部によって判定された場合に、前記表示制御部は、前記表示装置に、前記第１の仮想オブジェクトに対応する前記第１の呼び名を表示させる、請求項１に記載の情報処理装置。     the user's action is viewing the display device, the instruction information indicates the user's point of view on the display device;
    further comprising a determination unit that determines whether the viewpoint is positioned within the first virtual object displayed on the display device for a predetermined time or longer;
    When the determination unit determines that the viewpoint is positioned within the first virtual object for the predetermined time or longer, the display control unit causes the display device to display the first virtual object corresponding to the first virtual object. 2. The information processing apparatus according to claim 1, wherein the name of the name of the information processing apparatus is displayed.
  5. 　前記表示制御部は、前記ユーザが発話する音声の認識結果と前記第１の呼び名とが一致した場合に、前記第１の仮想オブジェクトに関する表示内容を変更させる、請求項１から請求項４のいずれか１項に記載の情報処理装置。 5. The display control unit according to any one of claims 1 to 4, wherein, when the recognition result of the voice uttered by the user matches the first name, the display content regarding the first virtual object is changed. 1. The information processing apparatus according to 1.
  6. 　ユーザの頭部に装着される表示装置に、仮想空間に配置される複数の仮想オブジェクトを表示させる表示制御部と、
    　前記ユーザが発話する音声であって、前記複数の仮想オブジェクトの少なくともいずれかの属性を表す音声の認識結果に基づいて、１以上の仮想オブジェクトを特定する仮想オブジェクト特定部と、
    　前記１以上の仮想オブジェクトの一部又は全部の仮想オブジェクトの各々に対応する呼び名を特定する呼び名特定部と
    　を備え、
    　前記表示制御部は、前記一部又は全部の仮想オブジェクトの各々に関連付けて、前記呼び名特定部により特定された前記対応する呼び名を前記表示装置に表示させる、
    　情報処理装置。     a display control unit for displaying a plurality of virtual objects arranged in a virtual space on a display device worn on the user's head;
    a virtual object identifying unit that identifies one or more virtual objects based on a recognition result of the voice uttered by the user and representing at least one attribute of the plurality of virtual objects;
    a nickname identifying unit that identifies a nickname corresponding to each of a part or all of the one or more virtual objects,
    The display control unit causes the display device to display the corresponding nickname specified by the nickname specifying unit in association with each of the part or all of the virtual objects.
    Information processing equipment.
  7. 　ユーザの頭部に装着される表示装置に、仮想空間に配置される複数の仮想オブジェクトを表示させる表示制御部と、
    　前記ユーザが発話した第１の音声の認識結果が、前記複数の仮想オブジェクトに対応する複数の第１の呼び名のいずれにも一致しない第２の呼び名である場合に、前記複数の第１の呼び名のうち、前記第２の呼び名に最も類似する第１の呼び名を特定する呼び名特定部と
    　を備え、
    　前記表示制御部は、前記呼び名特定部が特定した第１の呼び名を前記表示装置に表示させる、
    　情報処理装置。     a display control unit for displaying a plurality of virtual objects arranged in a virtual space on a display device worn on the user's head;
    When the recognition result of the first voice uttered by the user is a second nickname that does not match any of the plurality of first nicknames corresponding to the plurality of virtual objects, the plurality of first nicknames a nickname identifying unit that identifies the first nickname most similar to the second nickname among
    The display control unit causes the display device to display the first nickname specified by the nickname specifying unit.
    Information processing equipment.
  8. 　前記表示制御部は、前記呼び名特定部が特定した第１の呼び名が前記表示装置に表示された後、前記ユーザが発話した第２の音声の認識結果が、前記特定した第１の呼び名に一致する場合、前記複数の仮想オブジェクトのうち前記特定した第１の呼び名に対応する仮想オブジェクトに関する表示を変更させ、
    　前記第１の音声の認識結果が前記第２の呼び名となる回数が所定回数に達した場合、前記仮想オブジェクトを、前記第２の呼び名に対応付ける更新部を更に備える、
    　請求項７に記載の情報処理装置。     After the first nickname specified by the nickname specifying unit is displayed on the display device, the display control unit causes the recognition result of the second voice uttered by the user to match the specified first nickname. if so, changing the display of the virtual object corresponding to the identified first name among the plurality of virtual objects;
    Further comprising an updating unit that associates the virtual object with the second nickname when the number of times that the recognition result of the first voice becomes the second nickname reaches a predetermined number of times,
    The information processing apparatus according to claim 7.
  9. 　ユーザの頭部に装着される表示装置に、仮想空間に配置される複数の仮想オブジェクトを表示させる表示制御部と、
    　前記複数の仮想オブジェクトの一部又は全部に対応する複数の第１の呼び名を特定する呼び名特定部と
    　を備え、
    　前記表示制御部は、前記ユーザが発話する音声の認識結果が、前記複数の第１の呼び名のいずれにも一致しない第２の呼び名である場合に、前記呼び名特定部によって特定された前記複数の第１の呼び名の各々を、前記複数の仮想オブジェクトの一部又は全部のうち対応する仮想オブジェクトに関連付けて表示させる情報処理装置。     a display control unit for displaying a plurality of virtual objects arranged in a virtual space on a display device worn on the user's head;
    a nickname identifying unit that identifies a plurality of first nicknames corresponding to some or all of the plurality of virtual objects;
    When the recognition result of the voice uttered by the user is a second nickname that does not match any of the plurality of first nicknames, the display control unit controls the display of the plurality of nicknames specified by the nickname specifying unit. An information processing device for displaying each of the first nicknames in association with a corresponding virtual object out of some or all of the plurality of virtual objects.
  10. 　前記表示制御部は、前記呼び名特定部が特定した複数の第１の呼び名が前記表示装置に表示された後、前記ユーザが発話した第２の音声の認識結果が、前記特定した複数の第１の呼び名のいずれかに一致する場合、前記複数の仮想オブジェクトのうち前記一致した第１の呼び名に対応する仮想オブジェクトに関する表示を変更させ、
    　前記第１の音声の認識結果が前記第２の呼び名となる回数が所定回数に達した場合、前記仮想オブジェクトを、前記第２の呼び名に対応付ける更新部を更に備える、
    　請求項９に記載の情報処理装置。     The display control unit, after the plurality of first names specified by the name specifying unit are displayed on the display device, recognizes the recognition result of the second voice uttered by the user as the plurality of specified first names. if any of the names match, changing the display of the virtual object corresponding to the matched first name among the plurality of virtual objects;
    Further comprising an updating unit that associates the virtual object with the second nickname when the number of times that the recognition result of the first voice becomes the second nickname reaches a predetermined number of times,
    The information processing apparatus according to claim 9 .
PCT/JP2022/036695 2021-11-04 2022-09-30 Information processing device WO2023079875A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2021180306 2021-11-04
JP2021-180306 2021-11-04

Publications (1)

Publication Number Publication Date
WO2023079875A1 true WO2023079875A1 (en) 2023-05-11

Family

ID=86241425

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/036695 WO2023079875A1 (en) 2021-11-04 2022-09-30 Information processing device

Country Status (1)

Country Link
WO (1) WO2023079875A1 (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2012093422A (en) * 2010-10-25 2012-05-17 Denso Corp Voice recognition device
JP2020016784A (en) * 2018-07-26 2020-01-30 Ｚホールディングス株式会社 Recognition device, recognition method, and recognition program
WO2020110270A1 (en) * 2018-11-29 2020-06-04 マクセル株式会社 Video display device and method

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2012093422A (en) * 2010-10-25 2012-05-17 Denso Corp Voice recognition device
JP2020016784A (en) * 2018-07-26 2020-01-30 Ｚホールディングス株式会社 Recognition device, recognition method, and recognition program
WO2020110270A1 (en) * 2018-11-29 2020-06-04 マクセル株式会社 Video display device and method

Similar Documents

Publication Publication Date Title
US11995774B2 (en) Augmented reality experiences using speech and text captions
CN111652678B (en) Method, device, terminal, server and readable storage medium for displaying article information
US11127210B2 (en) Touch and social cues as inputs into a computer
US9390561B2 (en) Personal holographic billboard
US20130174213A1 (en) Implicit sharing and privacy control through physical behaviors using sensor-rich devices
WO2013106128A1 (en) Method and apparatus for enabling real-time product and vendor identification
CN103105926A (en) Multi-sensor posture recognition
US11532227B2 (en) Discovery of and connection to remote devices
WO2015163068A1 (en) Information-processing device, information-processing method, and computer program
CN111432245B (en) Multimedia information playing control method, device, equipment and storage medium
WO2019130991A1 (en) Information processing device
US20220319126A1 (en) System and method for providing an augmented reality environment for a digital platform
WO2023079875A1 (en) Information processing device
KR20200122754A (en) Smart glass system for providing augmented reality image
WO2023145892A1 (en) Display control device, and server
WO2023074852A1 (en) Information processing apparatus
WO2023149255A1 (en) Display control device
WO2023149379A1 (en) Information processing device
WO2023120472A1 (en) Avatar generation system
WO2023145265A1 (en) Message transmitting device and message receiving device
CN112487162A (en) Method, device and equipment for determining text semantic information and storage medium
WO2023112838A1 (en) Information processing device
WO2023149256A1 (en) Display control device
KR20220026114A (en) Electronic device, and method for identifing realted external electronic device in augmented reality of electronic device
WO2023149498A1 (en) Display control device

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22889699

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2023557896

Country of ref document: JP

Kind code of ref document: A