JP2014203194A - Virtual object display control apparatus, virtual object display control method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the problem that if virtual objects is arranged and kept at arbitrary positions in real space, while the virtual objects increase, the reality space is filled with the virtual objects, and the virtual objects is troublesome.SOLUTION: A user satisfying an erasure condition of a virtual object is determined on the basis of a conversation state when the virtual object is arranged and displayed. Then, the virtual object is erased on the basis of distance between the user satisfying the erasure condition and the virtual object.

Description

  The present invention relates to a virtual object display control device, a virtual object display control method, and a program.

  In recent years, systems using AR (Augmented Reality) technology that superimposes additional information on the real world and presents it to the user have been developed and provided. For example, in a system called Sekai Camera (registered trademark), a virtual tag associated with an arbitrary position in the real space is registered in the system in advance. Then, in the image captured by the terminal carried by the user, the tag associated with the position shown in the image is transmitted from the system server to the terminal, and is superimposed on the position and displayed in the image.

  Moreover, in the system described in Patent Document 1, a flight path is projected on the head-up display device. This flight path is calculated based on the position and posture of the aircraft and flight path data, and is displayed superimposed on the foreground on a head-up display device provided in front of the pilot. On the other hand, in a state where the landing position is close to the landing position and the landing position can be visually observed, the pilot usually lands by visual observation flight. In this case, since the display image of the head-up display device may be an obstacle, in Patent Document 1, when the distance between the landing position and the aircraft is equal to or less than a predetermined distance, the flight path displayed in a superimposed manner is displayed. Erase from the head-up display device.

Japanese Patent Laid-Open No. 11-268696

  When the AR technology is used, a virtual object can be arranged at an arbitrary position in the real space in a meeting place. Thereby, a virtual object can be browsed and shared as if a plurality of users are viewing the same object through each terminal.

  However, if the virtual objects are arranged at arbitrary positions in the real space, the virtual objects increase, but the virtual objects overflow in the real space, which is troublesome.

  According to the method of the above-mentioned patent document 1, a virtual object can be deleted according to the position of a device that arranges and displays the virtual object. However, when viewing the same virtual object by multiple persons, if the virtual object is deleted according to the position of the device on which the virtual object is arranged and displayed, it can be viewed on the device of another user who is viewing the virtual object. It may disappear.

  The virtual object display control apparatus according to the present invention includes a receiving unit that receives a display notification of a virtual object, an acquisition unit that acquires a conversation state of a notification source user of the display notification, and the virtual object based on the acquired conversation state. It has a determining means for determining an erasing condition user as an erasing condition for an object, and an erasing means for erasing the virtual object according to the position of the erasing condition user and the position of the virtual object.

  According to the present invention, a virtual object can be deleted when the virtual object is no longer needed. Therefore, the real space is not overwhelmed with virtual objects, and usability is improved.

1 is a block diagram of a display device in Example 1. FIG. 1 is a block diagram of a virtual object display control device in Embodiment 1. FIG. 3 is a flowchart of self-position direction specifying processing in the first embodiment. It is a figure which shows the example of the self-position direction management table in Example 1. FIG. 3 is a flowchart of speech detection processing in the first embodiment. It is a figure which shows the example of the speech management table in Example 1. FIG. It is a figure which shows the example of the planar arrangement | positioning figure of the user in Example 1. FIG. 3 is a flowchart of conversation state detection processing in the first embodiment. It is a figure which shows the example of the conversation management table in Example 1. FIG. 6 is a diagram illustrating an example of a planar arrangement diagram of a user and a virtual object in Embodiment 1. FIG. 6 is a flowchart of an erasing condition setting process in Embodiment 1. 6 is a diagram illustrating an example of an erasing condition table in Embodiment 1. FIG. It is a figure which shows the example of the self-position direction management table and conversation management table in Example 2. FIG. 9 is a flowchart of an erasing condition setting process in Embodiment 2. 10 is a diagram illustrating an example of an erasing condition table in Embodiment 2. FIG. It is a figure which shows the example of the self-position direction management table in Example 3, and a conversation management table. It is z which shows the example of the planar arrangement figure of the user in Example 3, and a virtual object. 10 is a flowchart of an erasing condition setting process in Embodiment 3. FIG. 10 is a diagram illustrating an example of an erasing condition table in the third embodiment.

  Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. The following embodiments are merely examples, and are not intended to limit the scope of the present invention.

(Example 1)
FIG. 1 is a block diagram illustrating a configuration of a display device 100 that displays a virtual object. An example of the display device 100 is an HMD (Head-Mounted Display) worn on the user's head. The user can wear the HMD like glasses, and the subject in the user's line-of-sight direction can be imaged by the imaging unit 101 attached to the front surface of the HMD. In addition, the display unit 105 attached to the front surface of the HMD allows the user to view the subject imaged by the imaging unit 101 and the virtual object associated with the subject while wearing the HMD. Note that a virtual object is additional information superimposed on a real-world image. For example, it may be a tag representing information as described above, or may be a text, an icon, a mark, a photograph, a moving image, or the like. The display device 100 may be a portable terminal device such as a smartphone or a tablet.

  The display device 100 includes an imaging unit 101, a CPU 102, an utterance detection unit 103, an operation unit 104, a display unit 105, a ROM 106, a RAM 107, a large capacity storage unit 108, a network controller 109, a self-position / direction acquisition unit 110, and an image processing unit 111. Have.

  The CPU 102 controls the entire display device 100 based on a boot loader program and a display device control program stored in the ROM 106. In addition, the self-position direction acquisition unit 110 acquires the self-position direction based on an application program stored in the large-capacity storage unit 108. The self-position direction refers to the position and orientation of the display device 100.

  The imaging unit 101 captures captured image data by capturing an image of a subject viewed by a user wearing the display device 100. The captured image data acquired by the imaging unit 101 includes a marker image for calculating the self-position direction. A marker image is an image showing a predetermined mark added to a predetermined position in the real space.

  The utterance detection unit 103 detects an utterance uttered by a person wearing the display device 100 through a microphone or the like, and acquires voice data. Details will be described later. The operation unit 104 is used to select a virtual object to be displayed. The image processing unit 111 interprets the virtual object selected by the operation unit 104 and generates image data. The display unit 105 displays the image data generated by the image processing unit. The ROM 106 stores a boot loader program and a display device control program. The RAM 107 functions as a work area for the CPU 102. The network controller 109 communicates with the virtual object display control device 200 connected to the external network. The large-capacity storage unit 108 stores application programs, and temporarily stores captured image data captured by the imaging unit 101 and audio data acquired by the utterance detection unit 103.

  FIG. 2 is a block diagram illustrating a configuration of the virtual object display control apparatus 200. The virtual object display control device 200 functions as a database server. The virtual object display control apparatus 200 includes a CPU 201, ROM 202, RAM 203, large capacity storage unit 204, operation unit 205, display unit 206, network controller 207, conversation state detection unit 208, deletion condition setting unit 209, and deletion unit 210.

  In FIG. 2, a CPU 201 controls the entire virtual object display control device based on a boot loader program and a database server control program stored in a ROM 202. The conversation status detection unit 208 detects the conversation status based on the application program stored in the large capacity storage unit 204. An erasing condition setting unit 209 sets erasing conditions for a virtual object based on an application program stored in the large capacity storage unit 204. The erase unit 210 erases the virtual object based on the erase condition set by the erase condition setting unit 209. The RAM 203 functions as a work area for the CPU 201. The large-capacity storage unit 204 stores application programs, and stores a self-location direction management table 400 and an utterance management table 600. Details of these tables will be described later. The operation unit 205 controls an input device such as a keyboard and a pointing device. The display unit 206 performs display based on the database server control program by the CPU 201.

  Next, a self-position direction acquisition method of the display device 100 operated by the user using the display device 100 and the virtual object display control device 200 will be described with reference to the flowchart of FIG. The process shown in the flowchart of FIG. 3 is repeatedly performed at predetermined timings. There are various methods for acquiring the self-position direction. For example, there are a technique using a feature point in a captured image and a technique of extracting a feature point from a marker image. In this embodiment, a technique using a marker image will be described, but other techniques may be used. 3, steps S301 to S304 are processing on the display device 100 side, and steps S351 to S354 are processing on the virtual object display control device 200 side. Steps S301 to S304 in the flowchart of FIG. 3 are executed under the control of the CPU 102 of the display device 100. In FIG. 3, the processing on the display device 100 side and the processing on the virtual object display control device 200 side are collectively described for ease of explanation, but each processing is performed separately and independently. is there.

  In step S <b> 301, the imaging unit 101 captures a real space image in the line-of-sight direction of the user wearing the display device 100 and acquires captured image data. The captured image data is sent to the CPU 102 at predetermined intervals and stored in the RAM 107. The predetermined interval can be set to an arbitrary value such as 5 frames per second.

  In step S <b> 302, the self-position direction acquisition unit 110 detects a marker image from captured image data indicating a real space image stored in the RAM 107. A marker image is an image whose three-dimensional position is known in real space. The marker image is detected by matching, for example, a marker image whose shape, color, area, etc. are designated in advance with a real space image.

  In step S303, the self-position direction acquisition unit 110 acquires the self-position direction from the distortion of the marker image detected in step S303. For example, the self-position direction acquisition unit 110 acquires the self-position direction by projecting a two-dimensional position on the imaging surface of a known marker image included in the real space image to a three-dimensional position. Information indicating the self-position direction of the display device 100 is acquired as a component including an X, Y, Z component representing a position as a Cartesian coordinate in space, and a component representing an azimuth angle and an elevation angle. Note that a method for acquiring the self-position direction from the marker image can be performed using a known technique (for example, Japanese Patent Application Laid-Open No. 2005-33319), and thus detailed description thereof is omitted. In this embodiment, the self-position direction is described as being detected by detecting a marker image. However, the self-position direction may be detected by detecting a feature point of an object in real space, or may be combined with various sensors or GPS. The method of acquiring the self-position direction may be used. In this example, the self-position direction will be described using an example of a component including an X, Y, and Z component representing a position as a Cartesian coordinate in space, and a component representing an azimuth angle and an elevation angle. However, it may be information indicating only a self-position that is an X, Y, Z component representing a position as a Cartesian coordinate of space. The self position direction acquisition unit 110 adds time information at the time of acquisition in the acquired self position direction.

  In step S <b> 304, the CPU 102 transmits the self-position direction acquired in step S <b> 304 and the time information added to the virtual object display control device 200 via the network controller 109 together with the user ID of the user who uses the display device 100. In this embodiment, an example in which a user ID is used as an ID will be described, but an ID for identifying the display device 100 may be used.

  Next, processing on the virtual object display control apparatus 200 side will be described using steps S351 to S354. Steps S351 to S354 are executed under the control of the CPU 201 of the virtual object display control device 200. Note that the CPU 201 reads the self-position / direction management table from the large-capacity storage unit 204 to the RAM 203 in advance.

  In step S <b> 351, the virtual object display control device 200 receives the self-position direction transmitted from the display device 100 via the network controller 207. As described above, the time information and the user ID of the display device 100 are added to the self-position direction transmitted from the display device 100. The CPU 201 stores the received self-position direction in the RAM 203.

  In step S <b> 352, the CPU 201 determines whether the received user ID added in the self-position direction is in the self-position direction management table managed by the virtual object display control device 200. The self position direction management table is a table for managing the self position direction transmitted from each display device. When determining that the user ID is not in the self-position / direction management table, the CPU 201 advances the processing to step S353. In step S353, the CPU 201 newly registers the self position direction, user ID, and time information received in step S351 in the self position direction management table. On the other hand, if it is determined that the user ID is not in the self-location direction management table, the CPU 201 advances the process to step S354. In step S354, the CPU 201 updates the self position direction and time information associated with the matching user ID in the self position direction management table with the information received in step S351.

  FIG. 4 is a diagram illustrating an example of the self-position / direction management table 400 managed by the virtual object display control apparatus 200. The self-position / direction management table 400 includes the user ID of each user, time information, self-position information, and self-direction information. Here, the unit of position is described as millimeters, but a unit representing a meter or other distance may be used. Here, the unit of orientation is set to degree, but the unit representing the orientation may be radian. Assuming that the azimuth angle is 0 degree north, the east, south, and west are represented as 90 degrees, 180 degrees, and 270 degrees, respectively. As the azimuth angle, an arbitrary azimuth can be set to 0 degree, and it is only necessary to know how many directions of all users are from the reference azimuth. The elevation angle is 0 degrees in the horizontal direction, 90 degrees in the zenith direction, and -90 degrees in the vertical direction.

  Next, a user utterance detection method using the display device 100 and the virtual object display control device 200 will be described with reference to the flowchart of FIG. Steps S501 to S503 in FIG. 5 are processing on the display device 100 side, and steps S551 to S554 are processing on the virtual object display control device 200 side. In the flowchart of FIG. 5, the processing of the display device 100 and the virtual object display control device 200 is collectively shown for ease of explanation, but each processing is performed independently. The process of the flowchart shown in FIG. 5 is repeatedly performed at every predetermined timing.

  First, processing on the display device 100 side will be described. Steps S501 to S504 in FIG. 5 are executed under the control of the CPU 102 of the display device 100. In step S501, the utterance detection unit 103 of the display device 100 determines whether there is an utterance. Specifically, it is determined that there is an utterance when the volume of the voice data collected by the utterance detection unit 103 exceeds a certain level. When it is determined that there is an utterance, in step S502, the CPU 102 sets the utterance status of the display device managed in the RAM 107 as utterance. In step S503, the CPU 102 performs Fourier transform on the audio data. Then, time information at that time is added to the Fourier-transformed audio data.

  On the other hand, when the speech detection unit determines in step S501 that there is no speech, the process proceeds to step S504. In step S <b> 504, the CPU 102 sets the utterance status of the display device managed in the RAM 107 as “no utterance”.

  In step S <b> 505, the CPU 201 transmits utterance information including the utterance status, time information, and the user ID of the display device 100 to the virtual object display control device 200 via the network controller 109. If it is determined in step S501 that there is an utterance, the utterance information including the voice data converted in step S503 is transmitted to the virtual object display control apparatus 200.

  Next, processing on the virtual object display control apparatus 200 side will be described using steps S551 to S554 in FIG. The flowchart in FIG. 5 is executed by the CPU 201 of the virtual object display control apparatus 200. The CPU 201 reads the utterance management table 600 from the large capacity storage unit 204 to the RAM 203 in advance.

  In step S <b> 551, the virtual object display control apparatus 200 receives the utterance information transmitted from the display apparatus 100 via the network controller 207. The received utterance information is stored in the RAM 203 by the CPU 201.

  In step S552, the CPU 201 determines whether or not the user ID included in the received utterance information is in the utterance management table 600 managed by the virtual object display control device. If the user ID is not in the utterance management table 600, the CPU 201 newly registers the user ID, time information, and utterance status included in the utterance information in the utterance management table 600 in step S553. When the user ID is in the utterance management table 600, in step S554, the CPU 201 updates the time information associated with the user ID and the utterance status with the time information and utterance status included in the received utterance information.

  Note that the processing of the speech detection method of FIG. 5 may be performed simultaneously with the self-position direction acquisition processing shown in FIG. That is, the display device 100 may be a process of transmitting the utterance detection result and the self-position direction together to the virtual object display control device 200.

  FIG. 6 is a diagram showing an example of the utterance management table 600 managed by the virtual object display control apparatus 200. The utterance management table 600 includes the user ID, time information, and utterance status of each user.

  Next, the conversation status detection process will be described with reference to FIG. 7 showing the planar arrangement of each user from the self-position direction management table of FIG. 4, the utterance management table of FIG. 6, and the flowchart of FIG. I do.

  In FIG. 7, it can be seen that the user ID 0001 and the user ID 0004 substantially face each other. Further, from FIG. 6, the user ID 0001 and the user ID 0004 have an utterance status of utterance. From the above points, the user ID 0001 and the user ID 0004 may have a conversation. Here, the process for determining whether or not a conversation is in progress will be described with reference to the flowchart of FIG.

  The flowchart of FIG. 8 is executed by the conversation state detection unit 208 according to the control of the CPU 201 of the virtual object display control device 200. The process shown in the flowchart of FIG. 8 is repeatedly performed at predetermined timings. First, in step S <b> 801, the conversation status detection unit 208 extracts a user ID having an utterance status from the utterance management table 600. In the example of FIG. 6, user ID 0001 and user ID 0004 are extracted. Next, in step S802, the conversation state detection unit 208 determines whether there are two or more user IDs with speech. If there is only one, it is considered that the conversation does not hold, and the process returns to step S801.

  If there are two or more user IDs with utterances, in step S803, the conversation state detection unit 208 selects two arbitrary user IDs from the user IDs with utterances extracted in step S801. Then, the conversation state detection unit 208 extracts the Fourier-transformed voice data transmitted from the display device in association with the selected user ID. Then, the conversation state detection unit 208 compares the frequency information of the extracted Fourier transformed voice data to obtain the similarity. As a method for calculating the similarity, a known method similar to the method for obtaining the similarity of the one-dimensional data after Fourier transform is used.

  In step S804, the conversation status detection unit 208 proceeds to step S805 if the degree of similarity is equal to or greater than the threshold value as a result of the comparison in step S803. In step S805, since the voice data identified by the two user IDs are similar, the conversation status detection unit 208 determines that the users identified by the two user IDs are having a conversation. On the other hand, if the result of comparison in step S803 is less than the threshold value, the conversation status detection unit 208 proceeds to step S806. In step S806, since the voice data identified by the two user IDs is not similar, the conversation status detection unit 208 determines that the users identified by the two user IDs are not in conversation. In step S807, when the conversation status detection unit 208 compares the voice data with all combinations of user IDs with speech, the process ends.

  Next, a method for determining whether or not a conversation is performed based on the similarity of audio data will be described. Here, it is assumed that user A and user B are having a conversation. The fact that the user A and the user B are having a conversation is considered that the user A and the user B are located close to each other. The fact that the user A and the user B are located close to each other is detected by the speech detection unit of the display device of the user A as well as the speech detection unit of the display device of the user B. Will be. Whether or not the user A and the user B have a conversation by comparing the voice data from the display device of the user A and the voice data from the utterance detection unit of the user B display device by using this fact. Can be determined. It should be noted that the method for determining whether or not a conversation is made based on the similarity of voice data is not the main point of the present embodiment, but may be determined using various other known methods. In the above example, the example in which it is determined whether or not the conversation is performed based on the similarity of the voice data has been described. Further, as described above, whether or not the user A and the user B are conversing may be determined depending on whether or not the user A and the user B are facing each other. For example, before step S805 in FIG. 8, a step of determining whether or not the users are facing each other is included, and if they are facing, the process proceeds to step S805 to determine that the two users are in conversation. Also good.

  FIG. 9 is a diagram showing a conversation management table 900 that stores the result of the conversation state detection process managed by the virtual object display control apparatus 200. The conversation management table includes a conversation ID of each conversation, time information, and user IDs of conversation participants. The conversation ID is assigned to each conversation determined to be in conversation in step S805 in FIG. The method of assigning a conversation ID when three or more people are having a conversation is as follows. First, when it is determined that the user A and the user B are having a conversation, a new conversation ID is assigned, and the IDs of the user A and the user B are managed in association with the conversation ID as conversation participant IDs. The If it is determined in the next comparison that the user A and the user C are talking, the user ID of the user C is added to the conversation participant ID because the user A already has a conversation ID. Will be.

  In this embodiment, a combination that is once in a conversation is in a conversation even if there is no utterance information at a certain timing thereafter. As described above, the self-position direction and the utterance information are repeatedly transmitted to the virtual object display control device at every predetermined timing. Therefore, when only the user A is speaking at a certain timing, the conversation information of the user B may not be transmitted. Therefore, in the present embodiment, the condition that the state changes when the conversation is not being performed is that the distance between the users is more than a certain distance (for example, 3 meters) compared to the distance when it is determined that the conversation is being performed. And The CPU 201 of the virtual object display control device 200 performs the determination that the distance between the users is separated based on the self-position direction from the display device of each user. When the CPU 201 changes the state from a conversation to a conversation not being performed, the CPU 201 deletes the user ID of the corresponding user from the conversation participant ID associated with the conversation ID of the corresponding user from the conversation management table 900. . As a result of deletion, when the conversation participant ID becomes one, the conversation ID is also deleted. Further, the CPU 201 changes the utterance status of the corresponding user in the utterance management table 600 from “present” to “none”.

  Next, the virtual object deletion condition setting process will be described. In this embodiment, the virtual object display control device determines whether or not to delete the virtual object according to the distance between the virtual object and the user. That is, it is determined whether or not to erase the virtual object displayed on each display device used by each user. The virtual object is displayed on the display device by using a virtual tag associated with an arbitrary position in the real space by the virtual object display control device. Therefore, the virtual object displayed on the display device can be erased by eliminating this association in the virtual object display control device. In this embodiment, whether or not to delete the virtual object is determined by the distance between the virtual object and the user. Here, the user is a user who has arranged and displayed a virtual object when not having a conversation, and a user who has a conversation with a user who has arranged and displayed a virtual object when having a conversation. . In a conversation, it is assumed that a user who has placed and displayed a virtual object displays the virtual object to show it to the conversation partner user. In such a situation, when the user who placed the virtual object is in the middle, it is fully conceivable that the remaining user wants to continue browsing the virtual object. Therefore, in the case of a conversation, whether to delete the virtual object is determined according to the user who is speaking with the user who has placed the virtual object and the distance of the virtual object.

  Since a technique for arranging a virtual object by associating the virtual object with a predetermined position in the real space and a technique for displaying the arranged virtual object on a user's display device are well known, the description thereof will be omitted.

  Next, for example, consider a case where virtual objects are arranged as shown in FIG. The position of each user is assumed to be as shown in the self-position / direction management table 400 of FIG. Further, it is assumed that the user identified by the user ID 0004 arranges and displays the virtual object ID 0001, and the user identified by the user ID 0002 arranges and displays the virtual object ID 0002. The erasing condition setting process will be described with reference to the flowchart of FIG. The flowchart of FIG. 11 is mainly executed by the erasing condition setting unit 209 under the control of the CPU 201.

  In step S <b> 1101, the virtual object display control apparatus 200 receives the virtual object display notification transmitted from the display apparatus 100 via the network controller 207. The virtual object display notification is a notification that a virtual object is arranged and displayed on each display device. That is, it is a notification informing the virtual object display control device that the user of the display device has associated the virtual object with an arbitrary position in the real space. The virtual object display notification includes a position where the virtual object is displayed (pasted), a virtual object ID that identifies the virtual object, and a notification source user ID. The virtual object display notification may include time information, data of the virtual object itself, and data such as the orientation of the virtual object. Note that the position and orientation in which the virtual object is displayed specify the position and orientation in which the virtual object is pasted from spatial feature points, marker images, and the like, as in the processing for obtaining the position of the display device. be able to. The received virtual object display notification is stored in the RAM 203 by the CPU 201.

  Next, in step S1102, the erasure condition setting unit 209 determines whether the notification source of the virtual object display notification received in step S1101 is registered in the conversation participant ID of the conversation management table 900. If it is determined that it is registered, the erasure condition setting unit 209 determines that the virtual object notification source user is in conversation and advances the process to step S1103. The fact that the user who is notified of the virtual object display notification is in a conversation is assumed that the virtual object is arranged and displayed for sharing and viewing with the conversation partner. That is, it can be considered that it is not necessary to display the virtual object when there is no conversation partner. Accordingly, in step S1103, the erasure condition setting unit 209 sets the conversation partner user ID in the erasure condition.

  On the other hand, when it is determined in step S1102 that the user is not registered, the erasure condition setting unit 209 determines that the virtual object notification source user is not in conversation and advances the process to step S1104. The fact that the user who is the notification source of the virtual object display notification is not in a conversation is assumed that the display object has arranged and displayed the virtual object so that it can be seen by itself. That is, it can be considered that it is not necessary to display the virtual object when there is no display person. Accordingly, in step S1104, the erasure condition setting unit 209 sets the display user ID in the erasure condition.

  In step S1105, the erasure condition setting unit 209 calculates an erasure distance indicating how far the user set in the erasure condition is away from the virtual object. In this embodiment, a distance that is further a predetermined distance (for example, 3 meters) away from the distance between the virtual object and the user set in the erasing condition when the virtual object is displayed is set as the erasing distance. That is, the erasing condition setting unit 209. The position of the notification source user ID or the conversation partner user ID that has the same or recent time information as the virtual object display notification and is the deletion condition is acquired from the self-position direction management table 400 and deleted. Set the distance. Thereafter, the erasure unit 210 compares the position of the user ID, which is an erasure condition in the self-position / azimuth management table 400, with the position of the virtual object, and erases the virtual object when the erasure distance is reached.

FIG. 12 is a diagram showing an erase condition table 1200 as a result set by the erase condition setting unit 209. The deletion condition table 1200 includes a virtual object ID, time information, a virtual object position, a virtual object orientation, a user ID arranged and displayed on the virtual object, a user ID set in the deletion condition, and an deletion distance. It is assumed that the virtual object position and the virtual object direction are known from information included in the virtual object display notification. In the example of FIG. 12, the erase distance of the virtual object ID 0001 is calculated as follows. First, the distance √ ((1506−3000) ² + (1120−520) ² + (1860-1500) ² ) ≈1650 millimeters between the virtual object ID 0001 and the user ID 0004 of the erasure condition is calculated. And 3 meters is added to the distance 1650 millimeters, and it becomes 4650 millimeters. That is, the virtual object ID 0001 is erased when the distance between the virtual object ID 0001 and the user ID 0004 in the erasing condition is more than 4650 millimeters.

  Although the present embodiment has been described above, the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the scope of the invention. For example, the virtual object can be deleted even when the condition set by the deletion condition setting unit is met. For example, the virtual object can be erased by the user explicitly notifying the CPU 102 of an erase instruction through the operation unit 104 of the display device 100. In addition, when a virtual object is displayed alone, it is erased when the viewer moves away from the virtual object by a certain distance, but if the viewer just sits away from the spot and wants to return immediately and see the virtual object You can also say that. Therefore, it is also possible to temporarily erase when the display person is away from the virtual object, redisplay when the display person returns within a predetermined time, and delete when the display person does not return.

  In the above example, the conversation partner is set as the user of the erasure condition when the conversation is in progress. However, in addition to the conversation partner, the display person who displayed the virtual object can also be set as the erasure condition. Good. That is, when a conversation is in progress, the virtual object may be deleted when both the display person who displayed the virtual object and the conversation partner are away from the virtual object by a certain distance.

(Example 2)
In the first embodiment, the virtual object deletion condition is set based on the detection result of the conversation state detection unit when the virtual object is displayed. However, the conversation situation can change from moment to moment. For example, two people started a conversation at first, but there are cases where the number increases to three and four. Therefore, in this embodiment, a process for resetting the erasure condition when the conversation state changes will be described. For example, as described in the first embodiment, FIG. 4 shows the self-position direction management table and FIG. 6 shows the utterance management table when the time is 12/12/2012 15:00:00. Further, FIG. 9 shows a conversation management table of detection results of the conversation status detection unit, and FIG. 12 shows an erase condition table of setting results of the erase condition setting unit. Consider a case where a user ID 0002 participates in a conversation between the user ID 0001 and the user ID 0004 after 10 minutes.

  FIG. 13A is a diagram showing the self-position direction management table 1300 at this time point. FIG. 13B is a diagram showing the conversation management table 1350. The processing of the erasure condition setting unit 209 in this case will be described using the flowchart of FIG. The flowchart of FIG. 14 is executed by the erasing condition setting unit 209 according to the control of the CPU 201.

  In step S1401, the CPU 201 determines whether there is a change in the conversation management table. If it is determined that there is a change, in step S1402, the erasure condition setting unit 209 identifies the user ID of the conversation participant associated with the conversation ID that has changed. Then, the erasure condition setting unit 209 selects a virtual object ID associated with the specified user ID from the erasure condition table. Referring to FIG. 13B, user IDs 0001, 0002, and 0004 are specified as conversation participant IDs associated with the conversation ID 0001 that has changed. Then, the virtual object ID associated with these user IDs is selected from the erasing condition table. More specifically, it is determined whether these user IDs match either the user ID of the user who placed and displayed the virtual object or the user ID of the user set in the deletion condition. Then, the virtual object ID associated with the matched user ID is selected. In the case of FIG. 12, the virtual object ID 0001 and the virtual object ID 0002 correspond. In the following, a user who arranges and displays a virtual object is referred to as a display user, and a user ID of the display user is referred to as a display user ID. Similarly, a user set as an erase condition is referred to as an erase condition user, and a user ID of the erase condition user is referred to as an erase condition user ID.

In step S1403, the deletion condition setting unit 209 determines whether the display user ID associated with the selected virtual object ID is different from the deletion condition user ID. The virtual object ID 0001 is different because the display user ID is 0001 and the deletion condition user ID is 0004. Accordingly, the erasure condition setting unit 209 advances the process to step S1404. In step S1404, the erasure condition setting unit 209 resets the erasure condition user. As described in the first embodiment, since the conversation partner with the display user ID 0001 is set as the deletion condition user, the deletion condition user ID is reset to 0002 and 0004. Next, in step S1405, the erasure condition setting unit 209 recalculates the erasure distance for each erasure condition user ID, and sets the farthest distance as the new erasure distance. 13A, the distance between the user ID 0002 and the virtual object 0001 is √ ((2516-3000) ² + (2311-520) ² + (1550-1500) ² ) ≈1856 millimeters. Since the distance between the user ID 0004 and the virtual object 0001 is 1650 millimeters, 3 meters is added to 1856 millimeters that are far away, resulting in 4856 millimeters. By setting a new erasure distance using a distance associated with a long-distance user ID, an appropriate erasure can be performed according to the conversation scale. FIG. 15 shows an erase condition table in which the erase condition of the virtual object 0001 is changed.

  Note that regarding the virtual object ID 0002 displayed by the user identified by the user ID 0002, since it is determined in step S1403 that the display user and the erasure condition user match, the resetting in step S1404 is not performed. Even if the user identified by the user ID 0002 moves, the virtual object 0002 remains displayed on the display device of the user ID 0002 because the erase distance is not reached.

  According to the second embodiment described above, it is possible to appropriately set the erasing condition even if the conversation situation changes due to another user participating in the conversation during the conversation.

Example 3
Access rights may be set for virtual objects to be displayed. The access right includes a read that can be displayed only by placing a virtual object, and a read / write that can be displayed and edited. Furthermore, there is a Look that only browses virtual objects that other users are reading. For example, it is assumed that a person belonging to a certain department X has a Read right for the virtual object O. In this case, a person who belongs to the department X can arrange and display the virtual object O. On the other hand, it is assumed that persons belonging to other departments Y do not have Read authority. In this case, a person who belongs to the department Y cannot arrange or display the virtual object O. Even under such conditions, a person in department X may have a conversation with another person in department Y while viewing virtual object O. In that case, the Look authority for the virtual object O is given to the person in the department Y. In this case, the virtual object O can be arranged and displayed by the person in the department X, and the person in the department Y can only browse the virtual object O arranged and displayed by the person in the department X. Under such circumstances, it is inconvenient for security for the person in department X to keep the virtual object O displayed when there is only a person in department Y who does not have Read authority. This is because only a person with the Look authority is not allowed to display (that is, read) the virtual object O in the first place. Therefore, when the person in the department X is away from the virtual object O by a certain distance and only the person in the department Y is around the virtual object O, it is preferable to delete the virtual object O. In the third embodiment, an erasing process in such a case will be described. The authority of the virtual object is assumed to be stored in association with the virtual object data managed by the virtual object display control apparatus 200.

  FIG. 16A shows a self-position direction management table when the time is December 13, 2012, 13:00:00. In FIG. 16A, four user IDs 0001 to 0004 are registered. In addition, FIG. 16B shows a conversation management table at the same time. As shown in FIG. 16 (b), at that time, four users with user IDs 0001 to 0004 are in conversation. FIG. 17 is a diagram showing a planar layout when the user ID 0001 displays a virtual object in the state shown in FIG. In addition, it is assumed that the user IDs 0001 and 0002 have the virtual object Read authority. Further, it is assumed that the user IDs 0003 and 0004 do not have the Read authority for the virtual object but have only the Look authority.

  FIG. 18 is a flowchart of the erasure condition setting process in the third embodiment. The flowchart in FIG. 18 is mainly executed by the erasing condition setting unit 209 according to the control of the CPU 201. The flowchart in FIG. 18 is substantially the same as the flowchart in FIG. 11, and thus the same processes are denoted by the same reference numerals and description thereof is omitted. When the erasure condition setting unit 209 determines in step S1102 in FIG. 18 that the user is in conversation, the process proceeds to step S1803. In step S1803, the erasure condition setting unit 209 sets the display person who displayed the virtual object and the conversation partner having the Read authority as the erasure condition user. In step S1803, in addition to the conversation partner having the Read authority, the display person who displayed the virtual object is included in the erasure condition. This is to cope with a case where a person with Read authority displays a virtual object but all conversation partners do not have Read authority. That is, in order to avoid the virtual object not being deleted when the display person with the Read authority leaves the seat and only the person without the Read authority remains.

  FIG. 19 is a diagram showing an erasing condition table created as a result of performing the processing of the flowchart of FIG. 18 in the situation shown in FIG. In FIG. 19, a user ID 0001 that is a display person is set for the deletion condition user, in addition to the user ID 0002 that has the read authority for the virtual object at the conversation partner.

  As described above, the erasure condition for the virtual object for which the access right is set can be appropriately set.

  Further, the present invention is not limited to the above-described example, and various modifications can be made without departing from the gist. For example, it is possible for the display person to sit in the middle of a conversation participant when the only user who has the Read permission to the virtual object is the display person. In that case, it is possible to temporarily give Read permission to the conversation participant before the display person is in the middle so that the virtual object can be seen even while the display person is in the middle. At this time, since the Read authority for the conversation participant is temporary, the Read authority may be lost when the display person returns or when a certain time elapses.

<Other examples>
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads the program. It is a process to be executed.

Claims (18)

Receiving means for receiving a display notification of a virtual object;
Acquisition means for acquiring a conversation status of a notification source user of the display notification;
Determining means for determining an erasure condition user to be an erasure condition of the virtual object based on the acquired conversation state;
A virtual object display control device comprising: erase means for erasing the virtual object according to the position of the erase condition user and the position of the virtual object.   The virtual object display control apparatus according to claim 1, wherein, when the conversation status indicates that there is a conversation, the determination unit determines the conversation partner user of the notification source user as an erasure condition user.   2. The virtual object according to claim 1, wherein when the conversation status indicates that there is a conversation, the determination unit determines the notification source user and the conversation partner user of the notification source user as deletion condition users. Display control device.   4. The virtual object display control device according to claim 1, wherein when the conversation status indicates no conversation, the determination unit determines the notification source user as the erasure condition user. 5. .   5. The virtual object display control device according to claim 1, wherein the determination unit resets an erasure condition user according to a change in a conversation situation of a user who participates in a conversation. 6.   6. The virtual object display control device according to claim 1, wherein the erasing unit erases the virtual object when the erasing condition user moves away from the virtual object by a certain distance. 6. .   7. The virtual according to claim 1, wherein the acquisition unit acquires the conversation state based on a similarity of audio data transmitted from each display device that displays a virtual object. 8. Object display control device.   The virtual object display control device according to claim 7, wherein the acquisition unit further acquires the conversation state according to a position and an orientation of each display device. A management means for managing the conversation status;
9. The virtual object display according to claim 1, wherein the management unit changes the conversation state from conversation to no conversation according to a distance between users having conversation. Control device. Re-displaying means for re-displaying the virtual object erased by the erasing means;
The re-display means performs the re-display when the erasing condition user returns to the position where the virtual object was displayed within a predetermined time from the time when the re-erasing means erased. The virtual object display control device according to any one of 1 to 9. The virtual object is given Read authority, which is authority to arrange and display the virtual object,
The virtual object display control apparatus according to claim 1, wherein the determination unit determines the erasure condition user further based on the Read authority.   When the conversation status indicates that there is a conversation, the determining means sets the user who is the conversation partner of the notification source user and the read source user of the virtual object, and the notification source user as the deletion condition user. The virtual object display control device according to claim 11, wherein the virtual object display control device is determined.   13. The granting means for temporarily granting Read authority to a conversation partner user of the notification source user when the erasure condition user determined by the determination means is only the notification source user. The virtual object display control device according to 1.   The virtual device according to any one of claims 1 to 13, wherein the erasing unit erases the virtual object by deleting the association of the virtual object associated with a predetermined position in the real space. Object display control device. Imaging means;
Transmitting means for transmitting the position information of the user and the collected voice data;
A display device having display means for displaying image data including captured image data captured by the imaging means and a virtual object superimposed on the captured image data;
The virtual object displayed by the display means is erased by the virtual object display control device according to any one of claims 1 to 14 in response to transmission of the position information and the audio data. Display device. A receiving step for receiving a display notification of the virtual object;
An acquisition step of acquiring a conversation state of a notification source user of the display notification;
A determination step of determining an erasure condition user as an erasure condition of the virtual object based on the acquired conversation state;
A virtual object display control method comprising: an erasing step of erasing the virtual object according to the position of the erasing condition user and the position of the virtual object. Imaging step;
A transmission step of transmitting the self location information and the collected voice data;
A display method including a display step of displaying image data including the captured image data captured in the imaging step and a virtual object superimposed on the captured image data,
The display method according to claim 16, wherein the virtual object displayed in the display step is erased by the virtual object display control method according to claim 16 in response to transmission of the position information and the audio data.   A program for causing a computer to function as the virtual object display control device according to claim 1.

Images