JP5050919B2 - Remote conference system and control method thereof - Google Patents

Description

  The present invention relates to a remote conference system that draws a virtual object corresponding to a real space in a virtual space, and a control method thereof.

  As a conventional remote conference system, for example, “In each user station 2-14, an image of a three-dimensional computer model including an avatar representing an attendee is displayed. The image data of the user wearing the headphones 30 with the markers 70 and 72 and the lights 56 to 64 is processed to determine the user's movement and the point in the display image being watched. It is transmitted to the user station, and the movement of the head of the avatar is animated so as to correspond to the converted movement of the head of the user. ”(For example, see Patent Document 1).

JP 2000-244886 A

  2. Description of the Related Art In recent years, with the development of information communication networks, remote conference systems have been developed that allow people in remote locations to participate in conferences using communication technology.

Among such remote conference systems, for example, as in Patent Document 1 above, a conference participant existing in a remote location communicates information via a network to a virtual space managed by a server. You can operate your own character that exists as an avatar, and share the conference space in the virtual space with the participant's avatar to enhance the realism of the remote conference or occur in a regular video-only conference Systems have been developed that can avoid problems such as the inability to share common gazes.
Here, an avatar is a model that represents a user who exists in a virtual space and operates by physical movements of conference participants (hereinafter also referred to as “users”) or by inputting instructions to the apparatus. Avatars are often represented by a three-dimensional model.

When using a system such as the one above to conduct a remote conference between remote locations, prepare a room or facility that is somewhat similar to the real space, and create a virtual space that resembles the room or facility. Then, it is considered that by accessing the virtual space, it is possible to obtain a conference environment similar to the situation in which the user participates in the conference and actually gathers in the conference room and performs the conference.
Furthermore, it is considered that a more similar conference environment can be obtained by associating an object existing in the real space as a virtual object in the virtual space.

  However, when realizing the above configuration, the virtual object moved in the virtual space and the position and orientation (orientation) of the object actually existing in the real space are different (hereinafter also referred to as “contradiction”). In some cases, there was a problem that it felt uncomfortable.

  As a method that does not cause this contradiction, for example, a system is configured so that data (virtual object) in virtual space is completely determined according to real space, an object is actually moved in real space, and only by the sensing result, A method of moving the data in the virtual space can be considered.

  However, with such a configuration, for example, an operation unique to a system using a virtual space such as moving an object on the virtual space by a simple operation by a mouse operation on a client (PC) cannot be used. was there.

  Therefore, a teleconference system and a control method therefor have been desired that can reduce the difference in position and orientation (orientation) between a virtual object moved in the virtual space and an object existing in the real space.

  In the remote conference system according to the present invention, a plurality of clients are connected to a server via a communication line, and a virtual object corresponding to an arbitrary object in the real space is drawn in a virtual space defined by the server, A teleconferencing system that displays on each client and changes the drawing position or direction of the virtual object according to operation information from the client, wherein the real space information relates to the relative position and / or relative direction between at least two or more of the objects And an object moving means for moving at least one of the objects based on an operation command from the server, the server based on the real space information, the relative position between the objects And / or the difference between the relative direction and the relative position and / or relative direction between the virtual objects corresponding to the object. A contradiction detection unit that detects whether or not the difference amount is greater than or equal to a predetermined amount; and a control unit that obtains the operation command to move the object so that the difference amount is less than the predetermined amount when the difference amount is equal to or greater than the predetermined amount; It is what has.

  In the remote conference system control method according to the present invention, a plurality of clients are connected to a server via a communication line, and a virtual object corresponding to an arbitrary object in the real space is drawn in the virtual space defined by the server. Then, a remote conference system control method for displaying on each client and changing the drawing position or direction of the virtual object according to operation information from the client, the relative position between at least two or more objects and / or Detecting real space information regarding a relative direction; based on the real space information; a relative position and / or relative direction between the objects; and a relative position and / or relative direction between the virtual objects corresponding to the object; Detecting whether or not the difference amount is greater than or equal to a predetermined amount, and when the difference amount is greater than or equal to the predetermined amount, And determining the operation command for moving the object to be less than the amount, the object moving means for moving at least one of the object, in which a step of operating on the basis of the operation command.

  In the present invention, when the amount of difference between the relative position and / or relative direction between objects and the relative position and / or relative direction between the virtual objects corresponding to the object is equal to or greater than a predetermined amount, the difference amount is a predetermined amount. Since the object is moved so as to be less than the difference, the difference in position and orientation between the virtual object in the virtual space and the object existing in the real space can be reduced.

Embodiment 1 FIG.
The remote conference system according to the first embodiment relates to a remote conference system that performs a conference using a virtual space constructed by using a computer via a network, and conference materials and microphones that exist simultaneously in the virtual space and the real space. Such an object is synchronized with the movement in the virtual space, and is also operated in the real space using a robot. Then, using a virtual space having a virtual object in common with an object existing in the real space, it is possible to detect a contradiction between the real space and the virtual space, and move at least one of the objects existing in the real space. An object moving means capable of detecting that the object moving means cannot move the object, notifying the user that a contradiction has occurred, and providing the user with information for resolving the contradiction of the object. It can be notified.

  FIG. 1 is a configuration diagram of a remote conference system according to the first embodiment. In FIG. 1, the remote conference system includes a network 101, a virtual space providing server 102, a client 103, and a robot 104. In this remote conference system, a client 103 is connected to a virtual space providing server 102 via a network 101, and corresponds to an arbitrary object 1019 in real space or a user in a virtual space defined by the virtual space providing server 102. An object or an avatar (hereinafter collectively referred to as “virtual object”) is drawn and displayed on the client 103, and the drawing position or direction of the virtual object is changed by operation information from the client 103. Note that a configuration in which a plurality of clients 103, robots 104, and objects 1019 exist is common, but illustration thereof is omitted in FIG.

  The virtual space providing server 102 includes a virtual space data creation unit 105, a control unit 106, a data communication unit 107, a virtual space data 108, and a contradiction detection unit 109. Provide services.

  The virtual space data creation unit 105 creates and updates the virtual space data 108 in response to an instruction from the control unit 106.

  The control unit 106 communicates with each component in the virtual space providing server 102 and transmits a command to each unit. The control unit 106 communicates with the virtual space data creation unit 105 to instruct creation and update of the virtual space data 108, communicates with the data communication unit 107, transmits the virtual space data 108 to the client 103, The virtual space data 108 is instructed to be updated from various data received from the robot 103 or the robot 104. In addition, the control unit 106 performs real-time data on the position and orientation (orientation) of the object 1019 acquired from the robot 104 and data on the position and orientation in the virtual space, and the object 1019 in the real space. An operation command to the robot 104, which is necessary to correspond to the virtual object in the virtual space, is created.

  The data communication unit 107 communicates with the robot 104 and the client 103 via the network 101 to transmit / receive data. Data transmitted / received from the data communication unit 107 includes a plurality of types of data. For example, the data transmitted from the client 103 and transmitted to the other client 103 via the virtual space providing server 102 includes, in addition to the virtual space data 108, audio and video streaming necessary for configuring the conference. Data etc. are included.

  The virtual space data 108 includes information on the position, size, and outer shape of the elements constituting the virtual space, information on the position, posture, and outer shape of each user's avatar that exists in the virtual space, and each object that exists in the virtual space. 1019 includes information on the position, posture, and outer shape. The virtual space data 108 is data necessary for the client 103 to display the virtual space and for the robot 104 to know the position of the object 1019 placed in the virtual space. The actual data format varies depending on the 3DCG (3 Dimensional Computer Graphics) technology to be used, but an existing general technology can be used.

  The contradiction detection unit 109 compares the real space information of the object 1019 transmitted from the robot 104 and the virtual object information expressed in the virtual space, and detects the contradiction between the real space and the virtual space from the result. To do.

  The object 1019 is an element that exists in the real space, such as a desk or chair installed in a conference room for performing a remote conference, a microphone or a camera of the client 103, and also exists as data in the virtual space. is there.

  The client 103 includes a data communication unit 1010, a control unit 1011, an information input unit 1012, and an information presentation unit 1013. The client 103 communicates with the virtual space providing server 102 in accordance with a user instruction or automatically, and provides a remote conference environment using the virtual space to the user. The user uses this client 103 to access the virtual space providing server 102 to operate an avatar or an object in the virtual space, or transmit voice or video.

  The data communication unit 1010 communicates with the virtual space providing server 102 and transmits / receives various data described below.

  The control unit 1011 communicates with the data communication unit 1010, the information input unit 1012, and the information presentation unit 1013, receives user input from the information input unit 1012, and updates virtual space data (described later) to the data communication unit 1010. ) To the virtual space providing server 102. In addition, virtual space display data (described later) received via the data communication unit 1010 is transmitted to the information presentation unit 1013 to present information.

  The information input unit 1012 includes, for example, a device that accepts user input such as a keyboard and a mouse, and a device that can constantly measure a user's physical motion such as motion capture and use the result as input. The information input unit 1012 receives information input from a user who uses the client 103 and operates, for example, an avatar in the virtual space. The information input unit 1012 has a microphone, a camera, or the like that acquires the user's voice or video as elements, and transmits the user's video or voice in a format that can be sent to another client via the virtual space providing server 102.

  The information presentation unit 1013 includes, for example, a display and a speaker, and presents virtual space display data (described later) received from the virtual space providing server 102, audio data, video data, and the like. Further, a configuration in which the user wears and uses the head mounted display is also possible.

FIG. 2 is an external view of the robot according to the first embodiment.
As shown in FIG. 1, the robot 104 includes at least a data communication unit 1014, a control unit 1015, a sensor unit 1016, an information presentation unit 1017, and a drive unit 1018. This robot 104 has the form shown in FIG. 2, for example, and moves the object 1019 existing in the real space or presents information prompting the user to move the object 1019 by an operation described later. Thus, the contradiction between the position and orientation of the object 1019 in the real space and the virtual object in the virtual space is reduced. Note that the robot 104 according to the first embodiment can travel on a desk or a floor and move at least one object 1019 existing in the real space by, for example, “pressing” it.

  The control unit 1015 and the sensor unit 1016 correspond to the object detection unit in the present invention, the information presentation unit 1017 corresponds to the information presentation unit in the present invention, and the control unit 1015 and the drive unit 1018 serve as the object movement unit in the present invention. Equivalent to.

  The data communication unit 1014 communicates with the virtual space providing server 102 via the network 101 to transmit / receive various data.

  The control unit 1015 receives the operation command transmitted from the virtual space providing server 102 via the data communication unit 1014 and transmits the operation command to the drive unit 1018. When the real space data transmission command is received, the information about the position and orientation of the object 1019 acquired by the sensor unit 1016 is processed and transmitted.

  The sensor unit 1016 has a function of receiving the command from the control unit 1015 and acquiring the position and orientation (orientation) of the object 1019 existing in the real space. The sensor unit 1016 can be mounted using, for example, an infrared sensor, a laser range finder, or an image measurement device.

  In response to an instruction from the control unit 1015, the information presenting unit 1017 receives information indicating that the position and orientation of the object 1019 do not match between the virtual space and the real space and a contradiction occurs, and to resolve the contradiction. In addition, it is shown to the user that the user should correct the position and orientation of the object 1019.

  The drive unit 1018 is composed of, for example, a wheel or a foot-like mechanism driven by a motor, and operates according to an operation command from the control unit 1015. The robot 104 is moved in the real space, and the object 1019 in the real space is moved by the movement.

  Next, an example of data that is transmitted and received between the virtual space providing server 102 and the client 103 when a remote conference is performed at remote locations using the remote conference system according to the first embodiment. This will be described with reference to FIG.

  FIG. 3 is a diagram showing a structure of transmission / reception data according to the first embodiment. FIG. 3A shows virtual space display data, which is data transmitted from the virtual space providing server 102 and necessary for displaying the virtual space by the client 103. FIG. 3 (b) shows audio data, and FIG. 3 (c) shows video data, which are transmitted and received between each client 103 via the virtual space providing server 102, and the user's voice and video are remotely located. This is data to provide to the user. FIG. 3D shows virtual space data update data, which is transmitted from the client 103 and required to update the virtual space data 108 by the virtual space providing server 102. This data includes various kinds of change data necessary for changing the position, posture, facial expression, outline, etc. of the avatar operated by the user on each client 103.

  Here, the ObjectID is an ID for identifying an object 1019 (object) existing in the virtual space. The outer walls that make up the virtual space can also be handled as a kind of object. Moreover, AvatarID is ID of each avatar. The UserID is an ID for uniquely identifying a user who operates the client 103. The operation ID is an ID indicating an operation that can be performed on an object in the virtual space associated with the ObjectID.

  The avatar or object in the virtual space can be operated by the virtual space data update data. For example, when an object is operated, the state (position and orientation) of the object in the virtual space can be changed by transmitting the operation ID to the ObjectID. As a specific example, when the user (UserID = userA) performs an “open” (operation ID = open) operation on a cabinet (ObjectID = cabinet1) existing in the virtual conference room, the virtual space providing server 102 On the other hand, virtual space data update data as shown in FIG. 3E is transmitted, the virtual space data is updated by the virtual space providing server 102 that has received the data, and the cabinet is opened in the virtual space.

  4 is an operation explanatory diagram of the remote conference system according to the first embodiment, FIG. 5 is an operation flowchart according to the first embodiment, FIG. 6 is a diagram illustrating a method for generating an operation command according to the first embodiment, and FIG. It is a figure which shows the case where the robot which concerns on Embodiment 1 cannot move an object. Hereinafter, the operation in the first embodiment will be described with reference to the drawings.

First, an outline of the operation will be described with reference to FIG.
As shown in FIG. 4A, by transmitting and receiving the above-described audio data and virtual space display data between the virtual space providing server 102 and the client 103, for example, the user A, which exists in the real space point 1, B and the users C and D existing in the real space point 2 are notified to the information presenting unit 1013 of the client 103 as if they existed as the avatar A′B′C′D ′ in the shared virtual space V. Is presented. With such a display, each user can communicate with a user at a distant location in real space with a sense of reality. The operation of generating and displaying such a virtual space has the same configuration as that of a general virtual space providing device, and thus detailed description thereof is omitted.

Next, an outline of the operation for eliminating the contradiction that occurs between the virtual space and the real space will be described.
In FIG. 4, microphones for inputting the user's voice are installed on the desks at the real space points 1 and 2 as one of the information input units 1012. Here, the microphone is one of the objects 1019 and also exists as a virtual object in the shared virtual space V.

  As shown in FIG. 4A, for example, it is assumed that the user C operates the client 103 and changes the position of the microphone on the shared virtual space V to the same state as that on the desk in the real space 2. In this case, the positions of the microphones in the shared virtual space V and the real space point 2 match. However, at the real space point 1, there is a contradiction between the position of the microphone in the real space and the position in the virtual space. That is, on the shared virtual space V, the microphone is disposed in front of the user A ′ on the desk, whereas at the real space point 1, the microphone is disposed on the left front of the user B on the desk. The relative position between the desk and the user and the microphone at the point 1 does not correspond to the relative position between the desk and the user and the microphone in the virtual space. For example, when the microphone attitude (orientation) in the virtual space is changed, a contradiction similarly occurs between the real space and the virtual space.

  The teleconference system detects this state and operates the robot 104 existing at the real space point 1 to reduce the contradiction. As an operation for reducing the contradiction, when the robot 104 can operate the microphone, as shown in FIG. 4B, the robot 104 moves to the position that matches the shared virtual space V by moving on the desk and pressing the microphone. . In addition, when the microphone 104 is too heavy to be operated by the robot 104 or when the microphone 104 cannot be operated depending on the position of the microphone, there is a microphone in the real space as shown in FIG. The power position is projected onto the real space using the information presentation unit 1017 of the robot 104, and the user is prompted to change the position of the microphone. As a result, the user receives information from the robot 104 and changes the position of the microphone in the real space. Such an operation reduces the contradiction between the real space and the virtual space.

  Details of such an operation will be described with reference to FIGS. 6 and 7 based on FIG. In FIG. 5, the left side shows the operation of the virtual space providing server 102, and the right side shows the operations of the client 103 and the robot 104.

  When the user performs an operation such as moving an object in the virtual space using the information input unit 1012 such as a mouse in step S3001, the control unit 1011 of the client 103 creates virtual space data update data, and in step S3002 Transmit to the virtual space providing server 102.

  In step S <b> 301, the virtual space providing server 102 receives virtual space data update data from the client 103. In step S302, the virtual space data creation unit 105 changes the virtual space data 108 according to the virtual space data update data. Next, the virtual space providing server 102 transmits a real space data transmission command to the robot 104 in step S303.

  In step S304, the robot 104 receives a real space data transmission command. In step S305, the sensor unit 1016 measures the position and orientation of an object 1019 in real space whose position and orientation (orientation) are to be measured, for example, a microphone. In step S <b> 306, the robot 104 transmits real space data regarding the measured position and orientation of the object 1019 to the virtual space providing server 102.

  In step S307, the virtual space providing server 102 receives the real space data, and in step S308, compares the data of the objects in the real space and the virtual space, and calculates the distance. In step S309, the contradiction detection unit 109 determines whether the distance value is a certain value or more.

  As an example of the measurement operation of the robot 104 and the comparison operation of the virtual space providing server 102, for example, the position data of the object 1019 and the virtual object are represented by planar coordinates, and the Euclidean distance between the two positions of the virtual space and the real space is expressed as follows. When calculating, the following operation is performed.

  The robot 104 defines two-dimensional or three-dimensional coordinates with an arbitrary position (for example, an edge) of the desk in the real space as an origin, and measures the origin position and a distance to an arbitrary measurement point on the microphone. The coordinates according to the measured distance are attached. Here, for the sake of simplicity, it is assumed that the movement is limited to a planar shape. The movable range of the robot 104 is limited to the desk. In this case, for example, as shown in FIG. 6, the robot 104 divides the plane on the desk on the grid using the measured origin position as a reference, and assigns coordinate values in the xy directions as shown. A coordinate value corresponding to the measured origin position and microphone distance is added as a coordinate value of the microphone position. For example, in the case of FIG. 6A, the microphone position is (x, y) = (2, 2). Then, the robot 104 transmits the coordinate information to the virtual space providing server 102 as real space data.

  The contradiction detecting unit 109 of the virtual space providing server 102 refers to the virtual space data 108 and defines the origin position in the virtual space, that is, the coordinates with the edge of the virtual object of the desk in the above example as the origin. In the same manner as described above, the plane on the desk is divided on the grid using the origin position as a reference, and coordinate values in the xy directions are assigned. And the coordinate in the measurement point of the microphone in virtual space is calculated | required.

Next, the contradiction detection unit 109 compares the microphone coordinates indicated by the real space data with the microphone coordinates in the virtual space to determine whether or not the difference in distance between the two is greater than or equal to a predetermined amount. If it is greater than or equal to the predetermined amount, it is determined that there is a contradiction between the real space and the virtual space. In the present embodiment, it is determined whether or not there is a contradiction depending on whether or not the coordinates are on the above-described grid. The coordinate interval is not limited to this, and finer coordinates may be added.
When measuring the posture (orientation) of the microphone, it is possible to detect a contradiction in posture by setting the above-described microphone measurement points to 2 or more and comparing the plurality of measurement points. For example, one point of the foot part where the microphone is in contact with the ground and one point of the windbreak part of the microphone are measured as measurement points, and the angle of a straight line connecting these two points and the angle of the object in the virtual space are calculated. Comparing postures can be detected by comparison.

  Next, when the control unit 106 of the virtual space providing server 102 determines that there is a contradiction in step S309, the control unit 106 performs an operation on the robot 104 from the two coordinate values in the real space and the virtual space in step S310. A command is created and transmitted to the robot 104 in step S311.

A method for creating this operation command will be described with reference to FIG.
Now, the microphone (information input unit 1012) that is the object to be moved by the robot 104 in FIG. 6A exists at the coordinate position (2, 2) in the real space, and the position of the microphone in the virtual space. That is, assume that the position to which the microphone should move is represented by (3, 2).

Considering coordinates as a vector and subtracting two vectors,
(3,2)-(2,2) = (1,0)
And a value of (1, 0) is obtained.

  This is an operation command for the robot 104 as it is. The robot 104 may move the microphone from the current position in a direction that increases by one in the x coordinate. Thereby, the operation command for moving the microphone so that the above-described distance difference amount is less than the predetermined amount can be obtained.

  When measuring the orientation of an object, for example, the angle between a straight line connecting two measurement points of the object described above and the coordinate axis is obtained with an accuracy of, for example, 45 °, and the real space is calculated from the angle in the virtual space. The rotation operation command can be obtained by subtracting the object angle at. For example, when the angle of the virtual object is 45 ° with respect to the x-axis and the angle between measurement points in the real space is 90 ° (parallel to the y-axis), the rotation operation command is −45 °.

  If the subtraction result is equal to or smaller than a certain value, for example, (0, 0), the process ends in step S312.

  In step S313, the robot 104 receives the operation command, operates the drive unit 1018, and tries to operate in step S314. After the operation is completed, in step S315, the position of the object is again measured by the sensor unit 1016, and in step S316, it is confirmed whether the object can be moved to the planned position. If it can be moved, the process ends in step S318. If it cannot move, the information presenting unit 1017 presents information regarding the position where the microphone (object) should be moved, that is, the microphone position where the contradiction is resolved, to the user in step S317. And promote the resolution of the contradiction. For example, using a projector provided as the information presentation unit 1017, a microphone image is projected at a position where the microphone should be moved in real space. The presentation of information is not limited to this, and information on a position where the microphone (object) should be moved may be presented by voice.

  Next, an example in which the robot 104 cannot move an object will be described with reference to FIG. As shown in FIG. 7, when the object (microphone) exists at the coordinate position (3, 1) and the position to be moved is (3, 2), the robot 104 does not have a function of drawing the object. The object cannot be moved to (3, 2) depending on the operation of the robot 104. In such a case, the robot 104 presents information indicating that the coordinate position (3, 2) is a place where the object is to be moved as information for prompting the resolution of the contradiction.

  In the above description, it is determined whether or not the robot 104 has moved in step S316. However, the present invention is not limited to this, and the virtual space providing server 102 transmits the operation command in step S311 and after a predetermined time has elapsed, again. Real space data may be acquired and it may be judged whether it was able to move at Step S316. Thereby, the configuration of the robot 104 can be simplified.

  Further, in the above description, the case where the information for moving the object is presented to the information presentation unit 1017 of the robot 104 has been described. However, the present invention is not limited to this, and the information may be presented to the information presentation unit 1013 of the client 103. good.

  In the first embodiment, the case where the robot 104 cannot move the object is presented with information for prompting the contradiction to be resolved. However, the present invention is not limited to this, and the contradiction detection unit 109 generates a contradiction. When this is detected, information indicating that there is a contradiction between the real space and the virtual space may be presented to the user.

  As described above, in the present embodiment, by detecting a contradiction in the arrangement of an object between the real space and the virtual space and moving the object using the robot 104 when the contradiction occurs. The difference in position and orientation between the virtual object in the virtual space and the object existing in the real space can be reduced.

  Even if the robot 104 cannot move the object to be moved, information that prompts the user to resolve the contradiction is presented to the user, and the user follows the information to the virtual object and the real space in the virtual space. Differences in position and orientation with existing objects can be reduced.

  In addition, by reducing the difference in position and orientation between virtual objects in the virtual space and objects existing in the real space, various elements (objects, users, etc.) can be placed synchronously in the virtual space and the real space. Each user can communicate with a user at a distant location in real space with a sense of reality.

Embodiment 2. FIG.
In the first embodiment, since the robot 104 actually tried to move the object in the real space and confirmed whether or not the object could move, it took time to resolve the contradiction. In the second embodiment, an ID tag that can transmit a value related to the weight and size of the object is provided in the object in the real space, and a function for reading the ID tag is added to the robot. Whether or not an object in the real space can be moved by the robot can be confirmed without actually trying to move, and the contradiction between the real space and the virtual space can be resolved more quickly.

  FIG. 8 is a configuration diagram of the remote conference system according to the second embodiment. The configuration of the remote conference system according to this embodiment will be described with reference to FIG. In addition, the same code | symbol is attached | subjected to the structure similar to the said Embodiment 1, and detailed description is abbreviate | omitted. The virtual space providing server 102 and the client 103 are not shown in FIG.

  The robot 401 of the teleconference system according to the second embodiment newly includes a tag reading unit 402, and reads values related to the weight and size of the object from the tag 403 provided on the object 404 in the real space. It is something that can be done.

  The tag reading unit 402 can read values related to weight, size, and difficulty in moving from the tag 403. In addition, when the tag reading is attempted and fails, the state can be detected.

  The tag 403, which is an information tag, is realized by, for example, RFID (Radio Frequency IDentification), a barcode, a two-dimensional code, and the like, and information on the weight and size of the object including the tag 403 and difficulty in moving by the robot is recorded. The Then, the information is transmitted to the tag reading unit 402. Examples of information related to difficulty in moving by the robot include information on the material (friction coefficient, etc.) and shape of the object surface.

  Similar to the first embodiment, the object 404 is an element that exists in the real space and also exists as data in the virtual space. Further, the object 404 is provided with a tag 403.

  FIG. 9 is an operation flowchart according to the second embodiment. Hereinafter, the operation of the present embodiment will be described with reference to FIG. The same operations as those in the first embodiment are denoted by the same reference numerals, and the description thereof is omitted.

  After the operations of Steps S301 to S314 (FIG. 5) as in Embodiment 1 described above, in Step S501, the robot 401 can read the tag 403 provided on the operation target object 404 using the tag reading unit 402. To check. If it cannot be read, a moving operation based on the operation command is performed in step S504, and the process returns to step S501 again.

  If it can be read, the tag is read in step S502, and values relating to the weight, size, and difficulty of movement by the robot recorded in the tag 403 are obtained.

  In step S503, the robot 401 determines whether or not the object 404 to be moved can be operated based on a difference from predetermined values relating to the weight, size, and difficulty of movement of the object that the robot 401 can operate. Determine whether. In this determination, for example, a threshold value is set for each value corresponding to the tag information, and when the threshold value is exceeded, it can be determined that the operation is impossible.

  If it is determined that the operation can be performed, the operation is performed in step S314 as in the first embodiment, the real space data is measured in step S315, and then it is confirmed whether the movement can be performed in step S316. On the other hand, if it is determined that the operation is not possible, information is presented to the real space in step S317 as in the first embodiment.

  As described above, in the present embodiment, by reading the tag 403 provided on the object 404, when the robot 401 cannot move the object 404 provided with the tag 403, an early determination is made and information for the user is obtained. Since the presentation is performed, it is not necessary to actually try an action on the object, and the contradiction between the real space and the virtual space can be reduced more quickly.

Embodiment 3 FIG.
In the first and second embodiments, the robot has a simple drive unit and can only perform simple operations such as pushing an object in real space. On the other hand, when the robot is configured with a complex drive unit that can work on objects such as arms and grippers, there are advantages that the robot can handle objects and the types of motion that can be performed on the objects increase. On the other hand, it is difficult for the robot alone to determine what kind of operation is performed on the robot. In other words, even if there are multiple actions that can be performed by the robot mechanism, such as grasping and lifting, it is difficult to grasp an object with a thin thickness, and it is difficult to lift an object with a heavy weight. Because there are limitations, it is difficult to determine what action should be performed depending on the object.

  In the third embodiment, a database for associating the type of object specified by the tag and the possible operation command of the robot is arranged on the virtual space providing server 102, and the operation is transmitted to the robot via the network. By transmitting data for instructing a possible motion to the target object, it is possible to determine the motion to be attempted even if the robot has a complicated structure.

  FIG. 10 is a configuration diagram of the remote conference system according to the third embodiment. The configuration of the remote conference system according to this embodiment will be described with reference to FIG. In addition, the same code | symbol is attached | subjected to the structure similar to the said Embodiment 1, and detailed description is abbreviate | omitted.

  The virtual space providing server 601 newly includes an object motion database 602, searches the object motion database 602 using the ID (identification information) of the object 404 sent from the robot 603, and the robot 603 corresponding to the object It further has a function of transmitting possible operation data.

  The object motion database 602 is a motion information database in which motion data related to the motion content of the robot 603 is recorded according to the object ID (identification information) and the robot 603 ID (identification information).

  The robot 603 has, for example, the form shown in FIG. 2B and includes a mechanism for gripping and lifting an object. In addition to the operation of the first or second embodiment, the control unit 604 of the robot 603 transmits the object ID read by the tag reading unit 402 and the ID of the robot 603 to the virtual space providing server 601 via the network 101. From the virtual space providing server 601, operation data that the robot 603 can perform on the object 404 is received.

  The tag 606 that is an information tag is realized by, for example, an RFID, a barcode, a two-dimensional code, and the like, and ID information for uniquely detecting the object 404 including the tag 606 is recorded. Then, the information is transmitted to the tag reading unit 402.

  The tag reading unit 402 can read the ID of the object 404 from the tag 606. In addition, when the tag reading is attempted and fails, the state can be detected.

  FIG. 11 is an operation flowchart according to the third embodiment, FIG. 12 is a diagram illustrating a structure of transmission / reception data according to the third embodiment, and FIG. 13 is a diagram illustrating an operation of the robot according to the third embodiment. Hereinafter, a specific example of operation and a data structure to be transmitted / received will be described with reference to FIGS. In addition, the same code | symbol is attached | subjected about the operation | movement similar to the said Embodiment 1 or 2, and detailed description is abbreviate | omitted.

  After the operations in steps S301 to S314 (FIG. 5) as in the first and second embodiments described above, in step S501, the robot 603 uses the tag reading unit 402 to tag 606 provided on the operation target object 404. Make sure you can read If it cannot be read, a moving operation based on the operation command is performed in step S504, and the process returns to step S501 again. If it can be read, the tag is read in step S502, and the ID recorded in the tag 606 is obtained.

  In step S701, the robot 603 transmits data including the ID of the read tag to the virtual space providing server 601. The transmitted data includes at least data as shown in FIG. Here, in FIG. 12A, the robot ID is data for uniquely indicating the type of the robot 603. By using this ID, data suitable for the features of the robot such as the mechanism can be searched.

  In step S702, the virtual space providing server 601 searches the object operation database 602 using the tag ID received from the robot 603 and the robot ID, and the robot having the robot ID searches for an object having the tag ID. Obtain possible motion data.

  Here, the record in the object action database 602 includes at least data as shown in FIG. In FIG. 12B, one object ID includes a plurality of robot IDs and a plurality of motion data records associated therewith. The motion data may be expressed at the level of a command to the robot such as lifting or pulling an object, or may be expressed at a level such as numerical data that specifically specifies the joint angle of the robot.

  In step S <b> 703, the control unit 604 uses the motion data acquired from the object motion database 602 to create motion data that is a motion command to the robot 603. That is, in order to operate an object, motion data including operations such as grabbing and lifting is created.

A method of generating operation data will be described with reference to FIG.
Consider a case in which an object (microphone) exists at the coordinate position (3, 1) and the position to be moved is (3, 2) as shown in FIG. It is assumed that the object motion database 602 describes that the robot 603 of the present embodiment is capable of, for example, lifting and lowering the object 404. The vector calculation is performed in the same manner as in step S310 in the first embodiment (FIG. 5), and the motion data described in the object motion database 602 is used, so that the robot 104 having a simple configuration in the first embodiment is used. In contrast, the robot 603 according to the third embodiment receives motion data indicating that an object (microphone) is lifted at a coordinate position (3, 1), then moved to a coordinate position (3, 2), and the object is lowered. Can be generated. It should be noted that these data can be generated by using a trajectory generation technique that is an existing technique used in the field of robotics.

  In step S704, the virtual space providing server 601 transmits the created operation data to the robot 603. It is assumed that there is one or more data transmitted at this time.

  In step S705, the robot 603 receives the motion data. In step S706, the robot 603 selects one of the motion data, performs the motion in step S314, and in step S315, executes the real space data. Measure.

  In step S316, as in the first embodiment, it is determined whether the planned operation is possible. If it is determined that the operation is not possible, the process proceeds to step S707, where it is confirmed whether other operation data other than the executed data remains. If remaining, the process returns to step S705. If no other motion data remains, the resolution of the contradiction caused by the robot motion is abandoned, the process proceeds to step S317, and information is presented to the real space as in the first embodiment.

  As described above, in the present embodiment, motion data corresponding to the ID of the object 404 to be moved and the ID of the robot 603 is acquired from the object motion database 602, and the robot 603 is operated based on the acquired data. Even if the structure of the robot 603 is complicated in order to cope with the shape properties of various objects 404 (elements), inconsistency between the virtual space and the real space can be reduced.

  The generation of the operation data may be performed not on the virtual space providing server 102 but on the robot 603 side. In that case, it is necessary to transfer the virtual space data 108 and the data of the object motion database 602 to the robot 603.

  In the first to third embodiments, the data structure shown as an example is merely an example, and other structures that realize the same function may be used.

  In the first to third embodiments, the case where the robot is not arranged in the virtual space has been described. However, the present invention is not limited to this, and the robot may have a property as an object and may be arranged in the virtual space. Absent.

  In the above first to third embodiments, the contradiction resolution instruction to the user by the information presentation unit 1017 has been described for the case of video projection. However, the present invention is not limited to this, and may be, for example, by voice. .

  In the first to third embodiments, the case where the robot includes the object detection unit, the information presentation unit, and the object movement unit according to the present invention has been described. However, the present invention is not limited thereto, and each unit is physically divided. It does not matter. There may be a plurality of robots.

  In the first to third embodiments, the remote conference system for performing a conference by connecting remote locations via a network has been described. However, the present invention is not limited to this, and a plurality of locations in a plurality of locations having similar elements may be used. A user who exists in a remote place by sharing a virtual space with users or objects and having elements similar to the real space at the same time, and transmitting, receiving, sharing, and using real space information and virtual space information It can be used in an information system that enables smooth information exchange and collaborative work.

1 is a configuration diagram of a remote conference system according to Embodiment 1. FIG. 2 is an outline view of the robot according to Embodiment 1. FIG. It is a figure which shows the structure of the transmission / reception data which concerns on Embodiment 1. FIG. 4 is an operation explanatory diagram of the remote conference system according to Embodiment 1. FIG. 3 is an operation flowchart according to the first embodiment. 5 is a diagram showing a method for generating an operation command according to Embodiment 1. FIG. It is a figure which shows the case where the robot which concerns on Embodiment 1 cannot move an object. 6 is a configuration diagram of a remote conference system according to Embodiment 2. FIG. 6 is an operation flowchart according to the second embodiment. FIG. 10 is a configuration diagram of a remote conference system according to a third embodiment. 10 is an operation flowchart according to the third embodiment. It is a figure which shows the structure of the transmission / reception data which concerns on Embodiment 3. FIG. FIG. 10 is a diagram illustrating an operation of the robot according to the third embodiment.

Explanation of symbols

  101 network, 102 virtual space providing server, 103 client, 104 robot, 105 virtual space data creation unit, 106 control unit, 107 data communication unit, 108 virtual space data, 109 inconsistency detection unit, 1010 data communication unit, 1011 control unit, 1012 Information input unit, 1013 information presentation unit, 1014 data communication unit, 1015 control unit, 1016 sensor unit, 1017 information presentation unit, 1018 drive unit, 1019 object, 401 robot, 402 tag reading unit, 403 tag, 404 object, 601 Virtual space providing server, 602 object motion database, 603 robot, 604 control unit, 606 tag.

Claims (9)

A plurality of clients are connected to a server via a communication line, and a virtual object corresponding to an arbitrary object in the real space is drawn in a virtual space defined by the server and displayed on each client. A teleconferencing system that changes the drawing position or direction of the virtual object according to the operation information of
Object detection means for detecting real space information relating to a relative position and / or a relative direction between at least two of the objects;
Object moving means for moving at least one of the objects based on an operation command from the server;
The server
Whether or not the amount of difference between the relative position and / or relative direction between the objects and the relative position and / or relative direction between the virtual objects corresponding to the object is greater than or equal to a predetermined amount based on the real space information A contradiction detector for detecting
And a control unit that obtains the operation command to move the object so that the difference amount is less than the predetermined amount when the difference amount is equal to or greater than the predetermined amount. Further comprising information presenting means for outputting information acquired from the server,
The control unit of the server
The remote conference system according to claim 1, wherein when the difference amount is equal to or greater than a predetermined amount, information indicating that a contradiction has occurred between the real space and the virtual space is output to the information presenting means. The control unit of the server
3. The remote conference system according to claim 2, wherein when the difference amount is equal to or larger than a predetermined amount, information on the position and / or direction of the object at which the difference amount is less than the predetermined amount is output to the information presentation unit. . The control unit of the server
When the operation command for moving the object is output and the difference amount is equal to or greater than a predetermined amount after a predetermined time has passed, the information regarding the position and / or direction of the object at which the difference amount is less than the predetermined amount is presented. 4. The remote conference system according to claim 2, wherein the remote conference system outputs the information to a means. The object detection means includes
Two-dimensional or three-dimensional coordinates with an arbitrary position of the object as an origin are defined, a coordinate according to a distance between the origin position and at least one measurement point of each object is attached, and the coordinate information is Output as real space information,
The server conflict detection unit
Defining two-dimensional or three-dimensional coordinates with a position in the virtual space corresponding to the origin position in the real space as an origin, and obtaining coordinates of a location corresponding to the measurement point of the object in the virtual object;
The distance between the measurement points in the virtual space is compared with the distance between the measurement points in the real space to detect whether the difference amount is a predetermined amount or more. The remote conference system according to any one of 1 to 4. The control unit of the server
Obtaining the operation command for moving the object based on the coordinate vector obtained by subtracting the coordinate vector of the measurement point of the object from the coordinate vector of the measurement point of the virtual object corresponding to the object to be moved; 6. The remote conference system according to claim 5, wherein An information tag attached to the object and recorded with at least one of information on the weight, size, and difficulty of movement of the object;
A tag reading unit that is provided in the object moving means and reads information recorded in the information tag;
The object moving means is
Read the information tag information of the object to be moved according to the operation command, based on the information, determine whether the object is movable,
The remote conference system according to claim 1, wherein when it is determined that the object can be moved, the object is moved based on the operation command. An information tag attached to the object and recording identification information of the object;
A tag reading unit that is provided in the object moving unit, reads information recorded in the information tag, and transmits the identification information of the object and the identification information of the object moving unit to the server;
An operation information database provided in the server, wherein operation data relating to operation contents of the object moving means is recorded according to the identification information of the object and the identification information of the object moving means;
The control unit of the server
Operation data corresponding to the identification information of the object to be moved and the identification information of the object moving means for outputting an operation command is acquired from the operation information database, and the operation command is obtained based on the acquired operation data. The remote conference system according to claim 1, wherein the remote conference system is obtained. A plurality of clients are connected to a server via a communication line, and a virtual object corresponding to an arbitrary object in the real space is drawn in a virtual space defined by the server and displayed on each client. A remote conference system control method for changing the drawing position or direction of the virtual object according to the operation information of
Detecting real space information regarding a relative position and / or a relative direction between at least two or more of the objects;
Whether or not the amount of difference between the relative position and / or relative direction between the objects and the relative position and / or relative direction between the virtual objects corresponding to the object is greater than or equal to a predetermined amount based on the real space information Detecting steps,
Obtaining the operation command for moving the object so that the difference amount is less than a predetermined amount when the difference amount is equal to or greater than a predetermined amount;
And a step of operating an object moving means for moving at least one of the objects based on the operation command.

Images