JP2012142727A - Conference system, conference terminal, and remote side terminal - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a video conference system which allows grasp of the state of a whole conference base.SOLUTION: A remote side terminal 3 transmits image data captured by a front camera 46 etc. and a remote control packet to a conference terminal 5 via a network 2. The conference terminal 5 displays on a front face display 61 etc. the image data received from the remote side terminal 3, and changes the orientation of the front face display 61 etc. according to the remote control packet. The conference terminal 5 transmits image data obtained by capturing an image of the direction equal to the orientation of the front face display 61 and image data obtained by capturing an image of the surrounding from the front position of the front face display 61, to the remote side terminal 3 via the network 2. The remote side terminal 3 displays on a display the image data received from the conference terminal 5.

Description

  The present invention relates to a conference system that executes a remote conference between terminals provided at a plurality of sites, and a conference terminal and a remote terminal used for the remote conference.

  2. Description of the Related Art Conventionally, a video conference system that performs a video conference between terminals provided at a plurality of sites is known. As a terminal used in such a video conference system, there is known a video conference apparatus that can enhance the realism of a conference by matching the direction in which the other party is watching and the display direction of the other party. (For example, refer to Patent Document 1).

JP 2009-65490 A

  In the video conference apparatus described in Patent Literature 1, the direction of a display for displaying a remote user changes according to the operation of the user. Then, the direction facing the display is captured by the camera, and the video is displayed on the terminal device used by the user at the remote location. However, if the camera captures only the direction facing the display, the user at the remote location must grasp the overall state of the conference venue (hereinafter referred to as “conference base”) where the video conference device is installed. I can't. As a result, for example, when a conference participant who is out of the shooting range of the camera speaks, a remote user cannot recognize where the conference participant is and impairs the smooth progress of the video conference. There was a fear.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to provide a conference system, a conference terminal, and a remote terminal that can grasp the state of the entire conference base.

  In the conference system according to the first aspect of the present invention, a remote terminal used by a remote user and a conference terminal installed at a conference venue are connected via a network, and the remote terminal and the A conference system that performs a remote conference by two-way communication with a conference terminal, wherein the remote terminal captures a remote camera that captures the user and a remote image captured by the remote camera. A remote image transmitting means for transmitting to the conference terminal via a network; an operation accepting means for accepting an operation instruction for the conference terminal; and the operation instruction accepted by the operation accepting means via the network. An operation instruction transmission means for transmitting to the conference terminal, wherein the conference terminal receives the remote image via the network. Receiving means, a conference side display for displaying the remote side image received by the remote side image receiving means, an operation instruction receiving means for receiving the operation instructions via the network, and received by the operation instruction receiving means. In response to the received operation instruction, the driving means for changing the orientation of the conference-side display, the first conference camera for photographing the same direction as the orientation of the conference-side display, and a position in front of the conference-side display A conference that transmits a second conference camera that captures the surroundings, a front image captured by the first conference camera, and a surrounding image captured by the second conference camera to the remote terminal via the network. Side image transmission means, and the remote terminal receives the front image and the surrounding image via the network. It includes a side image receiving unit, and a remote-side display that displays the front image and the surrounding image received by the conference side image receiving means.

  According to this, in the remote side terminal, the remote side image and the operation instruction are transmitted to the conference terminal via the network. In the conference terminal, the remote-side image is displayed on the conference-side display, and the orientation of the conference-side display is changed according to the operation instruction. In the conference terminal, a front image obtained by photographing the same direction as the direction of the conference display and a surrounding image obtained by photographing the surroundings from the front position of the conference display are transmitted to the remote terminal via the network. In the remote terminal, the front image and the surrounding image received from the conference terminal are displayed on the remote display. Thereby, the user of a remote place can grasp | ascertain the mode of the whole meeting base.

  In the conference system, the conference terminal may include azimuth adjusting means for adjusting the azimuth in the surrounding image based on the surrounding image captured by the second conference camera. In this case, since the orientation in the surrounding image is adjusted by the conference terminal, the user at the remote location can accurately recognize the orientation in the surrounding image.

  In the conference system, the conference terminal includes a conference-side display control unit that displays a detection image of a specific pattern on the conference-side display, and the azimuth adjustment unit is configured to display the detection image included in the surrounding image. A position opposite to the display position may be detected as the front in the surrounding image. In this case, based on the display position of the detection image displayed on the conference display, a direction opposite to the conference display as viewed from the second conference camera is detected as the front in the surrounding image. Therefore, it is possible to accurately and easily specify which direction in the surrounding image is the front.

  In the conference system, the azimuth adjusting unit may detect a position on the opposite side to the display position of the bezel portion of the conference-side display included in the surrounding image as the front in the surrounding image. In this case, based on the display position of the bezel portion included in the surrounding image, a direction opposite to the conference display as viewed from the second conference camera is detected as the front of the conference terminal. Therefore, it is possible to accurately and easily specify which direction in the surrounding image is the front.

  In the conference system, the conference terminal may include a terminal area deleting unit that deletes a display area of the conference terminal included in the surrounding image from the surrounding image captured by the second conference camera. . In this case, since the conference terminal is not displayed in the surrounding image, the visibility of the surrounding image can be improved.

  In the conference system, the conference terminal includes a support unit provided with the second conference camera, the first conference camera and the conference-side display, and is supported by the support unit so as to be rotatable. The driving means may change the orientation of the conference-side display and the shooting direction of the first conference camera by rotating the movable portion. In this case, the direction of the conference display and the shooting direction of the first conference camera can be changed without changing the shooting position and shooting direction of the second conference camera.

  A conference terminal according to a second aspect of the present invention is a conference terminal that is connected to a remote terminal used by a user at a remote location via a network, and is installed at a conference holding location. A remote image receiving means for receiving a remote image captured by the remote terminal, a conference display for displaying the remote image received by the remote image receiving means, and the network. An operation instruction receiving means for receiving an operation instruction for the conference terminal input from the remote terminal, and the orientation of the conference display is changed according to the operation instruction received by the operation instruction receiving means. The driving means, the first conference camera that captures the same direction as the conference-side display, and the surroundings are photographed at a position ahead of the conference-side display. The second conference camera, the front image captured by the first conference camera, and the surrounding image captured by the second conference camera are transmitted to the remote terminal via the network. Means.

  According to this, in the remote side terminal, the remote side image and the operation instruction are transmitted to the conference terminal via the network. In the conference terminal, the remote-side image is displayed on the conference-side display, and the orientation of the conference-side display is changed according to the operation instruction. In the conference terminal, a front image obtained by photographing the same direction as the direction of the conference display and a surrounding image obtained by photographing the surroundings from the front position of the conference display are transmitted to the remote terminal via the network. In the remote terminal, the front image and the surrounding image received from the conference terminal are displayed on the remote display. Thereby, the user of a remote place can grasp | ascertain the mode of the whole meeting base.

  The conference terminal may include azimuth adjusting means for adjusting the azimuth in the surrounding image based on the surrounding image captured by the second conference camera. In this case, since the orientation in the surrounding image is adjusted by the conference terminal, the user at the remote location can accurately recognize the orientation in the surrounding image.

  The conference terminal includes conference-side display control means for displaying a detection image of a specific pattern on the conference-side display, and the azimuth adjusting means is opposite to the display position of the detection image included in the surrounding image. The side position may be detected as the front in the surrounding image. In this case, based on the display position of the detection image displayed on the conference display, a direction opposite to the conference display as viewed from the second conference camera is detected as the front in the surrounding image. Therefore, it is possible to accurately and easily specify which direction in the surrounding image is the front.

  In the conference terminal, the azimuth adjusting unit may detect a position on the opposite side to the display position of the bezel portion of the conference display included in the surrounding image as the front in the surrounding image. In this case, based on the display position of the bezel portion included in the surrounding image, a direction opposite to the conference display as viewed from the second conference camera is detected as the front of the conference terminal. Therefore, it is possible to accurately and easily specify which direction in the surrounding image is the front.

  The conference terminal may further include terminal area deletion means for deleting the display area of the conference terminal included in the surrounding image from the surrounding image captured by the second conference camera. In this case, since the conference terminal is not displayed in the surrounding image, the visibility of the surrounding image can be improved.

  In the conference terminal, a support unit provided with the second conference camera, a movable unit provided with the first conference camera and the conference-side display, and rotatably supported by the support unit, The drive means may change the direction of the conference display and the shooting direction of the first conference camera by rotating the movable part. In this case, the direction of the conference display and the shooting direction of the first conference camera can be changed without changing the shooting position and shooting direction of the second conference camera.

  A remote terminal according to a third aspect of the present invention is a remote terminal connected to a conference terminal installed at a conference venue via a network and used by a user at a remote location, A remote camera for photographing, a remote image transmitting means for transmitting a remote image captured by the remote camera to the conference terminal via the network, and an operation reception for receiving an operation instruction for the conference terminal Means, an operation instruction transmitting means for transmitting the operation instruction received by the operation receiving means to the conference terminal via the network, and a conference side provided in the conference terminal via the network The conference side that receives a front image obtained by photographing the same direction as the display direction and a surrounding image obtained by photographing the surroundings at a position in front of the conference side display. It includes an image receiving unit, and a remote-side display that displays the front image and the surrounding image received by the conference side image receiving means.

  According to this, in the remote side terminal, the remote side image and the operation instruction are transmitted to the conference terminal via the network. In the conference terminal, the remote-side image is displayed on the conference-side display, and the orientation of the conference-side display is changed according to the operation instruction. In the conference terminal, a front image obtained by photographing the same direction as the direction of the conference display and a surrounding image obtained by photographing the surroundings from the front position of the conference display are transmitted to the remote terminal via the network. In the remote terminal, the front image and the surrounding image received from the conference terminal are displayed on the remote display. Thereby, the user of a remote place can grasp | ascertain the mode of the whole meeting base.

1 is a diagram illustrating an overall configuration of a video conference system 1. FIG. 3 is a block diagram showing an electrical configuration of a remote terminal 3. FIG. It is a perspective view from diagonally upward of the terminal 5 for meetings. 3 is a front view of a conference terminal 5. FIG. FIG. 5 is a cross-sectional view taken along line AA in FIG. 4. FIG. 5 is a cross-sectional view in the direction of arrows BB in FIG. 4. 3 is a block diagram showing an electrical configuration of a conference terminal 5. FIG. 4 is a flowchart showing a remote conference process of a remote terminal 3; It is a flowchart which shows the remote conference process of the terminal 5 for a meeting. It is a flowchart which shows the azimuth | direction calculation process which concerns on 1st Embodiment. This is a specific example of a circular image 181. It is a specific example of the strip-shaped image 182 before orientation adjustment. It is a specific example of the strip | belt-shaped image 182 after orientation adjustment. It is a flowchart which shows an image adjustment process. This is a specific example of the belt-like image 183. It is a flowchart which shows the azimuth | direction calculation process which concerns on 2nd Embodiment. It is a specific example of the strip-shaped image 184 before orientation adjustment. It is a perspective view from diagonally upward of the conference terminal 200 which concerns on a modification. It is a perspective view from diagonally upward of the conference terminal 300 which concerns on a modification.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. The drawings to be referred to are used for explaining technical features that can be adopted by the present invention. The configuration of the apparatus, the flowcharts of various processes, and the like described in the drawings are not intended to be limited to these, but are merely illustrative examples.

  A first embodiment of the present invention will be described with reference to FIGS. In the present embodiment, the video conference system 1 that adjusts the orientation of the housing upper portion 71 (see FIG. 3) of the conference terminal 5 by remote operation using the remote terminal 3 is illustrated.

  The overall configuration of the video conference system 1 will be described with reference to FIG. In the video conference system 1, terminal devices respectively provided at a plurality of bases where users exist are connected via a network 2. A remote terminal 3 installed at a location where the conference is held (hereinafter referred to as a conference location), and a conference terminal 5 installed at a location different from the conference location (hereinafter referred to as a remote location) A case in which a video conference is executed is illustrated. Below, the user who exists in a conference base is called a meeting side user. A user existing at a remote site is called a remote user.

  In the video conference system 1, when a video conference is started between the remote terminal 3 and the conference terminal 5, the video data and audio data acquired by the remote terminal 3 and the conference terminal 5 are respectively stored in the network 2. Are transmitted to and received from each other. The remote terminal 3 outputs the video data and audio data acquired by the conference terminal 5 (that is, video data and audio data of the conference user). The conference terminal 5 outputs video data and audio data (that is, video data and audio data of the remote user) acquired by the remote terminal 3. In this way, the remote terminal 3 and the conference terminal 5 each output video data and audio data of the user existing at the other party's base in real time, and the video conference proceeds. In the present embodiment, the remote user can remotely operate the conference terminal 5 using the remote terminal 3, details of which will be described later.

  The electrical configuration of the remote terminal 3 will be described with reference to FIG. The remote terminal 3 is a general-purpose notebook PC, which includes a CPU 30, ROM 31, RAM 32, HDD 33, video controller 34, audio controller 35, communication device 36, I / O interface 40, keyboard 41, mouse 42, display 43, A microphone 44 and a speaker 45 are provided. Furthermore, the remote terminal 3 includes a plurality of cameras (a front camera 46, a right camera 47, and a left camera 48).

  The ROM 31, RAM 32, and I / O interface 40 are connected to the CPU 30. The video controller 34, the audio controller 35, the keyboard 41, the mouse 42, and the plurality of cameras are connected to the I / O interface 40, respectively. The display 43 is connected to the video controller 34. The microphone 44 and the speaker 45 are connected to the audio controller 35.

  The communication device 36 is a controller that transmits and receives data via the network 2. In the present embodiment, the communication device 36 transmits and receives audio data and video data to and from the conference terminal 5 when a video conference is performed. A remote operation command to be described later is also transmitted to the conference terminal 5 by the communication device 36. The RAM 32 functions as a frame buffer for video data / audio data acquired by the remote terminal 3.

  The front camera 46 is a camera that is disposed in front of the remote user and shoots a range including the front face of the remote user. The right camera 47 is a camera that is disposed on the right side of the remote user and shoots a range including the right profile of the remote user. The left camera 48 is a camera that is arranged on the left side of the remote user and shoots a range including the left profile of the remote user. The front camera 46, the right camera 47, and the left camera 48 of the present embodiment are small and lightweight general-purpose cameras (so-called WEB cameras) that can output captured video data to the network 2 in real time.

  The HDD 33 is a large-capacity hard disk drive, and various data for executing a video conference (for example, video data / audio data acquired by the remote terminal 3 and video data / audio data received from the conference terminal 5). And interrupt programs are stored. A conference processing program for executing a remote conference process (see FIG. 8) described later is also stored in the HDD 33.

  A physical structure of the conference terminal 5 will be described with reference to FIGS. In the following description, the lower right direction, the upper left direction, the lower left direction, and the upper right direction in FIG. 3 are defined as the front side, the rear side, the left side, and the right side of the conference terminal 5 (specifically, a casing lower portion 73 described later). The conference terminal 5 has a housing 70 provided with devices necessary for performing a video conference. The housing 70 includes a housing upper portion 71, a connecting portion 72, and a housing lower portion 73.

  The housing upper part 71 is a box-like body (that is, a vertically long rectangular parallelepiped shape) having a square shape in a plan view and having a vertical direction as a longitudinal direction. A front frame 74, a right frame 75, and a left frame 76, which are frame bodies into which a display is fitted, are respectively provided on the front surface, right surface, and left surface of the housing upper portion 71. A vertically long rectangular front display 61 is fixed to the front frame 74. A right surface display 62 having substantially the same shape as the front display 61 is fixed to the right surface frame 75. A left surface display 63 having substantially the same shape as the front display 61 is fixed to the left surface frame 76. That is, the housing upper part 71 is provided with a display on its front and left and right sides. A touch panel 57 is fixed to the front frame 74 so as to overlap the front display 61. Therefore, the conference terminal 5 can be operated by touching a menu or icon displayed on the front display 61 with a finger.

  A rear panel 77, an upper panel 78, and a lower panel 79 are provided on the rear surface, upper surface, and lower surface of the upper portion 71 of the casing. These panels form an internal space of the upper part 71 of the housing together with the frame body on which the various displays described above are fixed. However, a semicircular opening 79A is formed from the slightly rear side in the center of the bottom panel 79 to the front. Therefore, the internal space of the housing upper portion 71 communicates downward via the opening 79A.

  A front-view camera 67 is provided at a substantially central position in the upper frame portion of the front frame 74. By providing the front-view camera 67 on the same surface as the front display 61, the front-view camera 67 captures the same direction as the image displayed by the front display 61 (that is, in front of the upper portion 71 of the housing). A right microphone 64 is provided at a substantially central position in the upper frame portion of the right frame 75. A left microphone 65 is provided at a substantially central position in the upper frame portion of the left frame 76. The right microphone 64 and the left microphone 65 are a pair of binaural microphones provided on both side surfaces of the upper portion 71 of the housing. As a result, sound generated around the conference terminal 5 is collected from both side surfaces of the upper portion 71 of the casing in a binaural manner.

  The housing lower portion 73 is a box-like body that has a square shape larger than that of the housing upper portion 71 in a plan view and has a vertical direction as a short direction. A speaker 66 capable of outputting sound from the front side of the housing lower part 73 is provided inside the housing lower part 73. A concave portion 73 </ b> A having a square shape in plan view is formed at the center position on the upper surface of the housing lower portion 73. An omnidirectional camera 110 that captures the surroundings at a position in front of the front display 61 is provided at a substantially central position in the left-right direction at the front edge of the housing lower portion 73. The omnidirectional camera 110 according to the present embodiment is a fisheye camera whose imaging range is a predetermined range (for example, a radius of 5 meters), but may include a plurality of wide-angle cameras.

  The connecting portion 72 is a rod-like body having a square shape in plan view and having a vertical direction as a longitudinal direction. At the lower end portion of the connecting portion 72, a collar portion 72A that is a square plate portion larger than the connecting portion 72 in plan view is provided. The collar portion 72A is fixed in a state of being fitted into a concave portion 73A having the same shape in plan view. On the other hand, the upper portion of the connecting portion 72 enters the internal space of the housing upper portion 71 through the opening 79A.

  A drive mechanism 100 and a pair of connecting metal fittings 68 are provided inside the housing upper portion 71. The drive mechanism 100 is a mechanism for connecting the housing upper portion 71 and the connecting portion 72 to change the inclination of the housing upper portion 71 in the horizontal direction and the vertical direction. The pair of connecting fittings 68 are plate-like fittings provided on the back side of the front display 61 and the touch panel 57 and extending side by side from the upper frame portion to the lower frame portion of the front frame 74. The upper end portion and the lower end portion of each connection fitting 68 are fixed to the upper frame portion and the lower frame portion of the front frame 74, respectively.

  The driving mechanism 100 includes a pan motor 80 that adjusts the horizontal inclination of the upper housing 71, a tilt motor 90 that adjusts the vertical inclination of the upper housing 71, and a pair of arms connected to the upper housing 71. 92. The pan motor 80 is fixed to the upper end portion of the connecting portion 72 and has a vertical shaft 81 extending upward. The tilt motor 90 is fixed to the upper side of the pan motor 80 and has a horizontal shaft 91 extending to the left and right sides. The horizontal shafts 91 are connected to the rear end sides of the pair of arm portions 92, respectively. The pair of arm portions 92 are plate-like metal fittings extending in a direction orthogonal to the shaft 91, and the respective tip portions are fixed to the corresponding connection metal fittings 68. The drive mechanism 100 is fixed to the coupling metal 68 so as to support the position of the center of gravity of the housing upper portion 71 in a state where various members such as a plurality of displays are attached to the housing upper portion 71.

  With the above structure, as the pan motor 80 rotates the shaft 81, the tilt motor 90 also rotates around the vertical axis. The housing upper part 71 connected to the pair of arm portions 92 extending from the tilt motor 90 also rotates around the vertical axis. That is, the housing upper portion 71 is inclined in the vertical direction by driving the pan motor 80. As the tilt motor 90 rotates the shaft 91, the pair of arm portions 92 also rotate around the horizontal axis. The housing upper part 71 connected to the pair of arm portions 92 extending from the tilt motor 90 also rotates around the horizontal axis. In other words, the housing upper portion 71 is inclined in the horizontal direction by driving the tilt motor 90.

  As the casing upper portion 71 rotates, the orientation of the front display 61, the right display 62, and the left display 63 (hereinafter collectively referred to as the displays 61 to 63) provided in the casing upper portion 71, and the front-view camera. The shooting direction of 67 also changes. On the other hand, since the omnidirectional camera 110 is fixed to the housing lower portion 73, the surroundings are always photographed at a fixed position and orientation regardless of the rotation of the housing upper portion 71.

  The electrical configuration of the conference terminal 5 will be described with reference to FIG. The conference terminal 5 includes a CPU 50, ROM 51, RAM 52, HDD 53, video controller 54, audio controller 55, communication device 56, and I / O interface 60. Further, the conference terminal 5 includes the touch panel 57, the front display 61, the right display 62, the left display 63, the right microphone 64, the left microphone 65, the speaker 66, the front view camera 67, the pan motor 80, and the tilt motor 90 described above. Yes.

  The ROM 51, RAM 52, and I / O interface 60 are connected to the CPU 50. The video controller 54, the audio controller 55, the front view camera 67, the omnidirectional camera 110, the touch panel 57, the pan motor 80, and the tilt motor 90 are connected to the I / O interface 60, respectively. Each of the displays 61 to 63 is connected to the video controller 54. The right microphone 64, left microphone 65, and speaker 66 are connected to the audio controller 55.

  The communication device 56 is a controller that transmits and receives data via the network 2. In the present embodiment, the communication device 56 transmits and receives audio data and video data to and from the remote terminal 3 when a video conference is performed. A remote operation command to be described later is also received from the remote terminal 3 by the communication device 56. The RAM 52 functions as a frame buffer for video data / audio data acquired by the conference terminal 5.

  The HDD 53 is a large-capacity hard disk drive, and various data for executing a video conference (for example, video data / audio data acquired by the conference terminal 5, video data / audio data received from the remote terminal 3). And interrupt programs are stored. A conference processing program for executing a remote conference process (see FIG. 13) described later is also stored in the HDD 53.

  With reference to FIGS. 8-15, the process regarding the video conference performed with the video conference system 1 of 1st Embodiment is demonstrated. At the time of execution of the video conference, the remote terminal 3 and the conference terminal 5 execute processes described below.

  With reference to FIG. 8, the remote conference process of the remote side terminal 3 is demonstrated. This processing is executed by the CPU 30 based on a conference processing program stored in the HDD 33 when an input operation for instructing the start of the video conference is received at the remote terminal 3.

  In the remote conference process of the remote terminal 3, a conference participation request is first transmitted to a conference server (not shown) (S1). The conference participation request is a command for requesting participation in a video conference executed by the video conference system 1. When the conference server receives the conference participation request via the network 2, the conference server performs user authentication based on the authentication information included in the conference participation request. If the user authentication is successful, a permission for participation is returned to the requesting terminal device. When user authentication fails, a rejection of participation is returned to the requesting terminal device.

  After executing step S1, it is determined whether or not participation permission has been acquired (S3). When participation refusal is received from the conference server, it is determined that participation permission has not been acquired (S3: NO). In this case, the teleconference process (FIG. 8) is terminated without starting the video conference. When participation permission is received from the conference server, it is determined that participation permission has been acquired (S3: YES). In this case, an interrupt program is started from the HDD 33 (S5).

  Based on the interrupt program started in step S5, various data processing (specifically, video data transmission processing, audio data transmission processing, video data reception processing, audio data reception processing) during the video conference is executed.

  In the video data transmission process of the remote terminal 3, each video data acquired from the front camera 46, the right camera 47 and the left camera 48 is transmitted to the conference terminal 5. The video data includes a front face image of the remote user photographed by the front camera 46, a right face image of the remote user photographed by the right camera 47, and a left face of the remote user photographed by the left camera 48. And a face image.

  In the audio data transmission process of the remote terminal 3, the audio data acquired from the microphone 44 is transmitted to the conference terminal 5. This voice data includes the voice of the remote user collected by the microphone 44 and the like.

  In the video data reception process of the remote terminal 3, video data is received from the conference terminal 5 and transferred to the video controller 34. The video controller 34 displays the transferred video data on the display 43. As will be described later, this video data is obtained by projecting the conference base by the front-view camera 67 and the omnidirectional camera 110. Therefore, the remote user can see the face of the conference user and the like by the video displayed on the display 43.

  In the audio data reception process of the remote terminal 3, audio data is received from the conference terminal 5 and transferred to the audio controller 35. The audio controller 35 outputs the transferred audio data from the speaker 45. As will be described later, the audio data is collected by the right microphone 64 and the left microphone 65 at the conference site. Thus, the remote user can hear the conference user's voice or the like by the sound output from the speaker 45.

  After execution of step S5, it is determined whether there is a key input (S7). For example, when an input operation of the keyboard 41 or the mouse 42 is received, it is determined that there is a key input (S7: YES). In this case, it is determined whether or not the key input detected in step S7 is a remote operation of the conference terminal 5 (S9). In step S9, if the key input detected in step S7 matches the predetermined operation content, it is determined that the operation is remote operation of the conference terminal 5 (S9: YES).

  In this case, in response to the key input detected in step S7, a remote operation command for causing the conference terminal 5 to execute a predetermined operation content is generated (S11). For example, when it is desired to tilt the shooting direction of the front-view camera 67 provided in the conference terminal 5 downward, the remote user presses the “↑” key on the keyboard 41 to generate a tilt adjustment command that indicates “upward”. Is done. When the zoom magnification of the front view camera 67 is to be increased, the remote user presses the “+” key on the keyboard 41 to generate a zoom adjustment command for instructing “enlargement”.

  The remote operation command (any one of the pan adjustment command, the tilt adjustment command, and the zoom adjustment command) generated in step S11 is packetized and transmitted to the conference terminal 5 (S13). Hereinafter, the packet transmitted in step S13 is referred to as a remote operation packet. After execution of step S13, it is determined whether the video conference has ended (S15). For example, when an input operation for instructing the end of the video conference is accepted at the remote terminal 3 or a command for instructing the end of the video conference is received from the conference server, it is determined that the video conference has ended (S15). : YES) In this case, the interrupt program started in step S5 is terminated (S17), and the remote conference process (FIG. 8) is terminated. That is, transmission / reception of audio data and video data to / from the conference terminal 5 is interrupted, and the video conference ends.

  On the other hand, when there is no key input (S7: NO), when it is not remote operation of the conference terminal 5 (S9: NO), and when the video conference is not ended (S15: NO), the process returns to step S7. As described above, in the present embodiment, steps S7 to S15 are repeatedly executed at predetermined intervals (for example, a cycle of several milliseconds) from the start to the end of the interrupt program (that is, during execution of the video conference). Therefore, for example, while the “↑” key of the keyboard 41 is kept pressed, the remote operation packet of the tilt adjustment command instructing “upward” is continuously output.

  With reference to FIG. 9 and FIG. 10, the remote conference process of the conference terminal 5 will be described. This process is executed by the CPU 50 based on a conference processing program stored in the HDD 53 when an input operation for instructing the start of a video conference is accepted at the conference terminal 5.

  As shown in FIG. 9, in the remote conference process of the conference terminal 5, first, the origin detection process of the pan motor 80 and the tilt motor 90 is executed (S101). In the origin detection process of the pan motor 80, the housing upper portion 71 is rotated until the horizontal inclination becomes the initial state. Specifically, the casing upper portion 71 is rotated around the vertical axis until the displays 61 to 63 face the front side, the right side, and the left side of the conference terminal 5 (specifically, the casing lower portion 73). Thereby, the shooting direction of the front-view camera 67 coincides with the front of the conference terminal 5 (specifically, the housing lower portion 73).

  On the other hand, in the origin detection process of the tilt motor 90, the casing upper portion 71 is rotated until the vertical inclination becomes the initial state. Specifically, the housing upper portion 71 is rotated around the horizontal axis until the displays 61 to 63 are vertical (in other words, until the shooting direction of the front view camera 67 is horizontal). The housing upper part 71 set in the initial state in step S101 matches the conference terminal 5 (specifically, the housing lower part 73) in the front-rear direction, the left-right direction, and the up-down direction. Thereby, the effort which initializes the direction of the terminal 5 for a meeting at the time of the start of a video conference can be saved.

  After execution of step S101, an azimuth calculation process for adjusting the azimuth of video data acquired from the omnidirectional camera 110 is executed (S102). As shown in FIG. 10, in the azimuth calculation process, a QR code 99 (see FIGS. 11 to 13) of a specific pattern is first displayed on the front display 61 moved to the initial position (S151). In this state, an image taken from the omnidirectional camera 110 is acquired (S153). In this case, as in the example illustrated in FIG. 11, a circular image 181 in which 360 ° omnidirectional images are captured from the omnidirectional camera 110 is acquired. The circular image 181 shows the periphery of the omnidirectional camera 110 including the QR code 99 displayed on the front display 61 located on the rear side of the omnidirectional camera 110.

  The circular image 181 acquired in step S153 is converted into a horizontally long flat image corresponding to the display area shape of a standard display by a known image processing method (S155). In this case, as in the example shown in FIG. 12, a belt-like image 182 obtained by converting the circular image 181 into a flat image is acquired. The band-shaped image 182 is obtained by dividing the circular image 181 into upper and lower image portions and converting each image portion into a planar shape into an upper image portion 182A and a lower image portion 182B, respectively. Therefore, the left end of the upper image portion 182A and the right end of the lower image portion 182B are imagewise continuous, and the right end of the upper image portion 182A and the left end of the lower image portion 182B are imagewise continuous.

  The QR code 99 is detected from the belt-like image 182 acquired in step S155 (S157). Further, the coordinate position X (left-right direction position) of the image center of the QR code 99 is calculated (S159). Steps S157 and S159 may be executed by a known image analysis method. In the example shown in FIG. 12, the QR code 99 displayed in the belt-like image 182 is detected by pattern recognition. A center position C1 in the X direction (left-right direction) of the detected QR code 99 is calculated as the coordinate position X. In the present embodiment, the size of the coordinate position X is obtained with reference to the center position C2 in the X direction (left-right direction) of the belt-like image 182.

  After execution of step S159, the abscissa area of the front display 61 is calculated (S161). Specifically, based on the detection result of step S157, the coordinate position X (lateral position) of the lateral width ends of the QR code 99 is calculated. The width ratio between the QR code 99 and the front display 61 is set in advance. Based on the horizontal width ratio and the coordinate position X of the QR code 99, the coordinate positions X of both ends of the front display 61 are calculated. Further, a rectangular area having both sides extending in the Y direction (vertical direction) from the coordinate position X at both ends of the front display 61 is calculated as the abscissa area. In the example shown in FIG. 12, a rectangular area including the entire width of the front display 61 and extending in the vertical direction of the lower image portion 182B is calculated as the abscissa area D. After execution of step S161, the process returns to the remote conference process (FIG. 9).

  As shown in FIG. 9, after executing step S102, an interrupt program is started from the HDD 53 (S103). Based on the interrupt program started in step S103, various data processing (specifically, video data transmission processing, audio data transmission processing, video data reception processing, and audio data reception processing) during the video conference is executed.

  In the video data transmission process of the conference terminal 5, each video data acquired from the front view camera 67 and the omnidirectional camera 110 is transmitted to the remote terminal 3. This video data includes an image that shows the shooting direction of the front-view camera 67 (the front direction of the upper part 71 of the housing) and an image that shows the surroundings of the omnidirectional camera 110. In the present embodiment, the video data of the omnidirectional camera 110 transmitted to the remote terminal 3 is converted from a circular image to a band image by an image adjustment process (see FIG. 14) described later.

  In the audio data transmission process of the conference terminal 5, the audio data acquired from the right microphone 64 and the left microphone 65 is transmitted to the remote terminal 3. The audio data includes the conference user's voice and the like collected by the right microphone 64 and the conference user's voice and the like collected by the left microphone 65.

  In the video data reception process of the conference terminal 5, video data is received from the remote terminal 3 and transferred to the video controller 54. The video controller 54 displays the transferred video data on the respective displays 61 to 63. Specifically, the video data displayed on the front display 61 is taken by the front camera 46. The video data displayed on the right display 62 is taken by the right camera 47. The video data displayed on the left display 63 is taken by the left camera 48. Therefore, the conference side user can recognize the front face, the right side face, and the left side face of the remote side user from the images displayed on the displays 61 to 63, respectively.

  In the audio data reception process of the conference terminal 5, audio data is received from the remote terminal 3 and transferred to the audio controller 55. The audio controller 55 outputs the transferred audio data from the speaker 66. This audio data is collected by the microphone 44 at a remote site. Therefore, the conference side user can hear the voice of the remote side user by the voice output from the speaker 66.

  After execution of step S103, it is determined whether or not a remote operation packet has been received (S105). If the remote operation packet has been received (S105: YES), it is determined whether or not the remote operation command included in the remote operation packet is a tilt adjustment command (S107). When the remote operation command is a tilt adjustment command (S107: YES), a tilt control signal generated based on the tilt adjustment command is output to the tilt motor 90 (S109). The tilt control signal is a signal for rotationally driving the tilt motor 90 in accordance with the rotation angle designated by the tilt adjustment command.

  The tilt motor 90 adjusts the housing upper portion 71 downward or upward by rotating the housing upper portion 71 around the horizontal axis in accordance with the tilt control signal. For example, when the tilt adjustment command for instructing “downward” described above is transmitted to the conference terminal 5, the tilt motor 90 rotates the housing upper portion 71 clockwise (rightward in FIG. 5) in the right side view. As a result, the shooting direction of the front-view camera 67 is also tilted downward, so that the shooting range displayed on the display 43 of the remote terminal 3 is also moved downward. Therefore, even if the conference side user does not adjust the vertical inclination of the upper portion 71 of the casing, the remote side user can remotely adjust the tilt of the front-view camera 67.

  If the remote operation command is not a tilt adjustment command (S107: NO), it is determined whether or not the remote operation command is a pan adjustment command (S111). When the remote operation command is a pan adjustment command (S111: YES), a pan control signal generated based on the pan adjustment command is output to the pan motor 80 (S113). The pan control signal is a signal for rotationally driving the pan motor 80 in accordance with the rotation angle designated by the pan adjustment command.

  The pan motor 80 adjusts the housing upper portion 71 to the right or left by rotating the housing upper portion 71 around the vertical axis in accordance with the pan control signal. For example, when the pan adjustment command for instructing “rightward” described above is transmitted to the conference terminal 5, the pan motor 80 rotates the housing upper part 71 clockwise in a plan view (clockwise in FIG. 6). As a result, the shooting direction of the front view camera 67 is also tilted to the right, and the shooting range displayed on the display 43 of the remote terminal 3 is also moved to the right. Therefore, even if the conference-side user does not adjust the horizontal inclination of the upper portion 71 of the housing, the remote-side user can perform pan adjustment of the front-view camera 67 by remote operation.

  If the remote operation command is not a pan adjustment command (S111: NO), it is determined whether or not the remote operation command is a zoom adjustment command (S115). When the remote operation command is a zoom adjustment command (S115: YES), a zoom control signal generated based on the zoom adjustment command is output to the front view camera 67 (S117). The zoom control signal is a signal for driving a motor (not shown) that adjusts the zoom magnification of the front-view camera 67 in accordance with the zoom magnification enlargement ratio or reduction ratio instructed by the zoom adjustment command.

  The front view camera 67 changes the zoom magnification according to the zoom control signal received from the conference terminal 5. For example, when the zoom adjustment command for instructing “enlargement” is transmitted to the conference terminal 5, the front-view camera 67 increases the zoom magnification, so that the shooting range displayed on the display 43 of the remote terminal 3 is also narrow. Become. As a result, the video displayed on the display 43 is enlarged or reduced. Therefore, even if the conference-side user does not adjust the zoom magnification of the front-view camera 67, the remote-side user can perform zoom adjustment of the front-view camera 67 by remote operation.

  If a remote operation packet has not been received (S105: NO), or if the remote operation command is not a zoom adjustment command (S115: NO), it is determined whether the video conference has ended, as in step S17 described above. (S119). When the video conference ends (S119: YES). In this case, the interrupt program started in step S103 is terminated (S122), and the remote conference process (FIG. 13) is terminated. That is, transmission / reception of audio data and video data to / from the remote terminal 3 is interrupted, and the video conference ends.

  On the other hand, after execution of any of steps S109, S113, and S117, or when the video conference has not ended (S119: NO), the process returns to step S105. Thus, in this embodiment, steps S105 to S119 are repeatedly executed at predetermined intervals (for example, a cycle of several milliseconds) during the execution of the remote conference process (FIG. 9). Therefore, during the video conference, the remote operation of the conference terminal 5 is executed based on the remote operation command received from the remote terminal 3.

  The image adjustment process will be described with reference to FIG. In the present embodiment, every time video data is acquired from the omnidirectional camera 110 in units of frames in the video data transmission processing of the conference terminal 5, the following image adjustment processing is executed for each video data. Each video data subjected to the image adjustment processing is transmitted to the remote terminal 3 and displayed on the display 43.

  As shown in FIG. 14, in the image adjustment process, first, video data (specifically, a circular image) acquired from the omnidirectional camera 110 is converted into a belt-like image in the same manner as in step S157 (S171). The orientation of the belt-like image acquired in step S171 is shift-adjusted based on the coordinate position X calculated in step S159 (S173). That is, the display range of the belt-like image is shifted in the horizontal direction by the pixel corresponding to the coordinate position X acquired in step S159.

  In the example illustrated in FIG. 13, the display range of the band-shaped image 182 is shifted and adjusted so that the center position C1 of the QR code 99 matches the center position C2 of the band-shaped image 182. As a result, in the upper image portion 182A and the lower image portion 182B, each display range is shifted leftward by the pixel corresponding to the distance between the center positions C1 and C2. Thereby, the QR code 99 is displayed at the center in the left-right direction in the lower image portion 182B.

  The abscissa area D of the front display 61 calculated in step S161 is deleted from the band-shaped image after the shift adjustment (S175). Further, the interpolation processing for the area deleted in step S175 is executed (S177). In the processed belt-like image 183 illustrated in FIG. 15, the conference terminal 5 is deleted in the vertical direction of the lower image portion 182 </ b> B, and interpolation is performed so as to connect the left and right images of the deleted portion. After the execution of step S177, the image adjustment process is completed, and the processed belt-like image is transmitted to the remote terminal 3 and displayed on the display 43.

  In the present embodiment, the front display 61 is disposed behind the omnidirectional camera 110. Therefore, the conference terminal 5 (specifically, the lower part 73 of the housing) when the center position C1 of the QR code 99 displayed on the front display 61 (that is, the center position C1 in the circular image 181) is viewed from the omnidirectional camera 110 is displayed. Can be regarded as the back of Conversely, the opposite side of the center position C <b> 1 in the circular image 181 can be regarded as corresponding to the front of the conference terminal 5.

  Therefore, by shifting and adjusting the belt-like image 182 as described above, the center in the left-right direction of the lower image portion 182B coincides with the rear of the conference terminal 5. The center in the left-right direction of the upper image portion 182A coincides with the front of the conference terminal 5. That is, the center of the upper image portion 182A in the left-right direction can be matched with the shooting direction of the front view camera 67. By such adjustment of the orientation of the belt-like image, the deviation of the orientation in the belt-like image (for example, the front of the belt-like image does not coincide with the shooting direction of the front view camera 67) is corrected. Therefore, the remote user can accurately recognize the orientation in the strip image displayed on the display 43.

  Further, based on the display position of the QR code 99 displayed on the front display 61, a direction opposite to the front display 61 as viewed from the omnidirectional camera 110 is detected as the front in the band-shaped image 182. Therefore, the remote user can accurately and easily specify which direction in the belt-like image 182 displayed on the display 43 is the front. Furthermore, since the conference terminal 5 is not displayed on the belt-like image 182, the visibility of the belt-like image 182 can be improved.

  In the conference terminal 5 according to the present embodiment, the omnidirectional camera 110 is fixed to the housing lower part 73. Therefore, when the shooting direction of the omnidirectional camera 110 is set in advance, the shooting direction is less likely to be misaligned. In this case, for example, the belt-like image 182 is positioned so that the front of the circular image 181 is positioned at the center in the left-right direction of the upper image portion 182A and the publication of the circular image 181 is positioned at the center in the left-right direction of the lower image portion 182B. It only has to be generated. Thereby, the azimuth | direction adjustment (namely, azimuth | direction adjustment process shown in FIG. 10) of a strip | belt-shaped image can be abbreviate | omitted.

  As described above, according to the first embodiment, the remote terminal 3 transmits the video data captured by the front camera 46 and the like and the remote operation packet to the conference terminal 5 via the network 2. . In the conference terminal 5, video data captured by the front camera 46 or the like is displayed on the front display 61 or the like, and the orientation of the front display 61 or the like is changed according to the remote operation packet. In the conference terminal 5, video data obtained by photographing the same direction as the direction of the front display 61 and video data obtained by photographing the surroundings from the front position of the front display 61 (that is, a belt-like image) are remotely transmitted via the network 2. It is transmitted to the terminal 3. In the remote terminal 3, the video data received from the conference terminal 5 is displayed on the display 43. The remote user can grasp the state of the entire conference base by browsing the belt-like image.

  A second embodiment of the present invention will be described with reference to FIGS. 16 and 17. The second embodiment is basically the same as the first embodiment except for the azimuth adjustment processing (see FIG. 10). Only the differences from the first embodiment will be described below.

  As shown in FIG. 16, in the azimuth adjustment process according to the second embodiment (see step S117 in FIG. 9), first, an image taken from the omnidirectional camera 110 is acquired (S201). In this case, similarly to the example shown in FIG. 11, a circular image obtained by photographing 360 ° omnidirectional is acquired from the omnidirectional camera 110. However, the front display 61 on which the QR code 99 is not displayed is displayed in this circular image.

  The circular image acquired in step S201 is converted into a horizontally long strip-like image as in step S155 (S203). In this case, as in the example illustrated in FIG. 17, a belt-like image 184 obtained by converting a circular image into a planar shape is acquired. The band-shaped image 184 is basically the same as the band-shaped image 182 (see FIG. 12), but the QR code 99 is not displayed.

  Based on the belt-like image 184 acquired in step S203, the bezel portion 190 of the front display 61 is detected (S205). For example, a color (specific color) unique to the bezel portion 190 is detected from the belt-like image 184. Edge detection using an edge detection filter is executed for the detected specific color. Contour extraction is performed on the image obtained by the edge detection, and shape characteristics of the bezel portion 190 (for example, four corner portions of the bezel portion 190) are detected from the contour image.

  Based on the bezel portion 190 detected in step S205, the coordinate position X (left-right direction position) of the center of the bezel portion 190 is calculated (S207). In this case, as in the example illustrated in FIG. 17, the center position C3 in the X direction (left-right direction) of the shape feature group of the bezel portion 190 is calculated as the coordinate position X. In the present embodiment, the size of the coordinate position X is obtained with reference to the center position C3 in the X direction (left-right direction) of the belt-like image 184.

  Finally, the abscissa area of the front display 61 is calculated (S209). For example, the coordinate positions X of the lateral width ends of the front display 61 are calculated based on the bezel portion 190 detected in step S205. In the example shown in FIG. 17, the abscissa area D is set over the entire horizontal width of the front display 61 and over the vertical direction of the lower image portion 182B. After execution of step S209, the process returns to the remote conference process (FIG. 9).

  The image adjustment process (FIG. 14) according to the second embodiment is the same as that of the first embodiment. However, in step S173, the display range of the belt-like image is shifted in the horizontal direction by the pixel corresponding to the coordinate position X calculated in step S207 (S181). In the example illustrated in FIG. 17, the display range of the band-shaped image 184 is shifted and adjusted so that the center position C3 of the bezel portion 190 matches the center position C2 of the band-shaped image 184. As a result, in the upper image portion 184A and the lower image portion 184B, each display range is shifted leftward by the pixel corresponding to the distance between the center positions C2 and C3. As a result, the bezel portion 190 is displayed at the center in the left-right direction of the lower image portion 184B.

  In steps S175 and S177, the abscissa area D of the front display 61 calculated in step S209 and the interpolation process of the deleted area are executed. As a result, similarly to the example shown in FIG. 15, the conference terminal 5 is deleted in the vertical direction of the lower image portion 184B, and is interpolated so as to connect the left and right images of the deleted portion. After the execution of step S177, the image adjustment process ends, and the processed band-like image is transmitted to the remote terminal 3 and displayed on the display 43.

  As described above, according to the second embodiment, similarly to the first embodiment, the remote user can grasp the state of the entire conference base by browsing the belt-like image. Further, based on the display position of the bezel portion 190 included in the band-shaped image, a direction opposite to the front display 61 as viewed from the omnidirectional camera 110 is detected as the front in the band-shaped image. Therefore, the remote user can accurately and easily specify which direction in the belt-like image displayed on the display 43 is the front.

  In the above embodiment, the front camera 46 corresponds to the “remote camera” of the present invention. The communication device 36 corresponds to the “remote side image transmission unit”, “operation instruction transmission unit”, and “conference side image reception unit” of the present invention. The CPU 30 executing step S9 corresponds to the “operation receiving unit” of the present invention. The display 43 corresponds to the “remote display” of the present invention. The communication device 56 corresponds to the “remote side image receiving unit”, “operation instruction receiving unit”, and “conference side image transmitting unit” of the present invention. The front display 61 corresponds to the “conference side display” of the present invention. The drive mechanism 100 corresponds to the “drive means” of the present invention. The front-view camera 67 corresponds to the “first conference camera” of the present invention. The omnidirectional camera 110 and omnidirectional cameras 120 and 130 described later correspond to the “second conference camera” of the present invention. The CPU 50 that executes steps S153 to S159, S171, S173, and S201 to S207 corresponds to the “azimuth adjusting means” of the present invention. The CPU 50 that executes steps S161, S175, and S209 corresponds to the “terminal area deleting means” of the present invention.

  In addition, this invention is not limited to the said embodiment, The change in the range which does not change the summary of invention is possible. In the above embodiment, the azimuth calculation process (S102) is executed at the start of the remote conference process (FIG. 9), but may be executed at another timing. For example, when a remote user performs a predetermined operation on the remote terminal 3, the azimuth calculation process shown in FIG. 10 or FIG. 16 may be appropriately executed.

  The configuration of the omnidirectional camera provided in the conference terminal 5 can be changed as appropriate. The conference terminal 200 shown in FIG. 18 includes an omnidirectional camera 120 that can be placed at a position separated from the housing lower part 73. The omnidirectional camera 120 is connected to a control unit (such as the CPU 50) provided in the housing lower part 73 by a cable 121, and the omnidirectional camera 120 can be installed at a free position within the range in which the cable 121 extends. However, in order to accurately adjust the orientation of the video data captured by the omnidirectional camera 120, the front display 61 is set to the installation position of the omnidirectional camera 120 during execution of the azimuth calculation processing (see FIGS. 10 and 16). It is preferable to face.

  A conference terminal 300 shown in FIG. 19 includes an omnidirectional camera 130 that can be placed at a position separated from the housing lower portion 73. The omnidirectional camera 130 is connected to a control unit (CPU 50 and the like) and a drive motor (not shown) provided in the lower part 73 of the housing by a cable 131. In response to driving of a drive motor (not shown), the cable 131 extends from the inside of the housing lower part 73 and the omnidirectional camera 130 moves forward. The cable 131 is wound up in the housing lower part 73 and the omnidirectional camera 130 is moved. fall back. The advance / retreat operation of the omnidirectional camera 130 can be remotely operated by the remote user using the remote terminal 3 in the same manner as the pan / tilt control of the upper casing 71 described above.

  By making the omnidirectional camera separable from the conference terminal as described above, for example, the front display 61 can be prevented from obstructing the imaging range of the step, and the visibility of the photographic data of the omnidirectional camera can be improved. When the omnidirectional camera is separated from the conference terminal in this manner, the azimuth of the omnidirectional camera is not constant, and thus the azimuth shift occurs in the shooting data of the omnidirectional camera. However, since the deviation of the azimuth is corrected by adjusting the azimuth of the shooting data described above, the remote user can accurately and easily set the azimuth in the shooting data regardless of the position of the omnidirectional camera. Can be identified.

  In the first embodiment, the QR code 99 of a specific pattern is displayed on the front display 61. However, the QR code 99 may indicate various information. For example, the QR code 99 may indicate an identification code (ID) unique to the remote terminal 3 that is remotely operating the conference terminal 5. Specifically, when a plurality of conference terminals 5 are installed in one conference room, the front display 61 of each other conference terminal 5 is photographed by the omnidirectional camera 110 of each conference terminal 5. Sometimes. At this time, a plurality of QR codes 99 are captured by each omnidirectional camera 110, and each conference terminal 5 may not be able to identify the QR code 99 displayed by itself. According to the present modification, each conference terminal 5 uses the QR code 99 corresponding to the remote terminal 3 that remotely controls each conference terminal 5 according to the unique identification code indicated by the QR code 99. The displayed QR code 99 can be specified. As a result, even when a plurality of conference terminals 5 are used at the same time, each conference terminal 5 can accurately adjust the direction based on the identified QR code 99.

1 Video conferencing system 2 Network 3 Remote terminal 5 Conference terminal 30 CPU
36 Communication Device 43 Display 46 Front Camera 50 CPU
56 Communication Device 61 Front Display 67 Front View Camera 100 Drive Mechanism 110 Omnidirectional Camera 120 Omnidirectional Camera 130 Omnidirectional Camera 190 Bezel

Claims (13)

A remote terminal used by a user in a remote location and a conference terminal installed at a conference venue are connected via a network, and a remote conference can be performed by bidirectional communication between the remote terminal and the conference terminal. A conferencing system to execute,
The remote terminal is
A remote camera for photographing the user;
Remote-side image transmitting means for transmitting a remote-side image captured by the remote-side camera to the conference terminal via the network;
Operation accepting means for accepting an operation instruction for the conference terminal;
An operation instruction transmission means for transmitting the operation instruction received by the operation reception means to the conference terminal via the network;
The conference terminal is:
Remote-side image receiving means for receiving the remote-side image via the network;
A conference side display for displaying the remote side image received by the remote side image receiving means;
An operation instruction receiving means for receiving the operation instruction via the network;
Drive means for changing the orientation of the conference-side display in response to the operation instruction received by the operation instruction receiving means;
A first conference camera that captures the same direction as the direction of the conference display;
A second conference camera for photographing the surroundings at a position in front of the conference-side display;
A conference-side image transmission means for transmitting a front image captured by the first conference camera and a surrounding image captured by the second conference camera to the remote terminal via the network;
Further, the remote terminal is
Meeting-side image receiving means for receiving the front image and the surrounding image via the network;
A conference system comprising: a remote display that displays the front image and the surrounding image received by the conference image receiving means.   The conference system according to claim 1, wherein the conference terminal includes azimuth adjusting means that adjusts the azimuth in the surrounding image based on the surrounding image captured by the second conference camera. The conference terminal includes conference-side display control means for displaying a detection image of a specific pattern on the conference-side display,
The conference system according to claim 2, wherein the azimuth adjusting unit detects a position opposite to a display position of the detection image included in the surrounding image as a front in the surrounding image.   3. The conference according to claim 2, wherein the azimuth adjusting unit detects a position opposite to a display position of a bezel portion of the conference-side display included in the surrounding image as a front in the surrounding image. system.   The said conference terminal is provided with the terminal area | region deletion means which deletes the display area of the said conference terminal contained in the said surrounding image from the said surrounding image image | photographed with said 2nd conference camera. The conference system according to any one of 1 to 4. The conference terminal includes a support unit provided with the second conference camera, and a movable unit provided with the first conference camera and the conference-side display and rotatably supported by the support unit,
5. The conference according to claim 1, wherein the driving unit changes the orientation of the conference-side display and the shooting direction of the first conference camera by rotating the movable part. 6. system. A conference terminal connected to a remote terminal used by a user in a remote location via a network and installed at a conference venue,
Remote-side image receiving means for receiving a remote-side image captured by the remote-side terminal via the network;
A conference side display for displaying the remote side image received by the remote side image receiving means;
An operation instruction receiving means for receiving an operation instruction for the conference terminal input at the remote terminal via the network;
Drive means for changing the orientation of the conference-side display in response to the operation instruction received by the operation instruction receiving means;
A first conference camera that captures the same direction as the direction of the conference display;
A second conference camera for photographing the surroundings at a position in front of the conference-side display;
A conference-side image transmission unit configured to transmit a front image captured by the first conference camera and a surrounding image captured by the second conference camera to the remote terminal via the network. A featured conference terminal.   The conference terminal according to claim 7, further comprising an orientation adjustment unit that adjusts an orientation in the surrounding image based on the surrounding image captured by the second conference camera. A conference-side display control means for displaying a specific pattern detection image on the conference-side display;
9. The conference terminal according to claim 8, wherein the azimuth adjusting unit detects a position opposite to a display position of the detection image included in the surrounding image as a front in the surrounding image.   9. The conference according to claim 8, wherein the azimuth adjusting unit detects a position opposite to a display position of a bezel portion of the conference-side display included in the surrounding image as a front in the surrounding image. Terminal.   11. The terminal area deleting means for deleting a display area of the conference terminal included in the surrounding image from the surrounding image captured by the second conference camera. The conference terminal described in 1. A support portion provided with the second conference camera;
The first conference camera and the conference-side display are provided, and includes a movable portion that is rotatably supported by the support portion,
The conference according to any one of claims 7 to 11, wherein the driving means changes the orientation of the conference-side display and the shooting direction of the first conference camera by rotating the movable part. Terminal. A remote terminal connected to a conference terminal installed at a conference venue via a network and used by a user at a remote location,
A remote camera for photographing the user;
Remote-side image transmitting means for transmitting a remote-side image captured by the remote-side camera to the conference terminal via the network;
Operation accepting means for accepting an operation instruction for the conference terminal;
An operation instruction transmitting means for transmitting the operation instruction received by the operation receiving means to the conference terminal via the network;
A conference that receives, via the network, a front image obtained by photographing the same direction as a conference-side display provided in the conference terminal and a surrounding image obtained by photographing the surroundings at a position ahead of the conference-side display. Side image receiving means;
A remote terminal comprising: a remote display that displays the front image and the surrounding image received by the conference image receiving means.