JP7047520B2 - Management equipment, methods, programs and management systems - Google Patents

Description

The present invention relates to management devices, methods, programs and management systems.

In recent years, with the development of network and multimedia transmission technology, it has become easy to introduce telework that connects satellite offices in remote areas or between satellite offices in rural areas and the head office. For this reason, an increasing number of companies are promoting the introduction of telework as part of management reforms. As a document disclosed by the technique for supporting telework, for example, Patent Document 1 can be mentioned.

To support telework, it is conceivable to send and receive video and audio from multiple locations between the locations. For example, each base has a large screen terminal having an imaging function and a large display function for acquiring a bird's-eye view image, and a communication terminal used by each member having an imaging function and a display function for acquiring an image of each member. A terminal is provided. By displaying the bird's-eye view image acquired by the large screen terminal of another base on the large screen terminal, it is possible to realize the feeling that the members in different bases exist in the same base. In addition, it is possible to obtain a sense of realism like a dialogue through a window by having members existing in different bases have a dialogue using a large screen terminal. The dialogue between the members can also be realized by transmitting and receiving video and audio between the communication terminals used by each member.

Japanese Unexamined Patent Publication No. 2014-164654

As described above, the dialogue using the large screen terminal can realize a realistic feeling like a dialogue through a window. However, since the size of the member's face image included in the bird's-eye view image acquired by the large screen terminal differs depending on the position of the member, a sufficient sense of presence may not be obtained even in a dialogue using the large screen terminal.

Therefore, the present invention has been made in view of the above problems, and an object of the present invention is a new and improved management device capable of improving the sense of presence in dialogue between remote locations. , Methods, programs and management systems.

In order to solve the above problems, according to a certain viewpoint of the present invention, it is a management device connected to a first image pickup device for acquiring a first image and a second image pickup device for acquiring a second image. Then, the transmission of the second image to the communication partner device that receives the first image from the first image pickup device and displays the first image is transmitted to the position of the first image pickup device and the position of the first image pickup device. The second image pickup device includes a control unit that controls based on the distance from the position of the second image pickup device, and the second image pickup device is an image pickup device associated with a user who communicates via the communication partner device. , Management equipment is provided.

The control unit outputs the second image to the second image pickup device based on the distance between the position of the first image pickup device and the position of the second image pickup device being equal to or more than a threshold value. It may be transmitted to the communication partner device .

The control unit further transfers the second image pickup device to the communication partner device based on the relationship between the image pickup direction by the first image pickup device and the image pickup direction by the second image pickup device. The transmission of the second image may be controlled.

The control unit attaches the second image pickup device to the second image pickup device based on the difference between the image pickup direction by the first image pickup device and the image pickup direction by the second image pickup device being less than the reference. The image of the above may be transmitted to the second image pickup apparatus.

The control unit determines whether or not the difference between the image pickup direction by the first image pickup device and the image pickup direction by the second image pickup device is less than the reference, the object included in the first image. And may be determined based on the correspondence with the object included in the second image.

The second image pickup device may acquire the voice, and the control unit may cause the second image pickup device to transmit the voice to the communication partner device.

The control unit may calculate the distance between the position of the first image pickup device and the position of the second image pickup device based on the position of the second image pickup device in the first image. good.

Further, in order to solve the above problems, according to another aspect of the present invention, the management connected to the first image pickup device for acquiring the first image and the second image pickup device for acquiring the second image. A method executed by the device, wherein the second image is transmitted to a communication partner device that receives the first image from the first image pickup device and displays the first image. The second image pickup device includes the control based on the distance between the position of the first image pickup device and the position of the second image pickup device, and the second image pickup device is for a user who communicates via the communication partner device. A method , which is an associated imaging device, is provided.

Further, in order to solve the above problems, according to another aspect of the present invention, a computer is connected to a first image pickup device for acquiring a first image and a second image pickup device for acquiring a second image. The second image is transmitted to the communication partner device that receives the first image from the first image pickup device and displays the first image. A control unit that controls based on the distance between the position of the image pickup device and the position of the second image pickup device is provided, and the second image pickup device corresponds to a user who communicates via the communication partner device. A program is provided for functioning as a management device, which is an image pickup device .

Further, in order to solve the above problems, according to another aspect of the present invention, the management connected to the first image pickup device for acquiring the first image and the second image pickup device for acquiring the second image. A management system including the device, wherein the second image is transmitted to a communication partner device that receives the first image from the first image pickup device and displays the first image. The second image pickup device includes a control unit that controls based on the distance between the position of the image pickup device and the position of the second image pickup device, and the second image pickup device corresponds to a user who communicates via the communication partner device. A management system , which is an attached imaging device, is provided.

According to the present invention described above, it is possible to improve the sense of presence in dialogue between remote locations.

It is explanatory drawing which shows an example of the structure of the remote communication system by embodiment of this invention. It is explanatory drawing which shows the structure of the large screen terminal 10. The bird's-eye view image of the base A displayed on the large screen terminal 10B of the base B is shown. The bird's-eye view image of the base B displayed on the large screen terminal 10A of the base A is shown. It is explanatory drawing which shows the appearance structure of the communication terminal 20 by embodiment of this invention. It is explanatory drawing which shows the structure of the communication terminal 20 by embodiment of this invention. It is explanatory drawing which shows the structure of the terminal management server 30. It is explanatory drawing which shows the physical space and the bird's-eye view image of the base A. It is a top view of the physical space of the base A. It is explanatory drawing which shows the specific example of the dialogue screen. It is explanatory drawing which shows the specific example of the dialogue screen. It is explanatory drawing which shows the operation of the remote communication system by embodiment of this invention. It is explanatory drawing which shows the operation of the remote communication system by embodiment of this invention. It is explanatory drawing which shows the structure of the terminal management server 31 by an application example. It is explanatory drawing which shows the bird's-eye view image of the base acquired by a large screen terminal 10. It is explanatory drawing which shows the operation of the remote communication system by an application example. It is explanatory drawing which shows the operation of the remote communication system by an application example. It is a block diagram which showed the hardware composition of the terminal management server 30.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the present specification and the drawings, components having substantially the same functional configuration are designated by the same reference numerals, so that duplicate description will be omitted.

Further, in the present specification and the drawings, a plurality of components having substantially the same functional configuration may be distinguished by adding different alphabets after the same reference numerals. For example, a plurality of configurations having substantially the same functional configuration or logical significance are distinguished as necessary, such as communication terminals 20A and 20B. However, when it is not necessary to particularly distinguish each of the plurality of components having substantially the same functional configuration, only the same reference numerals are given to each of the plurality of components. For example, when it is not necessary to distinguish between the communication terminals 20A and 20B, each communication terminal is simply referred to as a communication terminal 20.

<1. Overview of remote communication system>
An embodiment of the present invention relates to a remote communication system that realizes communication between remote bases. Hereinafter, an outline of the remote communication system will be described with reference to FIG.

FIG. 1 is an explanatory diagram showing an example of a configuration of a remote communication system according to an embodiment of the present invention. As shown in FIG. 1, the remote communication system according to the embodiment of the present invention includes a large screen terminal 10, a communication terminal 20, and a terminal management server 30.

FIG. 1 illustrates communication between two remote bases, and the network 12 includes a large screen terminal 10A and a communication terminal 20A provided at the base A, a large screen terminal 10B provided at the base B, and the like. The communication terminal 20B and the terminal management server 30 are connected.

The network 12 may include a public line network such as a telephone line network, the Internet, and a satellite communication network, a LAN (Local Area Network), a WAN (Wide Area Network), and the like. Further, the network 12 may include a dedicated line network such as IP-VPN (Internet Protocol-Virtual Private Network).

(Large screen terminal)
The large screen terminal 10 is a terminal shared at a base, and has a video output unit 130 which is a large display for displaying a bird's-eye view image of another base. Members existing at the base can get a feeling as if the members of the other base exist in the same space by displaying the bird's-eye view image of the other base on the video output unit 130. With reference to FIG. 2, the configuration of such a large screen terminal 10 will be described more specifically.

FIG. 2 is an explanatory diagram showing the configuration of the large screen terminal 10. As shown in FIG. 2, the large screen terminal 10 has a video output unit 130, an image pickup unit 140, an audio input / output unit 150, a communication unit 160, an information storage unit 170, and a control unit 180.

The video output unit 130 is a display larger than the video output unit of the communication terminal 20 described later. The video output unit 130 displays a bird's-eye view image of another base obtained by the image pickup unit 140 of the large screen terminal 10 of the other base.

The image pickup unit 140 has a function of capturing a bird's-eye view image of the base. The large screen terminal 10 having such an image pickup unit 140 or an image pickup unit 140 can be said to be a first image pickup device for acquiring a first image. The audio input / output unit 150 has an audio input function and an output function. For example, the audio input / output unit 150 collects the sound of the base (including the environmental sound) and outputs the sound of the other base collected by the voice input / output unit 150 of the large screen terminal 10 of the other base. do.

The communication unit 160 communicates with the large screen terminal 10 and the communication terminal 20 provided at other bases via the network 12. For example, the communication unit 160 transmits the bird's-eye view video obtained by the imaging unit 140 and the audio obtained by the audio input / output unit 150 to the large screen terminal 10 existing in another base, and the communication unit 160 transmits the large screen terminal existing in the other base. Receives a bird's-eye view video of another base and audio of another base from the terminal 10.

The information storage unit 170 stores information used for the operation of the large screen terminal 10, information about the large screen terminal 10, and the like. The control unit 180 controls the overall operation of the large screen terminal 10. For example, the control unit 180 controls transmission / reception of a bird's-eye view video and audio by the communication unit 160.

Here, with reference to FIGS. 3 and 4, a specific example of the bird's-eye view image displayed on the large screen terminal 10 of each base will be described. FIG. 3 shows a bird's-eye view image of the base A displayed on the large screen terminal 10B of the base B, and FIG. 4 shows a bird's-eye view image of the base B displayed on the large screen terminal 10A of the base A. The bird's-eye view of the base A shown in FIG. 3 includes the members MA1, MA2 and MA3 existing in the base A, and the bird's-eye view of the base B shown in FIG. 4 includes the members MB1 and MB2 existing in the base B. And MB3 are included.

(Communication terminal)
The communication terminal 20 is a communication device for supporting communication between a plurality of bases. For example, the communication terminal 20 establishes a connection with a communication terminal 20 existing at another base. When the connection between the two communication terminals 20 existing at different bases is established, each member of the two communication terminals 20 can have a dialogue using the communication terminal 20.

FIG. 5 is an explanatory diagram showing an external configuration of the communication terminal 20 according to the embodiment of the present invention. As shown in FIG. 5, it includes an imaging unit 220 for acquiring video, a sound collecting unit 230 for acquiring audio, a video output unit 240, an audio output unit 250, and a light emitting unit 270. The video output unit 240 displays a bird's-eye view image of another base transmitted from the large screen terminal 10 of another base, and the audio output unit 250 outputs the audio of the other base transmitted from the large screen terminal 10. .. Hereinafter, the configuration of the communication terminal 20 will be described more specifically with reference to FIG.

FIG. 6 is an explanatory diagram showing the configuration of the communication terminal 20 according to the embodiment of the present invention. As shown in FIG. 6, the communication terminal 20 according to the embodiment of the present invention includes an imaging unit 220, a communication unit 224, a sound collecting unit 230, a video output unit 240, an audio output unit 250, an information storage unit 260, and a light emitting unit 270. And has a control unit 280. Some of these functions may be implemented in a server (not shown).

The image pickup unit 220 acquires an image of the subject. It can be said that the image pickup unit 220 or the communication terminal 20 having the image pickup unit 220 is a second image pickup device that acquires a second image. When the imaging direction of the imaging unit 220 is directed to the member who uses the communication terminal 20, the imaging unit 220 acquires the image of the member who uses the communication terminal 20. When the communication terminal 20 is connected to the communication terminal 20 of another base, the image of the member acquired by the image pickup unit 220 is transmitted from the communication unit 224 to the communication terminal 20 of the other base.

The communication unit 224 is an interface of another device. For example, the communication unit 224 functions as a receiving unit for receiving a bird's-eye view video and audio of another base from a large screen terminal 10 of another base, and a member video and sound collecting unit 230 acquired by the imaging unit 220. It has a function as a transmission unit that transmits the acquired voice of the member to the communication terminal 20 of another base. Further, the communication unit 224 receives various control signals from the terminal management server 30.

The sound collecting unit 230 acquires an audio signal by converting air vibration into an electric signal. For example, the sound collecting unit 230 acquires an audio signal of a voice emitted by a member who uses the communication terminal 20. In the present specification, for convenience of explanation, the voice signal acquired by the sound collecting unit 230 may be simply referred to as voice or input voice. When the communication terminal 20 is connected to the communication terminal 20 of another base, the voice of the member acquired by the sound collecting unit 230 is transmitted from the communication unit 224 to the communication terminal 20 of the other base.

The video output unit 240 is a display unit that outputs a bird's-eye view image of another base received from the large screen terminal 10 of another base by the communication unit 224. When the communication terminal 20 is connected to the communication terminal 20 of another base, the communication unit 224 receives the video of the member who uses the communication terminal 20 of the other base, and the video of the member is received by the video output unit 240. It is output.

The voice output unit 250 outputs the voice of another base received from the large screen terminal 10 of another base by the communication unit 224. When the communication terminal 20 is connected to the communication terminal 20 of another base, the communication unit 224 receives the voice of the member who uses the communication terminal 20 of the other base, and the voice of the member is received by the voice output unit 250. It is output. The audio output unit 250 may be composed of, for example, a speaker.

The information storage unit 260 stores terminal information such as the ID of the communication terminal 20 and the information of the members who use the communication terminal 20.

The light emitting unit 270 is composed of one or more light emitting elements. The light emitting unit 270 may be, for example, a plurality of LEDs (Light Emitting Diodes) arranged in a circumferential shape on the top surface of the communication terminal 20 as shown in FIG. The light emitting unit 270 emits light according to a light emitting instruction transmitted from the terminal management server 30.

The control unit 280 controls the overall operation of the communication terminal 20. For example, the control unit 280 controls transmission / reception of video and audio by the communication unit 224, light emission by the light emitting unit 270, and the like.

(Terminal management server)
The terminal management server 30 is an example of a management device that controls connections between communication terminals 20 existing at different bases, and connections between communication terminals 20 and large screen terminals 10 existing at different bases. The function of the terminal management server 30 may be implemented in an independent server using, for example, XMPP (eXtensible Communication and Presence Protocol), or may be implemented in any one of a plurality of communication terminals 20. , It may be distributedly mounted on a plurality of communication terminals 20, or it may be mounted on a large screen terminal 10. Hereinafter, a detailed configuration of such a terminal management server 30 will be described.

<2. Terminal management server configuration>
FIG. 7 is an explanatory diagram showing the configuration of the terminal management server 30. As shown in FIG. 7, the terminal management server 30 according to the embodiment of the present invention has a communication unit 324, a terminal information storage unit 328, a distance calculation unit 332, a connection destination determination unit 336, and a connection control unit 340.

(Communication department)
The communication unit 324 has a function of performing communication via the network 12, and communicates with the large screen terminal 10 and the communication terminal 20. For example, the communication unit 324 receives a connection request from the large screen terminal 10 and transmits a connection instruction instructing the connection destination communication terminal 20 to connect to the large screen terminal 10.

(Terminal information storage unit)
The terminal information storage unit 328 stores the terminal information of each large screen terminal 10 and each communication terminal 20. The terminal information of the communication terminal 20 includes the ID of the communication terminal 20 and information indicating the distance between the image pickup unit 140 of the large screen terminal 10 provided at the same base and the communication terminal 20 calculated by the distance calculation unit 332 described later. And so on. In this specification, an example in which the distance between the large screen terminal 10 and the communication terminal 20 is calculated by the distance calculation unit 332 will be described, but the distance between the large screen terminal 10 and the communication terminal 20 may be set in advance. ..

(Distance calculation unit)
The distance calculation unit 332 calculates the distance between the large screen terminal 10 and the communication terminal 20 with respect to the large screen terminal 10 and the communication terminal 20 provided at the same base. For example, when the light emitting unit 270 of the communication terminal 20 emits light, the large screen terminal 10 detects the light emission of the light emitting unit 270 in the bird's-eye view image, identifies the terminal coordinates of the communication terminal 20 from the position of the light emitting unit 270 in the bird's-eye view image, and determines the terminal coordinates of the communication terminal 20. The large screen terminal 10 transmits the terminal coordinates to the terminal management server 30. The distance calculation unit 332 can calculate the distance between the large screen terminal 10 and the communication terminal 20 based on the terminal coordinates of the communication terminal 20. This point will be described more specifically with reference to FIGS. 8 and 9.

FIG. 8 is an explanatory diagram showing the physical space of the base A and the bird's-eye view image. The upper part of FIG. 8 shows the physical space of the base A, and the upper part of FIG. 8 shows the three members working at the base A. The three members include members MA1 to MA2, and communication terminals 20A # 1 to # 2 used by members MA1 to MA2 are also shown.

The position of the base A in the physical space is represented using three-dimensional coordinate data represented by the axes of x, y, and z. The communication terminal 20 used by each member is associated with a position represented by using the three-dimensional coordinate data. For example, the position of the communication terminal 20A # 2 used by the member MA2 is expressed as (x1, y1, z1).

The lower part of FIG. 8 shows the bird's-eye view image of the base A, and the position in the bird's-eye view image is expressed by using the two-dimensional coordinate data represented by the axes of X and Y. For example, the position of the communication terminal 20A # 2 used by the member MA2 in the bird's-eye view image is expressed as (X1, Y1).

FIG. 9 is a plan view of the physical space of the base A described with reference to FIG. That is, FIG. 9 shows the plane of the base A represented by the x and y axes described with reference to FIG. The alternate long and short dash line extending from the image pickup unit 140 shown in FIG. 9 indicates the angle of view of the image pickup unit 140.

The distance calculation unit 332 is a distance L1 between the image pickup unit 140 and each of the communication terminals 20A # 1 to # 3, based on the terminal coordinates of the communication terminals 20A # 1 to # 3 received from the large screen terminal 10. ~ L3 is calculated. The distances L1 to L3 calculated by the distance calculation unit 332 are stored in the terminal information storage unit 328.

(Connection destination determination unit)
When the connection request is received from the large screen terminal 10 or the communication terminal 20, the connection destination determination unit 336 determines the connection destination related to the connection request. For example, when the name of a member existing in the base B is called to the large screen terminal 10 or the communication terminal 20 of the base A and a voice indicating the name is received as a connection request, the connection destination determination unit 336 receives the voice. The communication terminal 20 corresponding to the name indicated by may be determined as the connection destination. Alternatively, when the display of the member existing in the base B is touched in the bird's-eye view image displayed on the large screen terminal 10 or the communication terminal 20, and the information indicating the touch position in the bird's-eye view image is received as a connection request, the connection destination determination unit The 336 may determine the communication terminal 20 having the terminal coordinates corresponding to the touch position as the connection destination.

(Connection control unit)
The connection control unit 340 is a function of the control unit and controls the connection between the communication terminals 20 existing at different bases and the connection between the communication terminals 20 and the large screen terminal 10 existing at different bases. For example, regarding the connection between the large screen terminal 10A existing in the base A and the communication terminal 20B existing in the base B, the connection control unit 340 causes the large screen terminal 10A and the communication terminal 20B to transmit and receive voice. On the other hand, the connection control unit 340 controls whether or not to transmit / receive video between the large screen terminal 10A and the communication terminal 20B based on the positional relationship between the communication terminal 20B and the large screen terminal 10B existing in the base B. ..

In this regard, it is realistic for a member located near the large screen terminal 10B of the base B to interact with the member of the base A using the large screen terminal 10B instead of the communication terminal 20B owned by the member. Preferred from the point of view. On the other hand, when a member located away from the large screen terminal 10B of the base B talks with the member of the base A using the large screen terminal 10B, the display of the member of the base B displayed on the large screen terminal 10A of the base A is displayed. Because it is small, the sense of presence is reduced.

Therefore, when the distance between the communication terminal 20B and the large screen terminal 10B existing in the base B is less than the threshold value, the connection control unit 340 uses the large screen terminal 10B instead of the communication terminal 20B for video. Therefore, the video is not transmitted and received between the large screen terminal 10A and the communication terminal 20B. On the other hand, when the distance between the communication terminal 20B and the large screen terminal 10B existing in the base B is equal to or larger than the threshold value, the large screen terminal 10A and the communication terminal 20B are used for the video as well. Video is also sent and received between.

As a result, the communication terminal 20B displays the bird's-eye view image of the base A received from the large screen terminal 10A. As shown in FIG. 10, for example, the bird's-eye view image of the base A includes a display of a member MA4 who is a dialogue partner of a member who uses the communication terminal 20B. Further, the image of the member received from the communication terminal 20B is displayed on the large screen terminal 10A. For example, as shown in FIG. 11, the POPUP display D including the display of the member MB3 is superimposed on the bird's-eye view image of the base B displayed on the large screen terminal 10A.

<3. Operation>
The configuration of the terminal management server 30 according to the embodiment of the present invention has been described above. Subsequently, with reference to FIGS. 12 and 13, the operation of the remote communication system according to the embodiment of the present invention will be organized.

12 and 13 are explanatory views showing the operation of the remote communication system according to the embodiment of the present invention. First, the large screen terminal 10A existing in the base A, the large screen terminal 10B existing in the base B, and the communication terminal 20B existing in the base B are activated (S504). Then, the large screen terminal 10B and the communication terminal 20B notify the terminal management server 30 of the activation (S508). Subsequently, the large screen terminal 10A and the large screen terminal 10B start acquiring the bird's-eye view image (S512), start transmitting and receiving the bird's-eye view image (S516), and start displaying the received bird's-eye view image (S520).

After that, the terminal management server 30 instructs the communication terminal 20B to emit light (S524), and instructs the large screen terminal 10B to acquire the coordinates of the communication terminal 20B (S528). The light emitting unit 270 of the communication terminal 20B emits light based on the light emission instruction from the terminal management server 30 (S532), and the large screen terminal 10B specifies the terminal coordinates of the communication terminal 20B based on the detection of the light emission in the bird's-eye view image (S536). ).

Then, the large screen terminal 10B transmits the terminal coordinates of the specified communication terminal 20B to the terminal management server 30 (S540), and the distance calculation unit 332 of the terminal management server 30 transmits the large screen terminal 10B and the large screen terminal 10B based on the received terminal coordinates. The distance between the communication terminals 20B is calculated, and information indicating the distance between the large screen terminal 10B and the communication terminal 20B is stored in the terminal information storage unit 328 (S544).

After that, as shown in FIG. 13, when the large screen terminal 10A sends a connection request to the terminal management server 30 (S548), the connection destination determination unit 336 of the terminal management server 30 determines the connection destination related to the connection request (S548). S552). Here, consider the case where the communication terminal 20B is determined as the connection destination. In this case, the terminal management server 30 transmits a connection instruction to the large screen terminal 10A and the communication terminal 20B (S556).

Here, the terminal management server 30 performs different controls regarding the connection according to the positional relationship between the large screen terminal 10B and the communication terminal 20B. Specifically, when the distance between the communication terminal 20B and the large screen terminal 10B is less than the threshold value, the terminal management server 30 causes the large screen terminal 10A and the communication terminal 20B to start transmission / reception of voice (S560). The large screen terminal 10A and the communication terminal 20B start outputting the received voice (S564). At this time, the large screen terminal 10A and the communication terminal 20B do not transmit or receive video.

On the other hand, when the distance between the communication terminal 20B and the large screen terminal 10B is equal to or greater than the threshold value, the terminal management server 30 causes the large screen terminal 10A and the communication terminal 20B to start transmitting and receiving video and audio (S570). The large screen terminal 10A starts outputting the received video and audio, and the communication terminal 20B starts outputting the received audio and popping display of the received video (member video of the communication terminal 20B) (S574).

After that, when the large screen terminal 10A transmits a disconnection request to the terminal management server 30 (S584), the terminal management server 30 transmits a disconnection instruction to the communication terminal 20B and the large screen terminal 10A (S588, S592), and the communication terminal 20B and The connection between the large screen terminals 10A is disconnected (S596, S598).

<4. Action effect>
According to the embodiment of the present invention described above, various effects can be obtained. For example, a member who exists near the large screen terminal 10 can obtain a sense of presence like a dialogue through a window by interacting with a member of another base using the large screen terminal 10. Even when a member A who is away from the large screen terminal 10 interacts with a member B of another base, the image of the member A is displayed on the large screen terminal 10 of the other base in an appropriate size, so that the feeling of presence is felt. It is possible to reduce the decline.

<5. Application example>
The embodiment of the present invention has been described above. In the following, an application example of the embodiment of the present invention will be described with reference to FIGS. 14 and 15.

(Constitution)
FIG. 14 is an explanatory diagram showing the configuration of the terminal management server 31 according to the application example. As shown in FIG. 14, the terminal management server 31 according to the application example has a communication unit 324, a terminal information storage unit 328, a distance calculation unit 332, an image pickup direction analysis unit 334, a connection destination determination unit 336, and a connection control unit 342. .. Since the functions of the communication unit 324, the terminal information storage unit 328, the distance calculation unit 332, and the connection destination determination unit 336 are as described with reference to FIG. 7, detailed description thereof will be omitted here.

The image pickup direction analysis unit 334 analyzes the relationship between the image pickup direction by the large screen terminal 10 and the image pickup direction by the communication terminal 20 with respect to the large screen terminal 10 and the communication terminal 20 existing in the same base. The imaging direction analysis unit 334 performs the above analysis based on, for example, the correspondence between the object included in the bird's-eye view image of the base acquired by the large screen terminal 10 and the object included in the image acquired by the communication terminal 20. .. For example, the image pickup direction analysis unit 334 may be used to determine the position of the same object included in both the bird's-eye view image of the site acquired by the large screen terminal 10 and the image acquired by the communication terminal 20 in the bird's-eye view image of the site, and the communication terminal. Based on the relationship with the position in the image acquired by 20, the relationship between the image pickup direction by the large screen terminal 10 and the image pickup direction by the communication terminal 20 is analyzed. A specific example of the analysis will be described with reference to FIG.

FIG. 15 is an explanatory diagram showing a bird's-eye view image of the base acquired by the large screen terminal 10. As shown in FIG. 15, the bird's-eye view image of the base includes the door display E1 and the bookshelf display E2. The rectangular area shown by the broken line in FIG. 15 indicates the range of imaging by the communication terminal 20A # 1, and the image acquired by the communication terminal 20A # 1 also includes the door display E1 and the bookshelf display E2.

The imaging direction analysis unit 334 has a relationship with the positions of the door display E1 and the bookshelf display E2 in the bird's-eye view image of the base, and the positions of the door display E1 and the bookshelf display E2 in the image acquired by the communication terminal 20A # 1. Therefore, the relationship between the direction of image pickup by the large screen terminal 10 and the direction of image pickup by the communication terminal 20 is analyzed.

The connection control unit 342 controls the connection between the communication terminals 20 existing at different bases and the connection between the communication terminals 20 and the large screen terminals 10 existing at different bases. For example, regarding the connection between the large screen terminal 10A existing in the base A and the communication terminal 20B existing in the base B, the connection control unit 342 causes the large screen terminal 10A and the communication terminal 20B to transmit and receive voice. On the other hand, the connection control unit 342 determines whether or not to transmit / receive video between the large screen terminal 10A and the communication terminal 20B, in addition to the positional relationship between the communication terminal 20B and the large screen terminal 10B existing in the base B, and the imaging direction. Control is performed based on the relationship between the image pickup direction by the large screen terminal 10B analyzed by the analysis unit 334 and the image pickup direction by the communication terminal 20B.

In this regard, even if the distance between the communication terminal 20B and the large screen terminal 10B existing at the base B is less than the threshold value, when the member using the communication terminal 20B turns his back to the large screen terminal 10B, It is difficult for the member to have a dialogue with the member of the base A using the large screen terminal 10B. Therefore, in the connection control unit 342, the distance between the communication terminal 20B and the large screen terminal 10B existing in the base B is less than the threshold value, and the body of the member who uses the communication terminal 20B faces the large screen terminal 10B. In the case of presumed to be, since the large screen terminal 10B is used instead of the communication terminal 20B for the video, the video is not transmitted and received between the large screen terminal 10A and the communication terminal 20B. Here, since it is considered that the body orientation of the member using the communication terminal 20B is opposite to the direction of the image captured by the communication terminal 20B, the connection control unit 342 has a large screen analyzed by the image pickup direction analysis unit 334. The fact that the difference between the image pickup direction by the terminal 10B and the image pickup direction by the communication terminal 20B is less than the standard is treated as the body of the member using the communication terminal 20B facing the large screen terminal 10B.

On the other hand, when the distance between the communication terminal 20B and the large screen terminal 10B existing in the base B is equal to or larger than the threshold value, or when it is estimated that the body of the member who uses the communication terminal 20B does not face the large screen terminal 10B. In order to use the communication terminal 20B for the video, the video is also transmitted and received between the large screen terminal 10A and the communication terminal 20B.

(motion)
16 and 17 are explanatory views showing the operation of the remote communication system according to the application example. The processing of S504 to S544 of FIG. 16 is as described with reference to FIG. After S544, the terminal management server 31 requests video from the large screen terminal 10B and the communication terminal 20B (S545). The large screen terminal 10B and the communication terminal 20B transmit video to the terminal management server 31 in response to the request (S546). The terminal management server 31 analyzes the relationship between the image pickup direction by the large screen terminal 10B and the image pickup direction by the communication terminal 20B based on the images received from the large screen terminal 10B and the communication terminal 20B (S547).

After that, as shown in FIG. 17, when the large screen terminal 10A sends a connection request to the terminal management server 30 (S548), the connection destination determination unit 336 of the terminal management server 31 determines the connection destination related to the connection request (S548). S552). Here, consider the case where the communication terminal 20B is determined as the connection destination. In this case, the terminal management server 30 transmits a connection instruction to the large screen terminal 10A and the communication terminal 20B (S556).

Here, the terminal management server 31 controls different connections with respect to the connection according to the positional relationship between the large screen terminal 10B and the communication terminal 20B, and the relationship between the image pickup direction by the large screen terminal 10B and the image pickup direction by the communication terminal 20B. I do. Specifically, in the terminal management server 31, the distance between the communication terminal 20B and the large screen terminal 10B is less than the threshold value, and the difference between the image pickup direction by the large screen terminal 10B and the image pickup direction by the communication terminal 20B is If it is less than the standard, the large screen terminal 10A and the communication terminal 20B are started to send and receive voice (S562). The large screen terminal 10A and the communication terminal 20B start outputting the received voice (S566). At this time, the large screen terminal 10A and the communication terminal 20B do not transmit or receive video.

On the other hand, when the distance between the communication terminal 20B and the large screen terminal 10B is equal to or greater than the threshold value, or when the difference between the direction of imaging by the large screen terminal 10B and the direction of imaging by the communication terminal 20B is equal to or greater than the reference, terminal management The server 31 causes the large screen terminal 10A and the communication terminal 20B to start transmitting and receiving video and audio (S572). The large screen terminal 10A starts outputting the received video and audio, and the communication terminal 20B starts outputting the received audio and popping display of the received video (member video of the communication terminal 20B) (S576).

After that, when the large screen terminal 10A transmits a disconnection request to the terminal management server 31 (S584), the terminal management server 31 transmits a disconnection instruction to the communication terminal 20B and the large screen terminal 10A (S588, S592), and the communication terminal 20B and The connection between the large screen terminals 10A is disconnected (S596, S598).

<6. Hardware configuration>
The embodiment of the present invention has been described above. Information processing such as distance calculation and connection control described above is realized by the cooperation between the software and the hardware of the terminal management server 30 (terminal management server 31) described below.

FIG. 18 is a block diagram showing a hardware configuration of the terminal management server 30. The terminal management server 30 includes a CPU (Central Processing Unit) 371, a ROM (Read Only Memory) 372, a RAM (Random Access Memory) 373, and a host bus 374. The terminal management server 30 includes a bridge 375, an external bus 376, an interface 377, an input device 378, a display device 379, an audio output device 380, a storage device (HDD) 381, a drive 382, and a network. It can be equipped with an interface 383.

The CPU 371 functions as an arithmetic processing unit and a control device, and controls the overall operation in the terminal management server 30 according to various programs. Further, the CPU 371 may be a microprocessor. The ROM 372 stores programs, calculation parameters, and the like used by the CPU 371. The RAM 373 temporarily stores a program used in the execution of the CPU 371, parameters that appropriately change in the execution, and the like. These are connected to each other by a host bus 374 composed of a CPU bus or the like. By the cooperation between the CPU 371, the ROM 372 and the RAM 373 and the software, functions such as the distance calculation unit 332, the image pickup direction analysis unit 334, the connection destination determination unit 336, and the connection control unit 340 can be realized.

The host bus 374 is connected to an external bus 376 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 375. It is not always necessary to separately configure the host bus 374, the bridge 375, and the external bus 376, and these functions may be implemented in one bus.

The input device 378 is an input means for a member to input information such as a mouse, a keyboard, a touch panel, a button, a microphone, a sensor, a switch, and a lever, and an input signal generated based on the input by the member and output to the CPU 371. It may be composed of a control circuit or the like. The operation of the terminal management server 30 is usually performed by voice, but by operating the input device 378, various data can be input to the terminal management server 30 and a processing operation can be instructed. ..

The display device 379 includes, for example, a display device such as a CRT (Cathode Ray Tube) display device, a liquid crystal display (LCD) device, a projector device, an OLED (Organic Light Emitting Diode) device, and a lamp. The display device 379 corresponds to, for example, the video output unit 240. Further, the audio output device 380 includes an audio output device such as a speaker and headphones. The audio output device 380 corresponds to, for example, the audio output unit 250.

The storage device 381 is a data storage device configured as an example of the storage unit of the terminal information storage unit 328 according to the present embodiment. The storage device 381 may include a storage medium, a recording device for recording data on the storage medium, a reading device for reading data from the storage medium, a deleting device for deleting data recorded on the storage medium, and the like. The storage device 381 is composed of, for example, an HDD (Hard Disk Drive), an SSD (Solid Storage Drive), or a memory having an equivalent function. The storage device 381 drives the storage and stores programs and various data executed by the CPU 371.

The drive 382 is a reader / writer for a storage medium, and is built in or externally attached to the terminal management server 30. The drive 382 reads the information recorded in the removable storage medium 24 such as the mounted magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, and outputs the information to the RAM 373 or the storage device 381. The drive 382 can also write information to the removable storage medium 24.

The network interface 383 is, for example, a communication interface configured by a communication device or the like for connecting to the network 12. Further, the network interface 383 may be a wireless LAN (Local Area Network) compatible terminal or a wire terminal that performs wired communication.

<7. Supplement>
Although the preferred embodiments of the present invention have been described in detail with reference to the accompanying drawings, the present invention is not limited to these examples. It is clear that a person having ordinary knowledge in the field of technology to which the present invention belongs can come up with various modifications or modifications within the scope of the technical ideas described in the claims. , These are also naturally understood to belong to the technical scope of the present invention.

For example, in the above, the example in which the large screen terminal 10 and the communication terminal 20 existing in different bases are connected has been described, but the embodiment of the present invention can also be applied to the connection between the communication terminals 20 existing in different bases. Is. That is, when the communication terminal 20 existing in the first base is located far away from the large screen terminal 10 existing in the first base, the communication terminal 20 existing in the second base exists in the first base. On the bird's-eye view image of the first base acquired by the large screen terminal 10, the face image of the member who uses the communication terminal 20 existing in the first base may be displayed in POPUP.

In addition, each step in the processing of the remote communication system of the present specification does not necessarily have to be processed in chronological order in the order described as a sequence diagram. For example, each step in the processing of the remote communication system may be processed in an order different from the order described as the sequence diagram, or may be processed in parallel.

Further, the hardware such as the CPU, ROM, and RAM built in the large screen terminal 10, the communication terminal 20, and the terminal management server 30 has the same configuration as the above-mentioned large screen terminal 10, the communication terminal 20, and the terminal management server 30. It is also possible to create a computer program to bring out the functions of. A storage medium for storing the computer program is also provided.

10 Large screen terminal 12 Network 20 Communication terminal 30, 31 Terminal management server 130 Video output unit 140 Imaging unit 150 Audio input / output unit 160 Communication unit 170 Information storage unit 180 Control unit 220 Imaging unit 224 Communication unit 230 Sound collection unit 240 Video output Unit 250 Audio output unit 260 Information storage unit 270 Light emitting unit 280 Control unit 324 Communication unit 328 Terminal information storage unit 332 Distance calculation unit 334 Imaging direction analysis unit 336 Connection destination determination unit 340, 342 Connection control unit

Claims (10)

A management device connected to a first image pickup device for acquiring a first image and a second image pickup device for acquiring a second image.
The second image is transmitted to the communication partner device that receives the first image from the first image pickup device and displays the first image , at the position of the first image pickup device and the first image. It is equipped with a control unit that controls based on the distance between the position and the position of the image pickup device of 2.
The second image pickup device is an image pickup device associated with a user who communicates via the communication partner device.
Management device. Based on the fact that the distance between the position of the first image pickup device and the position of the second image pickup device is equal to or greater than the threshold value, the control unit outputs the second image to the second image pickup device. The management device according to claim 1, which is transmitted to the communication partner device. The control unit further transfers the second image pickup device to the communication partner device based on the relationship between the image pickup direction by the first image pickup device and the image pickup direction by the second image pickup device. The management device according to claim 1 or 2, which controls the transmission of the second video. The control unit attaches the second image pickup device to the second image pickup device based on the difference between the image pickup direction by the first image pickup device and the image pickup direction by the second image pickup device being less than the reference. The management device according to claim 3, wherein the image of the above is transmitted to the second image pickup device. The control unit determines whether or not the difference between the image pickup direction by the first image pickup device and the image pickup direction by the second image pickup device is less than the reference, the object included in the first image. The management device according to claim 3 or 4, wherein the management device is determined based on the correspondence with the object included in the second video. The second image pickup device acquires sound and receives sound.
The management device according to any one of claims 1 to 5, wherein the control unit causes the second image pickup device to transmit the voice to the communication partner device. The control unit calculates the distance between the position of the first image pickup device and the position of the second image pickup device based on the position of the second image pickup device in the first image. Item 2. The management device according to item 2. A method performed by a management device connected to a first image pickup device for acquiring a first image and a second image pickup device for acquiring a second image.
The second image is transmitted to the communication partner device that receives the first image from the first image pickup device and displays the first image , at the position of the first image pickup device and the first image. Control based on the distance between the position of the image pickup device 2 and
Including
The method, wherein the second image pickup device is an image pickup device associated with a user who communicates via the communication partner device. Computer,
A management device connected to a first image pickup device for acquiring a first image and a second image pickup device for acquiring a second image.
The second image is transmitted to the communication partner device that receives the first image from the first image pickup device and displays the first image , at the position of the first image pickup device and the first image. It is equipped with a control unit that controls based on the distance between the position and the position of the image pickup device of 2.
The second image pickup device is an image pickup device associated with a user who communicates via the communication partner device.
A program to function as a management device. A management system including a management device connected to a first image pickup device for acquiring a first image and a second image pickup device for acquiring a second image.
The second image is transmitted to the communication partner device that receives the first image from the first image pickup device and displays the first image , at the position of the first image pickup device and the first image. It is equipped with a control unit that controls based on the distance between the position and the position of the image pickup device of 2.
The second image pickup device is an image pickup device associated with a user who communicates via the communication partner device.
Management system.

Images