JP4485933B2 - Remote communication system, direction presentation method and direction presentation program used in this system - Google Patents

Description

  The present invention relates to a remote communication system that supports, for example, work of field workers from a remote location using image communication, and a direction presentation method and a direction presentation program used in this system.

  For example, when a conversation is conducted between a field worker and their supporter using communication, the listener understands the intention of the speaker by grasping the shape and location of the target object. Is very important above. However, it is generally difficult for a speaker to verbally and accurately convey the shape and location of an object.

Therefore, a camera and a laser pointer are installed on the listener side, the real space state on the listener side is imaged by the camera, and this is sent to the speaker side by voice image communication and displayed on the monitor. A technique has been proposed in which a listener can directly view an object and grasp its shape and the like by pointing the object or a range including the object by remotely operating the indicated position of the laser pointer (for example, , See Patent Document 1).
Using this technology, it is possible to easily match each other's visual objects or ranges between the speaker and the listener, thereby reducing the speaker's explanation burden and enabling smoother communication for support. Become.
JP 2002-125024 A

  However, with the techniques described above, if the speaker moves the pointing position to another object or range while the listener is looking away from the pointing object or range, the listener must re-find the pointing position. In the meantime, there is a problem that communication is forced to be interrupted. In general, the imaging range of the camera is limited to a part of the real space on the listener side. For this reason, if the listener turns in a different direction and the field of view deviates from the imaging range of the camera, the speaker cannot grasp the listener's field of view from the monitor screen, and in this case, communication cannot be performed smoothly.

  The present invention has been made paying attention to the above circumstances, and the purpose of the present invention is to make it possible to easily match each other's visual recognition target between a speaker and a listener, thereby making communication smoother. An object of the present invention is to provide a remote communication system that can be used and a direction presentation method and program used in this system.

In order to achieve the above object, according to a first aspect of the present invention, a first speaker and a second speaker existing in different spaces photograph an arbitrary object in a space where the first speaker exists by a photographing means. In the remote communication system in which communication is performed while viewing both of the above objects by displaying the captured image information on the display means installed in the space where the second speaker exists, the first speaker exists. The shooting range of the shooting means and the viewing range of the first speaker in the space to be detected are respectively detected. Then, based on the detected imaging range and visual range, to determine the direction of the shooting range of the visual range, and calculates the distance between the shooting scope and visual range, it is the the determined direction and the calculated The image information representing the distance is generated, and the generated image information is projected within the visual range of the first speaker.

  Therefore, according to the first invention, when the first speaker is viewing a range different from the imaging range presented to the second speaker, the direction from the viewing range to the imaging range is changed. Representing image information is generated and this image information is projected onto the viewing range of the first speaker. For this reason, the first speaker can easily recognize the direction of the imaging range viewed by the second speaker from the image projected in front of him, and his / her position based on the recognition result. Alternatively, by changing the posture, it is possible to smoothly match its own viewing range with the imaging range that the second speaker is viewing.

According to the first invention , the first speaker can recognize the distance from the current viewing range to the imaging range along with the direction of the imaging range viewed by the second speaker. It becomes possible. Therefore, the first speaker can quickly and accurately change his / her position or posture based on the recognition result, and thereby the imaging range in which the second speaker is viewing his / her viewing range. It becomes possible to make it correspond even more smoothly.

On the other hand, in order to achieve the above-mentioned object, the second invention provides a photographing means for photographing an arbitrary object in a space in which the first speaker and the second speaker existing in different spaces exist. In the remote communication system in which communication is performed while visually observing the target together by displaying the captured image information on the display means installed in the space where the second speaker is present. The photographing range of the photographing means and the visual range of the first speaker are detected in the space where the speaker is present. Then, based on the detected imaging range and visual range, to determine the direction of the visual range of the shooting range, and calculates the distance between the shooting scope and visual range, it is the the determined direction and the calculated Image information representing the measured distance is generated, and the generated image information is displayed on the display unit together with the captured image information obtained by the imaging unit.

  Therefore, according to the second invention, when the photographing means is photographing a range different from the visual range of the first speaker, the direction from the photographing range to the visual range of the first speaker is expressed. Image information is generated, and this image information is displayed on the display means being visually recognized by the second speaker. For this reason, the second speaker can easily recognize the direction of the visual range of the first speaker from the displayed image information. And based on this recognition result, for example, by remotely controlling the attitude of the imaging means, the imaging range of the imaging means can be smoothly matched with the visual range of the first speaker.

In addition, according to the second invention , the second speaker can measure the distance from the current shooting range by the shooting means to the first speaker's viewing range simultaneously with the direction of the first speaker's viewing range. Can be recognized. Therefore, the second speaker can quickly and accurately change the posture of the photographing means based on the recognition result, and thereby the photographing range of the photographing means is further increased with respect to the visual range of the first speaker. Smooth matching is possible.

Further, in the first and second inventions, the detection means projects a shooting range presentation image representing a shooting range by the shooting means into a space where the first speaker exists, and the first speaker. Means for projecting a visual range presenting image representing the visual range of the first speaker, a omnidirectional photographing means for photographing the space where the first speaker is present, and the omnidirectional photographing. It is also characterized by comprising means for detecting the photographing range presentation image and the visual range presentation image from the photographed image information obtained by the means.
In this way, it is possible to reliably detect the position of the first speaker's viewing range and the shooting range of the shooting means anywhere in the space where the first speaker exists. .

In short, in the present invention, the photographing range of the photographing means and the visual range of the first speaker in the space where the first speaker is present are detected, respectively, and based on the detected photographing range and visual range, together determine the direction of the imaging range to visually range, by calculating the distance between the shooting scope and visual range, the image information representing the the determined direction and the calculated distance in the visual range of the first speaker Or is displayed on the display unit together with photographed image information obtained by the photographing unit.

  Therefore, according to the present invention, it is possible to easily match the visual recognition target between the speaker and the listener, and thereby use the remote communication system that can perform communication more smoothly. A direction presentation method and a direction presentation program can be provided.

Hereinafter, an embodiment of a remote communication system according to the present invention will be described with reference to the drawings. In this embodiment, a description will be given of a so-called remote work support system in which a support worker supports the work of a field worker from a remote location via a communication network.
FIG. 1 is a diagram for explaining an outline of a remote work support system according to this embodiment. As shown in the figure, this system is a system in which a site-side system 1 and a support-side system 2 are connected via a communication network NW.

  The on-site system 1 is provided with a photographing apparatus CM. The photographing apparatus CM is configured to be able to change the photographing posture, photographs an arbitrary range Ec of the work space on site, and transmits the photographed image information to the support system 2 via the communication network NW. Used for. A projection device PJ is attached to, for example, the head of a field worker. The projection device PJ is used, for example, to project image information and character information for work support sent from the support side system 2 onto the field worker's visual range Ep.

  On the other hand, a support terminal MT is installed in the support side system 2. The support terminal MT is configured by, for example, a workstation or a personal computer, and receives photographed image information transmitted from the photographing apparatus CM of the field side system 1 via the communication network NW and displays it on the display unit. Further, the support information input by the support personnel while viewing the displayed photographed image information is transmitted to the projection device PJ of the on-site system 1 via the communication network NW, and thereby the viewing range Ep for the on-site worker. Project.

  Further, each of the on-site system 1 and the support-side system 2 is provided with a voice call function (not shown), and a voice call can be made between a field worker and a support person using these voice call functions. It has become. This voice call function is provided, for example, in a voice communication terminal carried by a field worker in the field side system 1, and is provided, for example, in a support terminal MT in the support side system 2.

By the way, in such a remote work support system, although the voice call is performed between the field worker and the support staff, the field worker's visual range Ep and the photographing range Ec of the photographing apparatus CM always coincide with each other. Is not limited.
Therefore, the remote operation support system according to this embodiment has a direction presentation function for notifying the other side support personnel and the field worker of the viewing direction of the field worker and the photographing direction by the photographing apparatus CM, respectively.

  For example, as shown in FIG. 2, this direction presentation function is realized by projecting an image Ip of the direction presentation content on the field worker's visual range Ep by the projection device PJ in the site side system. On the other hand, the support side system 2 is realized by displaying the image Ic of the direction presentation content on the display unit 22 of the support terminal MT so as to overlap the captured image information GS of the imaging apparatus CM. Both the images Ip and Ic of the direction presentation content are obtained by displaying black circles 32 and 42 in the horizontally long display ranges 31 and 41 having an elliptical shape as shown in the figure, and the black circles 32 in the display ranges 31 and 41 are displayed. , 42, the photographing direction Fc by the photographing device CM and the visual direction Fp of the field worker are respectively presented.

FIG. 3 is a block diagram showing a specific configuration for realizing the remote operation support system according to this embodiment.
First, the on-site system 1 is provided with an imaging device CM, a projection device PJ, an omnidirectional camera 15, a display processing unit 18, and a network interface (network I / F) 19. The photographing apparatus CM includes a camera 11, a camera driving unit 12 that changes the photographing posture of the camera 11, a drive control unit 13, and a photographing range presentation device 14. Among them, the drive control unit 13 controls the drive of the camera drive unit 12 in accordance with the attitude control signal sent from the support terminal MT. The photographing range presenting device 14 is composed of, for example, a laser pointer or a projector and operates integrally with the camera 11. Then, a frame pattern indicating the shooting range of the camera 11 is projected onto the shooting target Q.

  The projection device PJ includes a projector 16 and a projection range presenter 17. The projector 16 projects image information or character information onto a space within the field worker's visual range Ep. The projection range presenter 17 is composed of a laser pointer, for example, and projects a frame pattern representing the projection range Ep by the projector 16 onto the space. Note that the frame pattern may be projected from the projector 16 so as to overlap image information or character information. In this case, the projection range presenter 17 can be dispensed with.

  The omnidirectional camera 15 images the work space on the site in all directions. FIG. 15 is a schematic configuration diagram of the omnidirectional camera 15. The omnidirectional camera 15 includes a conical mirror 51 that can capture the entire periphery, and a light receiving unit 52 that receives the entire peripheral image reflected by the conical mirror 51. Then, the omnidirectional panoramic image is obtained by panning the received omnidirectional image by the angle α of the conical mirror 51 as shown in FIG. Note that the omnidirectional camera does not necessarily have to be capable of photographing over 360 °, and may be capable of photographing over, for example, 180 °. In short, the imaging range of the omnidirectional camera can be arbitrarily selected according to the range of the work space of the field system.

  The display processing unit 18 supplies display information for work support transmitted from the support terminal MT to the projector 16 for projection. When the center coordinate value of the imaging range Ec of the camera 11 and the center coordinate value of the projection range Ep of the projector 16 are sent from the support terminal MT, the projection range of the projector 16 is based on these center coordinate values. A direction presentation content image Ip representing the direction from Ep to the imaging range Ec of the camera 11 is generated. The generated direction presentation content image Ip is combined with the display information for work support, and then supplied to the projector 16 for projection.

  On the other hand, a support terminal MT installed in the support system 2 includes an input unit 21 using a keyboard, a mouse, a joystick, etc., a display unit 22 using a liquid crystal display (LCD), an image switching unit 26, A network interface (network I / F) 27 is provided, and an input data processing unit 23, a recognition unit 24, and a display processing unit 25 are further provided. Among these, the input data processing unit 23, the recognition unit 24, and the display processing unit 25 are all realized by causing a microcomputer to execute a program.

  The input data processing unit 23 generates camera drive control information for controlling the posture of the camera according to the operation when the support person performs a drive operation of the camera in the input unit 21. And this camera drive control information is transmitted to the drive control part 13 of the field side system 1 via the communication network NW.

  The recognizing unit 24 takes in the captured image information (panoramic image information) of the omnidirectional camera 15 transmitted from the field side system 1, and in this panoramic image information, an image pattern representing the imaging range Ec of the camera 11, and the projector 16. Each of the image patterns representing the projection range Ep is recognized. Then, the center coordinate value CO (X1, Y1) of the image pattern representing the recognized imaging range Ec of the camera 11 is calculated, and the center coordinate value PO (X2, Y2) of the image pattern representing the projection range Ep of the projector 16 is calculated. ) Is calculated. Then, the calculated center coordinate values CO (X1, Y1) and PO (X2, Y2) are given to the display processing unit 25 and transmitted to the display processing unit 18 of the site system 1 via the communication network NW. .

  The display processing unit 25 represents a direction representing the direction from the imaging range Ec of the camera 11 to the projection range Ep of the projector 16 based on the given center coordinate values CO (X1, Y1) and PO (X2, Y2). A presentation content image Ic is generated. Then, the generated direction presentation content image Ic is supplied to the display unit 22 via the image switching unit 26 and displayed.

  The image switching unit 26 selectively switches the captured image information from the camera 11 and the captured image information (panoramic image) from the omnidirectional camera 15 and supplies it to the display unit 22. It is switched when the personnel perform the switching operation. Generally, it is fixed to the camera 11 side in a steady state, and is switched to the omnidirectional camera 15 side only when necessary, for example, when a support person wants to check the state of the entire work space or to check the operation of the omnidirectional camera 15.

  The network I / Fs 19 and 27 transmit and receive captured image information, display information, and control information between the site-side system 1 and the support-side system 27 via the communication network NW. The communication network NW includes, for example, an IP (Internet Protocol) network and a plurality of access networks for accessing the IP network. Examples of the access network include a fixed network using DSL (Digital Subscriber Line) and an optical transmission line, a wireless LAN network, a PHS (Personal Handyphone System) (registered trademark) network, a PDC (Personal Digital Cellular) network, and an IMT2000 (International Mobile A mobile communication network such as the Telecommunication 2000) network is assumed.

Next, the operation of the remote work support system configured as described above will be described. FIG. 4 is a flowchart showing the control procedure and control contents.
In step 4a, the support terminal MT monitors the input of the end event by the support personnel. In this state, if there is no input of an end event, the process proceeds to step 4b, where the image switching unit 26 selects photographed image information from the omnidirectional camera 15, or selects photographed image information from the camera 11. Judge whether it is. If the result of this determination is that the image switching unit 26 has selected photographed image information from the camera 11, the process proceeds to step 4c where display processing of the direction presentation content images Ip and Ic is executed. Then, following this display process, the input operation of the input unit 21 is monitored in step 4d. When the input operation is detected, the process proceeds to step 4e, where the drive control process of the camera 11 corresponding to the input operation is executed.

  On the other hand, when the image switching unit 26 has selected the captured image information from the omnidirectional camera 15, the display processing of the direction presentation content images Ip and Ic is omitted, and the process proceeds to step 4d. In step 4d, the input operation of the input unit 21 is monitored. When the input operation is detected, the drive control process of the camera 11 corresponding to the input operation is executed in step 4e.

Now, the display processing of the direction presentation content images Ip and Ic is executed as follows. FIG. 5 is a flowchart showing the control procedure and control contents.
In the on-site system 1, an image pattern representing the imaging range Ec of the camera 11 is projected onto the work space by the imaging range presentation device 14, and an image pattern representing the projection range Ep of the projector 16 is projected by the projection presentation device 17. . Then, the work space on which these image patterns are projected is photographed by the omnidirectional camera 15, and the photographed image information, that is, panoramic image information, is transmitted to the support terminal MT of the support side system 2 via the communication network NW. .

  The support terminal MT first acquires panoramic image information photographed by the omnidirectional camera 15 from the field side system 1 via the communication network NW. Then, the recognition unit 24 performs pattern recognition processing on the acquired panoramic image information, for example, thereby recognizing the shooting range Ec of the camera 11 in step 5a and recognizing the projection range Ep of the projector 16 in step 5b. .

  Subsequently, in steps 5c and 5d, it is determined whether or not the imaging range Ec of the camera 11 recognized by the recognition unit 24 and the projection range Ep of the projector 16 overlap. When it is determined that they overlap, information to that effect is notified to the display processing unit 18 of the on-site system 1 and the display processing unit 25 of the support side system 2, respectively. The display processing units 18 and 25 display the direction presentation content images Ip and Ic indicating that the shooting range Ec of the camera 11 and the projection range Ep of the projector 16 overlap based on the information notified in step 5j. Generate. Then, the generated direction presentation content image Ip is supplied to the projector 16 and projected onto the projection range Ep, and the generated direction presentation content image Ic is supplied to the display unit 22 for display. The direction presentation content images Ip and Ic generated at this time are obtained by arranging a black circle 32 at the center of the ellipse range 31 as indicated by a two-dot chain line 32 'in FIG.

  On the other hand, it is assumed that the recognized shooting range Ec of the camera 11 and the projection range Ep of the projector 16 do not overlap as shown in FIG. In this case, the support terminal MT proceeds to step 5e to calculate the center coordinate value CO (X1, Y1) of the image pattern representing the imaging range Ec of the camera 11 recognized by the recognition unit 24 and the projector 16 The center coordinate value PO (X2, Y2) of the image pattern representing the projection range Ep is calculated.

The calculation of the central coordinate values CO (X1, Y1) and PO (X2, Y2) is performed as follows.
That is, first, the imaging range Eo of the omnidirectional camera 15 is represented by spatial coordinate systems X and Y as shown in FIG. In such panoramic image information represented by the spatial coordinate systems X and Y, it is assumed that the imaging range Ec of the camera 11 and the projection range Ep of the projector 16 are respectively captured at the positions shown in FIG.

At this time, the horizontal length of the panoramic image is LX, the vertical length is LY, the distance from the reference point of the spatial coordinates to the end of the imaging range Ec of the camera 11 is L1, and the projection range Ep of the projector The distance to the end of the camera 11 is L2, the horizontal length of the imaging range Ec of the camera 11 is LC, and the horizontal length of the projection range Ep of the projector is LP. X1 of the central coordinate value CO (X1, Y1) and X2 of the central coordinate value PO (X2, Y2) of the projection range Ep of the projector 16 are respectively calculated by the following equations.
X1 = L1 + (LC / 2)
X2 = L2 + (LP / 2)
Further, Y1 of the center coordinate value CO (X1, Y1) of the imaging range Ec of the camera 11 and Y2 of the center coordinate value PO (X2, Y2) of the projection range Ep of the projector 16 are calculated in the same manner.

  Thus, the center coordinate value CO (X1, Y1) of the imaging range Ec of the camera 11 and the center coordinate value PO (X2, Y2) of the projection range Ep of the projector 16 are calculated. FIG. 12B shows an example of the calculation result. Each length represented on the spatial coordinates represents the size of the image, and is represented by a pixel here. However, if the unit representing the size of the image is used uniformly, any unit is used. There is no problem with using.

  Next, the support terminal MT proceeds to step 5f, and supplies the center coordinate values CO (X1, Y1) and PO (X2, Y2) calculated by the recognition unit 24 to the display processing unit 25 and performs communication. The data is transmitted to the display processing unit 18 of the site system 1 via the network NW. When the center coordinate values CO (X1, Y1) and PO (X2, Y2) are received, the display processing unit 18 of the site side system 1 generates the direction presentation content image Ip in step 5g and supplies it to the projector 16. Further, the display processing unit 25 of the support side system generates the direction presentation content image Ic in step 5h and supplies it to the display unit 22.

  Hereinafter, a method of generating the direction presentation content images Ip and Ic will be described with reference to FIG. First, the display position and size of the direction presentation content image Ip in the projection range Ep of the projector 16 and the display position and size of the direction presentation content image Ic in the camera image display area of the display unit 22 are set in advance.

  For example, as shown in FIGS. 13A and 13B, the display position of the direction presentation content image Ip is set at the center upper end of the projection range Ep of the projector 16. In addition, the shape of the direction presentation content image Ip is set to an elliptical shape imitating human eyes. The direction presentation content image Ip is drawn in white within an elliptical range 31, and a black circle 32 imitating the black eyes of a human eye is placed therein. Then, the drawing is performed so that the center 33 of the black circle 32 moves along the elliptical locus 34 set in the elliptical range 31 according to the presentation direction. FIG. 14 shows an example of this. The center 33 of the black circle 32 is the center of the ellipse drawn in white (intersection of the major axis and minor axis) O (Ox, Oy) and the coordinates of the camera 11 photographing range Ec. The straight line connecting the center CO (X1, Y1) and the elliptical trajectory 34 are drawn so as to be located at the intersection.

  And the direction presentation content image Ip produced | generated as mentioned above is projected on the visual field Ep of a field worker by the projector 16 in step 5i. FIG. 9A shows an example of the display result, and shows a case where the imaging range Ec of the camera 11 is located in the right direction of the field worker's visual range Ep.

  Therefore, the field worker can grasp the direction of the shooting range Ec of the camera 11 from the direction presentation content image Ip. Then, if the direction of the head of his / her head is rotated to the right or the whole body is translated to the right, the own viewing range Ep can be matched with the shooting range Ec of the camera 11. FIG. 9B shows an example of a change in the direction presentation content image Ip accompanying the movement.

  At this time, in the support terminal MT of the support side system 2, the display processing unit 25 generates a direction presentation content image Ic in which the position of the black circle 32 is located to the left, and the generated direction presentation content image Ic is It is displayed on the display screen GS of the display unit 22. FIG. 10A shows an example of the display result. In this state, when the field worker rotates the head in the right direction as described above or translates the whole body in the right direction, the direction presentation content image Ic is as shown in FIG. 10B. To change.

  On the other hand, when the direction presentation content image Ic in which the black circle 32 is located on the left side of the elliptical range 31 is displayed on the display unit 22 of the support terminal MT, for example, the support staff presents the direction. It can be understood from the content image Ic that the field worker's visual range Ep is in the left direction of the photographing range Ec of the camera 11.

  When the support staff operates the joystick of the input unit 21 in this state, the support terminal MT shifts from step 4d to step 4e as shown in FIG. 4, where the posture control processing of the camera 11 is performed as follows. Execute. FIG. 6 is a flowchart showing the control procedure and control contents.

  That is, when the joystick of the input unit 21 is operated, camera control information is generated in the input data processing unit 25 according to the operation direction and the operation amount. This camera control information is transmitted to the drive control unit 13 of the site-side system 1 through the communication network NW in step 6a. The drive control unit 13 generates a drive signal for variably controlling the posture of the camera based on the transmitted camera control information, and supplies this drive signal to the camera drive unit 12 (step 6b). In step 6c, the camera driving unit 12 operates according to the driving signal to change the posture of the camera 11.

Thus, the shooting direction of the camera 11 moves to the left, and as a result, the shooting range Ec of the camera 11 coincides with the field worker's visual range Ep. FIG. 8B shows an example of the change in the direction presentation content image Ic accompanying the change in the posture of the camera 11.
At this time, in the on-site system 1, the direction presentation content image Ip changes from the state shown in FIG. 7A to the state shown in FIG. 7B by the left turn operation A of the camera 11 by the support staff.

  As described above, in this embodiment, in the site-side system 1, an image representing the imaging range Ec of the camera 11 is projected onto the work space by the imaging range presenting device 14, and an image showing the visual range Ep of the field worker. Is projected by the projection range presenter 17, the work space on which these images are projected is photographed by the omnidirectional camera 15, and the photographed image information is transmitted to the support side system 2 via the communication network NW. In the support-side system 2, the recognition unit 24 recognizes the shooting range Ec of the camera 11 and the visual range Ep of the field worker from the transmitted captured image information of the omnidirectional camera 15 and calculates the coordinate center thereof. . Based on the calculation result, the display processing unit 25 generates the direction presentation content image Ic indicating the direction from the shooting range Ec of the camera 11 to the visual field Ep of the field worker, and the generated direction presentation content image. Ic is displayed on the display unit 22. At the same time, the calculation result of the coordinate center is transmitted to the display processing unit 18 of the site-side system 1 via the communication network NW, and the display processing unit 18 uses the field worker's visual range Ep to the imaging range Ec of the camera 11. The direction presentation content image Ip indicating the direction to the direction is generated, and the generated direction presentation content image Ip is projected by the projector 17 onto the visual field Ep of the field worker.

  Accordingly, the field worker can smoothly grasp the direction of the shooting range Ec of the camera 11 from the direction presenting content image Ip projected on his / her viewing range Ep, and thereby, for example, the head direction of his / her head is set to the right direction. By rotating or translating the whole body in the right direction, the own viewing range Ep can be quickly matched with the shooting range Ec of the camera 11.

  Further, the support personnel can smoothly grasp the direction of the visual field Ep of the field worker from the direction presenting content image Ic displayed on the display unit 22, and thereby, for example, remotely control the posture of the camera 11. Thus, the photographing range Ec of the camera 11 can be quickly matched with the visual field Ep of the field worker.

  Further, in this embodiment, since the shooting range Ec of the camera 11 and the visual range Ep of the field worker are recognized based on the imaged image information of the omnidirectional camera 15, the field worker can identify which of the workspaces. Regardless of which direction the camera 11 is facing, and which direction the camera 11 is shooting, the direction can be reliably detected and presented.

  The present invention is not limited to the above embodiment. For example, based on the coordinate range of the shooting range Ec of the camera 11 and the visual range Ep of the field worker obtained from the captured image information of the omnidirectional camera 15, the shooting range Ec of the camera 11 and the visual range Ep of the field worker The distance is calculated. Then, a direction presentation content image representing the direction and the calculated distance may be generated, projected, and displayed. For example, when generating the direction presentation content images Ip and Ic imitating human eyes as shown in FIG. 13 and the like, the distance from the center of the elliptical range 31 to the center of the black circle 32 is set to the calculated distance. The distance is expressed by setting in proportion. The direction is the same as that in the above embodiment.

  In this way, both the field worker and the support staff can recognize the distance to the shooting range Ec of the camera 11 and the distance to the field worker's visual range Ep simultaneously with the viewing direction. It is possible to quickly and accurately correct the worker's visual range Ep and the photographing range Ec of the camera 11.

  In the above embodiment, the direction presentation content images Ic and Ip imitating human eyes are generated, projected, and displayed. However, other marks such as arrows may be used, and a direction presentation message using characters may be displayed. It may be generated and projected and displayed. Furthermore, in the said embodiment, although direction presentation content image Ic, Ip was projected and displayed in both the field side system 1 and the support side system 2, you may make it project or display only by either one.

  Furthermore, as a means for detecting the visual range of the field worker, for example, a direction sensor using a position sensor or a gyroscope is attached to the field worker, and the visual direction of the field worker is detected by this direction sensor. The detection information may be transmitted to the support system via a communication network.

  Further, in the embodiment, the support terminal MT of the support system 2 recognizes the imaging range Ec of the camera 11 and the visual field Ep of the field worker. For example, an information terminal carried by the field worker, You may make it recognize in an imaging device or a projection device. Further, the direction presentation content images Ic and Ip are not generated separately by the site-side system 1 and the support-side system 2, but may be generated comprehensively by either the site-side system 1 or the support-side system 2. Good.

  In addition, the purpose and type of remote communication, the system configuration, the type and configuration of the imaging device, the type and configuration of the projection device, the type and configuration of the communication network, the direction determination means, and the like are variously within the scope of the present invention. It can be implemented with deformation.

  In short, the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. Further, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, you may combine suitably the component covering different embodiment.

1 is a schematic configuration diagram showing an embodiment of a remote communication system according to the present invention. The figure for demonstrating the principal part of the system shown in FIG. The block diagram which shows an example of a structure which implement | achieves the system shown in FIG. The flowchart which shows the procedure and content of direction presentation control by the system shown in FIG. The flowchart which shows the procedure and content of a display process among the direction presentation controls shown in FIG. The flowchart which shows the procedure and content of a camera drive control process among the direction presentation controls shown in FIG. The figure for demonstrating an example of direction presentation operation | movement by the site side of the system shown in FIG. The figure for demonstrating an example of the direction presentation operation | movement by the side of the support by the system shown in FIG. The figure for demonstrating another example of direction presentation operation | movement by the site side of the system shown in FIG. The figure for demonstrating another example of direction presentation operation | movement by the support side of the system shown in FIG. The figure for demonstrating the imaging | photography range of the omnidirectional camera in the system shown in FIG. The figure for demonstrating the method of calculating the coordinate position of a camera imaging | photography range and a projector projection range from the image imaged with the omnidirectional camera shown in FIG. The figure for demonstrating the operation | movement which produces | generates the projection image and display image of direction presentation content based on the coordinate value calculated by the method shown in FIG. The figure for demonstrating the setting method of the position of the black circle in the projection image of the direction presentation content produced | generated by FIG. 13, and a display image. The figure for demonstrating schematic structure of the omnidirectional camera used with the system shown in FIG. The figure for demonstrating the principle of operation of the omnidirectional camera used with the system shown in FIG.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Site side system, 2 ... Support side system, NW ... Communication network, CM ... Imaging device, PJ ... Projection device, MT ... Supporting terminal, 11 ... Camera, 12 ... Camera drive part, 13 ... Drive control part, 14 DESCRIPTION OF SYMBOLS ... Shooting range presentation machine, 15 ... Omni-directional camera, 16 ... Projector, 17 ... Projection range presentation machine, 18 ... Display processing part, 19 ... Network system I / F of field system, 21 ... Input part, 22 ... Display part, 23 ... input data processing unit, 24 ... recognition unit, 25 ... display processing unit, 26 ... image switching unit, 27 ... network I / F of support side system.

Claims (6)

A first speaker and a second speaker existing in different spaces photograph an arbitrary object in the space where the first speaker exists by photographing means, and the second speaker captures the captured image information. In a remote communication system that communicates while viewing both of the above objects by displaying on a display means installed in an existing space,
Detecting means for detecting a shooting range of the shooting means and a visual range of the first speaker in a space where the first speaker exists;
Determination means for determining a direction from the visual range to the photographing range based on the detected photographing range and the visual range;
Based on the detected shooting range and viewing range, means for calculating the distance between the shooting range and the viewing range;
Projecting means for generating image information representing the determined direction and the calculated distance , and projecting the generated image information within the viewing range of the first speaker ;
And the detection means includes
Means for projecting a photographing range presentation image representing a photographing range by the photographing means to a space where the first speaker exists;
Means for projecting a visual range presentation image representing the visual range of the first speaker into a space where the first speaker exists;
Omnidirectional photographing means for photographing a space in which the first speaker exists;
Means for detecting the photographing range presentation image and the visual range presentation image from the photographed image information obtained by the omnidirectional photographing means;
Remote communication system, characterized in that it comprises a. A first speaker and a second speaker existing in different spaces pick up an image of an arbitrary target in the space where the first speaker exists, and the captured image information is captured by the second speaker. In a remote communication system that performs communication while visually observing both of the above objects by displaying on a display means installed in a space where
Detecting means for detecting a shooting range of the shooting means and a visual range of the first speaker in a space where the first speaker exists;
Based on the detected shooting range and viewing range, a determination unit that determines a direction from the shooting range to the viewing range;
Based on the detected shooting range and viewing range, means for calculating the distance between the shooting range and the viewing range;
Display control means for generating image information representing the determined direction and the calculated distance, and displaying the generated image information on the display means together with photographed image information obtained by the photographing means ;
And the detection means includes
Means for projecting a photographing range presentation image representing a photographing range by the photographing means to a space where the first speaker exists;
Means for projecting a visual range presentation image representing the visual range of the first speaker into a space where the first speaker exists;
Omnidirectional photographing means for photographing a space in which the first speaker exists;
Means for detecting the photographing range presentation image and the visual range presentation image from the photographed image information obtained by the omnidirectional photographing means;
Remote communication system, characterized in that it comprises a. A first speaker and a second speaker existing in different spaces photograph an arbitrary object in the space where the first speaker exists by photographing means, and the second speaker captures the captured image information. By displaying on the display means installed in the existing space, when communicating while viewing the above object together,
Detecting the shooting range and the viewing range of the first speaker in a space where the first speaker exists;
Based on the detected shooting range and viewing range, determining the direction from the viewing range to the shooting range;
Based on the detected shooting range and viewing range, a process of calculating the distance between the shooting range and the viewing range;
Generating image information representing the determined direction and the calculated distance , and projecting the generated image information within the viewing range of the first speaker ,
And the detecting process includes:
Projecting a shooting range presentation image representing the shooting range into a space where the first speaker exists;
Projecting a visual range presentation image representing the visual range of the first speaker into a space where the first speaker exists;
Photographing all directions of the space where the first speaker exists;
A process of recognizing the shooting range presentation image and the viewing range presentation image from the shooting image information obtained by shooting in all directions.
Direction presentation method, characterized in that it comprises a. A first speaker and a second speaker existing in different spaces photograph an arbitrary object in the space where the first speaker exists by photographing means, and the second speaker captures the captured image information. By displaying on the display means installed in the existing space, when communicating while viewing the above object together,
Detecting a shooting range of the shooting means and a viewing range of the first speaker in a space where the first speaker exists;
Based on the detected shooting range and viewing range, a process of determining the direction from the shooting range to the viewing range;
Based on the detected shooting range and viewing range, a process of calculating the distance between the shooting range and the viewing range;
Generating image information representing the determined direction and the calculated distance, and displaying the generated image information together with the captured image information on the display means,
And the detecting process includes:
Projecting a shooting range presentation image representing the shooting range into a space where the first speaker exists;
Projecting a visual range presentation image representing the visual range of the first speaker into a space where the first speaker exists;
Photographing all directions of the space where the first speaker exists;
A process of recognizing the shooting range presentation image and the viewing range presentation image from the shooting image information obtained by shooting in all directions.
Direction presentation method, characterized in that it comprises a. A first speaker and a second speaker existing in different spaces photograph an arbitrary object in the space where the first speaker exists by photographing means, and the second speaker captures the captured image information. Projecting means for performing communication while visually observing both of the objects by displaying on a display means installed in an existing space, and projecting an image on the viewing range of the first speaker, and a computer A direction presentation program used in a remote communication system,
A process for detecting the shooting range and the viewing range of the first speaker in the first space;
Based on the detected shooting range and viewing range, processing to determine the direction from the viewing range to the shooting range;
Based on the detected shooting range and viewing range, a process for calculating the distance between the shooting range and the viewing range;
Processing for generating image information representing the determined direction and the calculated distance ;
Causing the computer to execute a process of projecting the generated image information onto the viewing range of the first speaker by the projection unit ;
In the detection process,
A process of projecting a shooting range presentation image representing the shooting range in a space where the first speaker exists;
A process of projecting a visual range presentation image representing the visual range of the first speaker into a space where the first speaker exists;
A process of photographing all directions of the space where the first speaker exists;
Processing for recognizing the shooting range presentation image and the viewing range presentation image from the shooting image information obtained by shooting in all directions.
The direction presentation program characterized Rukoto is executed on the computer. A first speaker and a second speaker existing in different spaces photograph an arbitrary object in the space where the first speaker exists by photographing means, and the second speaker captures the captured image information. A direction presenting program used in a remote communication system equipped with a computer that performs communication while visually observing both of the above objects by displaying on a display means installed in an existing space,
A process for detecting the shooting range of the shooting means and the viewing range of the first speaker in the first space;
Based on the detected shooting range and viewing range, processing to determine the direction from the shooting range to the viewing range;
Based on the detected shooting range and viewing range, a process for calculating the distance between the shooting range and the viewing range;
Processing for generating image information representing the determined direction and the calculated distance ;
Causing the computer to execute a process of displaying the generated image information on the display unit together with the captured image information obtained by the imaging unit ,
In the detection process,
A process of projecting a shooting range presentation image representing the shooting range in a space where the first speaker exists;
A process of projecting a visual range presentation image representing the visual range of the first speaker into a space where the first speaker exists;
A process of photographing all directions of the space where the first speaker exists;
Processing for recognizing the shooting range presentation image and the viewing range presentation image from the shooting image information obtained by shooting in all directions.
The direction presentation program characterized Rukoto is executed on the computer.

Images