JP5120185B2 - Imaging range control device, imaging range control method, and computer program - Google Patents

Description

  The present invention relates to a shooting range control device, a shooting range control method, and a computer program. More specifically, the present invention relates to a shooting range control device, a shooting range control method, and a computer program for a camera of a video conference system that connects a plurality of points via a network. .

2. Description of the Related Art Conventionally, there has been known a video conference system that realizes a conference between persons at remote locations by connecting a plurality of terminals via a network and bidirectionally transmitting and receiving images and sounds. In this video conference system, a technique for controlling the photographing range of the camera of the partner terminal has been proposed (see, for example, Patent Document 1). In this video conference system, the terminal that started the control has the right to control the shooting range of the other camera.
Japanese Patent Laid-Open No. 2003-274255

  However, in the conventional video conference system, when a video conference is performed by connecting a plurality of three or more points, the shooting range of the camera from the terminal other than the terminal that currently controls the shooting range of the camera. There was a problem that even if you wanted to control it, you couldn't. In this case, when the first point operates the camera at the second point, the third point can be confirmed only after the camera has actually moved with respect to the newly determined angle of view. Therefore, the object that the third point was looking at may not be captured, the camera must be operated again, and a smooth conference cannot be performed.

  The present invention has been made in order to solve the above-described problem, and in the video conference system, control of the shooting range of the camera of the video conference system that can limit the operation of the camera even from a terminal that does not operate the camera. It is an object to provide an apparatus, an imaging range control method, and a computer program.

  In order to achieve the above object, an imaging range control apparatus according to a first aspect of the present invention is an imaging means for imaging an image shown to a conference participant participating in a remote conference, and imaging range information indicating the imaging range of the imaging means. And a remote conferencing device comprising switching means for switching the photographing range of the photographing means at a plurality of points, and each remote conference device transmits images taken by itself to other remote conferences via a network. An imaging range control device for controlling an imaging range of the imaging means of each remote conference device in a remote conference implemented by displaying an image transmitted from another remote conference device while transmitting to the device A shooting control means for controlling the shooting means so as to take an image of the entire range that can be shot by the shooting means, and an image of the entire range that can be shot by the shooting means. The image composition means for creating a single still image and the field angle of the current photographing range of the photographing means among the still images of the photographing range of the photographing means of the remote conference device to be operated. Current frame information creating means for creating current frame information, selection frame information creating means for creating selection frame information that is information of an angle of view to be photographed next in the still image in the shootable range, and the operation An operation information receiving unit that receives operation information for operating the selection frame information from the target remote conference device itself or another remote conference device, and a remote that is operating the imaging range of the imaging unit to be imaged next Designated coordinate information receiving means for receiving designated coordinate information that is coordinate information of a location to be photographed next in the still image from the remote conference apparatus other than the conference apparatus, and the operation information received by the operation information receiving means On the basis of the When the selection frame information creating unit creates selection frame information, the selection frame information creating unit is controlled to create the selection frame information within a range including the designated coordinate information received by the designated coordinate information receiving unit. Shooting control information for creating selection frame control means and shooting control information for controlling the shooting means so that the angle of view indicated by the selection frame information created by the selection frame information creation means is controlled by the selection frame control means The image forming apparatus includes: a creating unit; and a transmitting unit that transmits the shooting control information created by the shooting control information creating unit to the remote conference apparatus that is an operation target.

  According to a second aspect of the present invention, in addition to the configuration of the first aspect of the present invention, when the designated coordinate information receiving unit receives a plurality of the designated coordinate information, the photographing range control device is provided. The creating means creates the shooting control information so that the center of gravity of the plurality of coordinates is set to the center of the angle of view to be shot next.

  According to a third aspect of the present invention, in addition to the configuration of the second aspect of the present invention, a photographing range control device is provided with weight giving means for giving weights to the plurality of designated coordinate information, and the photographing control information creating means Is characterized in that, based on the weight assigned by the weight assigning means, the center of gravity of a plurality of coordinates is determined and the photographing control information is created so as to be the center of the angle of view to be photographed next.

  According to a fourth aspect of the present invention, in addition to the configuration of the first aspect of the present invention, the photographing range control device includes group information indicating groups belonging to the remote conference device when participating in the remote conference. And a selection frame information transmission unit that transmits the selection frame information only to a remote conference device that has received the same group information.

  According to a fifth aspect of the present invention, in addition to the configuration of the first aspect of the present invention, when the plurality of designated coordinate information items are received, the plurality of designated coordinate information items are stored. When the angle of view is not entered, the selection frame information is operated by the operation information from the remote conference device that is operating the selection frame information, and the designated coordinate information to be included in the angle of view can be selected. To do.

  According to a sixth aspect of the present invention, in addition to the configuration of the invention according to any one of the first to fifth aspects, the photographing range control device includes a server that connects each remote conference device. It is characterized by that.

  According to a seventh aspect of the present invention, in addition to the configuration of the invention according to any one of the first to fifth aspects, the photographing range control device includes any one of the remote conference devices. It is characterized by that.

According to an eighth aspect of the present invention, there is provided a photographing range control method comprising: photographing means for photographing an image shown to a conference participant participating in a remote conference; and the photographing means based on photographing range information indicating a photographing range of the photographing means. Remote conference devices provided with switching means for switching the shooting range of each of the remote conference devices, each remote conference device transmits an image captured by itself to other remote conference devices via a network, In a remote conference implemented by displaying an image transmitted from another remote conference device, a shooting range control method of a shooting range control device for controlling the shooting range of the shooting means of each remote conference device, the shooting range control device includes a photographing control step of controlling the imaging means to capture the entire range image capturing possible of the photographing means, taking the photographing means has taken An image compositing step for composing all possible range images to create a single still image, and the image capturing means among the still images in the image capturing range of the image capturing means of the remote conference device to be operated A current frame information creating step for creating current frame information of the angle of view of the current shooting range, and selection frame information which is information of the angle of view to be taken next in the still image of the shootable range. A selection frame information creating step; an operation information receiving step for receiving operation information for operating the selection frame information from the remote conference device itself or another remote conference device to be operated; and A designated coordinate information receiving step for receiving designated coordinate information, which is coordinate information of a location to be photographed next in the still image, from a remote conference device other than the remote conference device operating the photographing range; When creating a selection frame information in the selected frame information creation step on the basis of the operation information received by the operation information receiving step, creating the selection frame information in a range including the designated coordinates information received by the specified coordinate information receiving step A selection frame control step for controlling to perform, and a shooting control information creation step for creating shooting control information for controlling the photographing means so as to have an angle of view indicated by the selection frame information created by the control of the selection frame control step When, and executes a transmission step of transmitting to the remote conference apparatus which is an imaging control information created in the imaging control information creating step and operation target.

According to a ninth aspect of the present invention, there is provided a computer program that causes a computer that controls the photographing range control device to execute each step of the photographing range control method according to the eighth aspect.

  In the photographing range control device of the invention according to claim 1, when the first remote conference device is controlling the angle of view of the photographing range of the photographing means of the second remote conference device, the third remote conference device Transmits the designated coordinate information, which is the coordinate information of the location to be photographed next, in one still image obtained by synthesizing the entire range of images that can be photographed by the photographing means of the second teleconference device to the photographing range control device. Thus, it is possible to make a point to be taken next from a remote conference device other than the first remote conference device. Therefore, it is possible to prevent an object that was viewed at another point from being lost, to eliminate the need to operate the camera again, and to perform a smooth conference.

  In addition, in the imaging range control device of the invention according to claim 2, in addition to the effect of the invention of claim 1, when receiving a plurality of designated coordinate information, the imaging control information creating means Since the photographing control information is created so that the center of gravity is set to the center of the angle of view to be photographed next, the angle of view can be determined around the point where the plurality of remote conference apparatuses want to photograph next. The angle of view can be determined around a point desired by many users of the imaging range control device.

  According to a third aspect of the present invention, in addition to the effect of the second aspect of the present invention, the photographing control information creating means includes a plurality of coordinate centers of gravity based on the weight given by the weight assigning means. Since the shooting control information is created so as to be at the center of the angle of view to be shot next, the angle of view can be determined from a point with a higher priority.

  According to a fourth aspect of the present invention, in addition to the effects of the first to third aspects, the selection frame information transmitting means is a remote control that receives the same group information from the group information receiving means. Since the selection frame information is transmitted only to the conference device, only the remote conference device within a limited range can control the angle of view to be photographed next.

  In addition, in the imaging range control device of the invention according to claim 5, in addition to the effect of the invention according to any one of claims 1 to 4, when a plurality of designated coordinate information is received, the plurality of designated coordinate information are If the selection frame information is operated by the operation information from the remote conference device that is operating the selection frame information and the designated coordinate information to be included in the angle of view can be selected. You can select a point and put it in the selection frame.

  Further, in the shooting range control device of the invention according to claim 6, in addition to the effect of the invention according to any one of claims 1 to 5, since the server can be used as the shooting range control device, the capability of the remote conference device is high. Various processes can be performed by the server even without them.

  In addition, in the shooting range control device of the invention according to claim 7, in addition to the effect of the invention of any one of claims 1 to 5, the remote conference device can be used as the shooting range control device, so that the server can be omitted.

  In the shooting range control method of the invention according to claim 8, when the first remote conference device controls the angle of view of the shooting range of the shooting means of the second remote conference device, the third remote conference device The designated coordinate information, which is the coordinate information of the location to be photographed next in one still image obtained by synthesizing the entire range of images that can be photographed by the photographing means of the second teleconference device, is sent to the photographing range control device. By transmitting, it is possible to make a point to be taken next from a remote conference device other than the first remote conference device. Therefore, it is possible to prevent an object that was viewed at another point from being lost, to eliminate the need to operate the camera again, and to perform a smooth conference.

In the computer program of the invention according to claim 9, by causing a computer that controls the shooting range control device to execute each step of the shooting range control method according to claim 8, the effect of the invention according to claim 8 is achieved. Can be played.

  Hereinafter, a video conference system 1 according to an embodiment of the present invention will be described with reference to the drawings. First, the configuration of the video conference system 1 will be described with reference to FIG. FIG. 1 is a block diagram showing a configuration of the video conference system 1.

  As shown in FIG. 1, the video conference system 1 includes a plurality of terminals 3 provided for each point, and a conference server 4 that supervises a conference held between the plurality of terminals 3. 2 to each other. For the sake of explanation, it is assumed that the terminal installed at point A is terminal 3A, the terminal installed at point B is terminal 3B, and the terminal installed at point C is terminal 3C. In this video conference system 1, a remote conference is performed by transmitting and receiving images and sounds between the terminals 3 via the network 2.

  Next, the electrical configuration of the terminal 3 will be described with reference to FIG. FIG. 2 is a block diagram showing an electrical configuration of the terminal 3. The terminal 3 is provided with a CPU 20 serving as a controller for controlling the terminal 3. The CPU 20 mediates data transfer between a ROM 21 storing BIOS and the like, a RAM 22 temporarily storing various data. An I / O interface 30 is connected. A hard disk drive 31 (hereinafter referred to as HDD 31) is connected to the I / O interface 30. The HDD 31 is provided with at least a program storage area 312 and other information storage areas 313.

  The program storage area 312 stores a main program of the terminal 3, a conference execution program for executing a conference with another terminal 3, and the like. In the other information storage area 313, other information used in the terminal 3 is stored. When the terminal 3 is a dedicated machine that does not include the HDD 31, various programs are stored in the ROM 21.

  The I / O interface 30 includes a communication device 25 for communicating with the network 2, a mouse 27, a video controller 23, a key controller 24, a card reader control unit 32, and a user for photographing. A camera control unit 36 that controls the direction of the camera 34 and the zoom of the lens, a microphone 35 for capturing the user's voice, and a CD-ROM drive 26 are connected. A display 28 is connected to the video controller 23, a keyboard 29 is connected to the key controller 24, and a card reader control unit 32 is used to identify a user stored in an identification card (not shown) owned by each user. A card reader 33 for reading the user ID is connected, and a camera 34 is connected to the camera control unit 36. The camera 34 is provided with an electric head 37 for changing the direction up, down, left and right and an electric zoom lens 38. These are driven by the control of the camera control unit 36, and the direction and angle of view of the camera. Is changed.

  The CD-ROM 114 inserted into the CD-ROM drive 26 stores the above main program, conference execution program, shooting range control method program, and the like. At the time of introduction, these various programs are loaded from the CD-ROM 114. The hard disk drive 31 is set up and stored in the program storage area 312.

  Next, the electrical configuration of the conference server 4 will be described with reference to FIG. FIG. 3 is a block diagram showing an electrical configuration of the conference server 4. The conference server 4 is provided with a CPU 40 as a controller that controls the conference server 4. The CPU 40 includes a ROM 41 that stores BIOS and the like, a RAM 42 that temporarily stores various data, and an intermediary for data transfer. Is connected to the I / O interface 50. A hard disk drive 51 (hereinafter referred to as “HDD 51”) is connected to the I / O interface 50. The HDD 51 is provided with at least various storage areas described later (see FIG. 4).

  Further, a communication device 45 for communicating with the network 2, a mouse 47, a video controller 43, a key controller 44, a CD-ROM drive 46, and a timing device 52 are connected to the I / O interface 50. ing. A display 48 is connected to the video controller 43, and a keyboard 49 is connected to the key controller 44.

  The CD-ROM 115 inserted into the CD-ROM drive 46 stores a main program, a conference management program, a shooting range control method program, and the like which will be described later. To the HDD 51 and stored in the program storage area 516.

  Next, various storage areas of the HDD 51 will be described with reference to FIG. FIG. 4 is a conceptual diagram showing each storage area of the HDD 51. The HDD 51 includes a user information table storage area 511, a login table storage area 512, a conference table storage area 513, a participation reception table storage area 514, a camera operation participation information table storage area 515, a program storage area 516, At least a weight storage area 517 that stores the weight for each terminal and other information storage areas 518 are provided.

  The user information table storage area 511 stores a user information table 5111 (see FIG. 5). The login table storage area 512 stores a login table 5121 (see FIG. 6). In the conference table storage area 513, a conference table 5131 (see FIG. 7) is stored. In the participation reception table storage area 514, a participation reception table 5141 (see FIG. 8) is stored. The camera operation participation information table storage area 515 stores a camera operation participation information table 5151 (see FIG. 9). The program storage area 516 stores a main program of the conference server 4, a conference management program for managing a conference held between the terminals 3, and the like. The weight storage area 517 stores a weight for calculating the center of gravity by weighting each coordinate of a plurality of operation restriction information (pointers) described later. The other information storage area 518 stores other information used by the conference server 4. When the conference server 4 is a dedicated machine that does not include the HDD 51, various programs are stored in the ROM 41. Hereinafter, each table will be described in order.

  First, the user information table 5111 will be described with reference to FIG. FIG. 5 is a conceptual diagram of the user information table 5111. The user information table 5111 is a table used for authentication when the user logs in to the video conference system 1. The user information table 5111 stores user information corresponding to each user. Specifically, a user ID for identifying a user, a password for logging in, a name, a company name, and the like are stored. The user registers these pieces of user information in the user information table 5111 in advance. With this user information table 5111, users who use the video conference system 1 can be managed.

  Next, the login table 5121 will be described with reference to FIG. FIG. 6 is a conceptual diagram of the login table 5121. The login table 5121 stores the user ID of the user who has logged into the network 2 and the terminal ID of the terminal 3 in which the user ID is registered. That is, a user ID stored in an identification card (not shown) read by the card reader 33 is stored in the user ID column of the login table 5121. Further, a terminal ID for identifying the terminal 3 that transmitted the user ID is stored in the terminal ID column. The terminal ID is a Mac address of the terminal 3 or the like.

  For example, when the user “Taro Brothers” (see FIG. 5) logs in, the card reader 33 reads the identification card owned by Brother Taro. In this case, the user ID = “A0001” stored in the identification card and the terminal ID = “0001” of the terminal 3 that transmitted the user ID are stored in the login table 5121, respectively. Other users, such as Jiro Jiro, Jiro Saburo, Taro Sales, and Jiro Sales, are similarly set. Thus, the login table 5121 manages the users who have logged into the network 2 and the terminals 3 to which the users are registered.

  Next, the conference table 5131 will be described with reference to FIG. FIG. 7 is a conceptual diagram of the conference table 5131. The conference table 5131 stores a user ID and a conference ID for identifying a conference in which the user ID is registered in advance as a participating member. For example, Brother Taro, Brother Jiro, Brother Saburo, Sales Taro, and Sales Jiro are registered in advance in the conference table 5131 as participating members of the conference with conference ID = 001. Thereby, the participant of a meeting can be managed.

  Next, the participation reception table 5141 will be described with reference to FIG. FIG. 8 is a conceptual diagram of the participation reception table 5141. In the participation reception table 5141, for a conference ID registered in advance, a user ID currently logged in to a conference corresponding to the conference ID is stored as a conference participation ID. For example, the first line of the participation reception table 5141 indicates that the user of the conference participation ID = “A0001”, “A0002”, “B0001” is currently logged in for the conference ID = “001”. .

  Next, the camera operation participation information table 5151 will be described with reference to FIG. FIG. 9 is a conceptual diagram of the camera operation participation information table 5151. The camera operation participation information table 5151 stores the terminal ID of the terminal 3 participating in the operation for adjusting the angle of view of the camera 34.

  Next, the storage area of the RAM 22 of the terminal 3 will be described with reference to FIG. FIG. 10 is a conceptual diagram of a storage area of the RAM 22 of the terminal 3. As illustrated in FIG. 10, the RAM 22 of the terminal 3 includes a composite image of a shootable range shot by the camera 34 included in the terminal 3, a local terminal shooting information storage area 221 that stores the shootable range and the like, and other terminals 3. The other terminal photographing information storage area 222 for storing the composite image of the other terminal 3 received from the above and the shootable range, the area for storing other data, and the like are provided.

  Next, the screen displayed on the display 28 of the terminal 3 will be described with reference to FIG. FIG. 11 is a diagram illustrating the divided display units 281 to 284 displayed on the display 28 of the terminal 3A at the point A. For example, as shown in FIG. 1, a case is assumed where there are three points A to C in the video conference system 1. A terminal 3 with terminal ID = 0001 is installed at point A, a terminal 3 with terminal ID = 0002 is installed at point B, and a terminal 3 with terminal ID = 0003 is installed and installed at point C. Yes.

  From the terminal A at the point A, one participant logs into the conference, from the terminal 3B at the point B, two users log into the conference, and from the terminal 3C at the point C, two users Assume that a user logs in to a meeting.

  In such a situation, the four display units 281 to 284 shown in FIG. 11 are displayed on the display 28 of each terminal 3 at the points A to C. For example, the display unit 281 displays an image of the participant taken by the camera 34 (see FIG. 2) at the point A. On the display unit 282, an image of the participant photographed by the camera 34 (see FIG. 2) at the point B is displayed. The display unit 283 displays an image of the participant photographed by the camera 34 at the point C (see FIG. 2). At this stage, nothing is displayed on the display unit 284. In accordance with this, since the voices of the points A, B, and C are output from the speakers (not shown) of the terminals 3A to 3C by the microphones 35 (see FIG. 2) of the terminals 3A to 3C, they are far away. A conference can be held between other points A, B, and C.

  Next, processing performed in the video conference system 1 having the above configuration will be described. First, processing performed at each terminal 3 participating in a video conference will be described with reference to the flowchart shown in FIG. FIG. 12 is a flowchart of the main control of the terminal 3. In the terminal 3, when the power is turned on, first, the CPU 20 controls the camera control unit 36, and the electric zoom lens 38 of the camera 34 rotates to obtain the maximum wide angle of view. Next, the electric camera platform 37 on which the camera 34 is placed rotates up to the maximum angle of up, down, left, and right, and the entire range that can be captured by the camera 34 is imaged (S11). The shooting data is stored in the own terminal shooting information storage area 221 of the RAM 22 shown in FIG. Next, a plurality of still images stored in the own terminal photographing information storage area 221 of the RAM 22 are combined to create a single combined image (S11). A well-known image composition technique may be employed to create this composite image. For example, image recognition of each still image may be performed, and processing may be performed by combining the same image portions. The composite image is stored in the own terminal photographing information storage area 221 of the RAM 22 (S11). Further, shootable range data consisting of the vertical and horizontal angles with respect to the center of the image of the camera 34 corresponding to the composite image of the shootable range is also stored in the own terminal shooting information storage area 221 (S11).

  Next, the composite image of the own terminal shootable range stored in the own terminal shooting information storage area 221 of the RAM 22, the shootable range data, and the enlargement ratio data when the zoom lens 38 is zoomed in are transmitted to the other terminals 3. (S12). At this time, if a group code described later is stored, the group code is also transmitted to the other terminal 3 (S12).

  Next, the composite image of the shootable range of the other terminal, the shootable range data, and the enlargement ratio data obtained when the zoom lens 38 is zoomed in are received from the other terminal 3 (S13). Is stored in the other terminal photographing information storage area 222 (S13). At this time, if a group code to be described later is received, the group code is also stored in the other terminal photographing information storage area 222 (S13). Next, a multipoint conference is started between the plurality of terminals 3 (S14). Next, the coordinates (XY) of the current center position of the camera 34 and the enlargement ratio of the zoom lens 38 are transmitted together with the video (S15).

  Next, the video from the other terminal 3, the coordinates (XY) of the current center position of the camera 34, and the enlargement ratio of the zoom lens 38 are received, and the composite image shown in FIG. 11 is updated (S16).

  Next, shooting range control of the camera 34 in the terminal 3 will be described with reference to the flowcharts shown in FIGS. 13 to 15 and FIGS. 11, 16, and 17. FIG. 13 is a flowchart of processing performed by the conference server 4. FIG. 14 is a flowchart of control performed by the non-operation instruction terminal 3 that does not instruct the operation of the photographing range. FIG. 15 is a flowchart of adjustment of the camera 34 at the terminal 3 that receives an instruction to operate the shooting range from another terminal 3. FIGS. 16 and 17 are diagrams of images displayed on the display 28 of the terminal 3.

  Here, as an example, a video conference is held between the terminal 3A at the point A, the terminal 3B at the point B, and the terminal 3C at the point C, and the terminal 3A at the point A operates the camera 34 of the terminal 3B at the point B. It shall be. First, as shown in FIG. 11, the conference server 4 displays the composite image shown in FIG. 11 on the display 28 of the terminal 3 at the points A, B and C. In this composite image, the image of the point A is displayed on the upper left display unit 281, the image of the point B is displayed on the upper right display unit 282, the image of the point B is displayed on the lower left display unit 283, and the lower right image is displayed. The display unit 284 is displayed with a blank.

  Next, when the conference server 4 receives the operation instruction signal of the camera 34 of the terminal 3B at the point B from the terminal 3A at the point A (S21: YES), as shown in FIG. 16, all of the terminals 3A, 3B, and 3C The still image 2840 of the operation target terminal 3B, the selection frame 2842 for the shooting range to be photographed next, and the current frame 2841 that is the current display range are displayed on the display unit 284 of the display 28 (S22). Next, the terminal 3A at the point A, which is the operation instruction terminal, is permitted to operate the selection frame 2842 for the next shooting range (S23).

  At this time, the control of the terminal 3C at the point C where the photographing range of the camera 34 of the terminal 3B is not controlled will be described with reference to the flowchart of FIG. The terminal 3C at the point C receives information on the terminal 3B that is the operated terminal and information on the terminal 3A that is the operation instruction terminal from the conference server 4 (S31). With this information, the terminal 3C can know which terminal is controlling which terminal.

  Next, the still image 2840 of the operation target terminal 3B, the selection frame 2842 for the shooting range to be photographed next, and the current frame 2841 that is the current display range are displayed on the display unit 284 of the screen of the display 28 (S32). . Next, when it is desired to limit the movement range from the terminal 3C to the shooting range selection frame 2842 (S34: YES), as shown in FIG. 17, the still image displayed on the display unit 284 of the screen of the display 28 is displayed. On the image 2840, a pointer 2845 for indicating a place to be viewed is displayed (S35). The instruction to display the pointer 2845 is performed by a predetermined operation of the mouse 27 and the keyboard 29 of the terminal 3 shown in FIG.

  Next, a signal for determining the position of the pointer 2845 is received from the conference server 4 by communicating with the conference server 4 (S36). Thereafter, the coordinates (X, Y) on the still image 2840 displayed on the display unit 284 of the screen of the display 28 at the center of the pointer 2845 are transmitted to the conference server 4 as operation restriction information (S37). Thereafter, the process ends. If the movement range of the shooting range selection frame 2842 is not limited from the terminal 3C (S34: NO), the process ends.

  On the other hand, in the conference server 4, when the operation restriction information (the coordinates (X, Y) on the still image 2840 at the center of the pointer 2845) of the shooting range selection frame 2842 is received from the terminal 3C (S24: YES), Only in the range including the received coordinates (X, Y), the operation of the selection frame 2842 for the photographing range is permitted to the terminal 3A as the operation terminal (S25). If a shooting range confirmation signal is received from the terminal 3A as the operation terminal (S26: YES), the coordinates of the center of the shooting range selection frame 2842 (terminal B) as the operation target terminal (operated terminal) ( X, Y) and the enlargement ratio of the selection frame 2842 are created and transmitted as camera control signals (shooting control information) (S27). Thereafter, the process ends. In the determination process of S24, when the operation restriction signal is not received within a certain time (S24: NO), the process proceeds to S26.

  Next, with reference to the flowchart shown in FIG. 15, the adjustment of the camera 34 performed at the terminal 3B as the operated terminal will be described. The terminal 3B waits until the control signal of the camera 34 is received (S41: NO), and when the control signal of the camera 34 is received (S41: YES), the CPU 20 of the terminal 3B performs the electric cloud based on the received control signal. The direction of the camera 34 and the angle of view of the zoom lens 38 are adjusted by the table 37 (S42). Thereafter, the process ends.

  As described above, in the above embodiment, even when the terminal 3A is operating the selection frame 2842 for the shooting range to be photographed after the camera 34 of the terminal 3B, the terminal C other than the terminal 3A wants to view it. By designating the position with the pointer 2845, it is possible to perform shooting in a range in which the position is always included.

  Next, referring to FIG. 18 and FIG. 19, the operation restriction information (the coordinates (X1, Y1) on the still image 2840 of the pointer 2845 and the still image 2840 of the pointer 2846 on the shooting range selection frame 2842 from a plurality of terminals). The process when the coordinate (X2, Y2)) is received will be described. FIG. 18 is a diagram of an image displayed on the display 28 of the terminal 3. FIG. 19 is a flowchart of the process of the conference server 4 when the operation restriction information of the shooting range selection frame 2842 is received from a plurality of terminals. In this example, the terminal 3A controls the camera 34 of the terminal 3B, and the terminal 3C transmits the operation restriction information (the coordinates (X1, Y1) of the pointer 2845 on the still image 2840) of the selection frame 2842 of the shooting range. Then, the terminals 3 other than the terminal 3A, the terminal 3B, and the terminal 3C transmit the operation restriction information (the coordinates (X2, Y2) of the pointer 2846 on the still image 2840) (see FIG. 18).

  In the conference server 4, as shown in the flowchart of FIG. 18, the operation restriction information (the coordinates (X1, Y1) of the pointer 2845 on the still image 2840) and the still image 2840 of the pointer 2846 from a plurality of terminals. When the upper coordinates (X2, Y2)) are received (FIG. 19: S60: YES), the coordinates of the center of the angle of view are determined by the coordinates of the center of gravity of the received plurality of operation restriction information (pointers). In such a case (S61: YES), when a shooting range finalization signal is received (S62: YES), if weighting is set for each terminal 3 (S63: YES), a plurality of operation restriction information (pointers) ) To calculate the center of gravity (S66). This weight is stored in advance in the weight storage area 517 of the HDD 51 (see FIG. 4). For example, when coordinates (X1, Y1) are received from the terminal 3C and coordinates (X2, Y2) are received from the other terminal 3, the weight of the terminal 3C is “3” and the weight of the other terminal 3 is “2”. When stored in the weighted storage area 517 of the HDD 51, the coordinates (X = (3X1 + 2X2) / 5, Y = (3Y1 + 2Y2) / 5) are adopted as the coordinates of the operation restriction information (pointer) (S63). . Thereafter, the calculated coordinates (X = (3X1 + 2X2) / 5, Y = (3Y1 + 2Y2) / 5) are set as the coordinates of the center of the selection frame 2842 of the shooting range (S65).

  If weighting is not set for each terminal 3 (S63: NO), the center of gravity is simply calculated from the coordinates of the plurality of operation restriction information (pointers) (S64). For example, when coordinates (X1, Y1) and coordinates (X2, Y2) are received, the coordinates of the operation restriction information (pointer) are coordinates (X = (X1 + X2) / 2, Y = (Y1 + Y2) / 2. ) Is adopted (S64). Thereafter, the calculated coordinates (X = (X1 + X2) / 2, Y = (Y1 + Y2) / 2) are set as the coordinates of the center of the shooting range selection frame 2842 (S65). Thereafter, the process ends. Note that the operation restriction information (the coordinates (X1, Y1) of the pointer 2845 on the still image 2840 and the coordinates (X2, Y2) of the pointer 2846 on the still image 2840) of the shooting range selection frame 2842) is received from a plurality of terminals. In this case (S60: YES), if the angle of view is not set by the coordinates of the center of gravity of the received plurality of operation restriction information (pointer) (S61: NO), the process ends. In the determination process of S60, the process waits until the operation restriction information of the shooting range selection frame 2842 is received from a plurality of terminals (S60: NO). Further, in the determination process of S62, the process waits until a shooting range confirmation signal is received (S62: NO).

  Next, a method for permitting the operation of the camera 34 of the operated terminal only when the operated terminal and the operation instruction terminal belong to the same group will be described with reference to the flowchart shown in FIG. FIG. 20 is a flowchart of processing in the conference server 4 that permits the operation of the camera 34 of the operated terminal only when the operated terminal and the operation instruction terminal belong to the same group. Here, when belonging to the same group, the setting has the same group code. The group code is a predetermined code determined for a predetermined organizational unit (for each company or each department) that conducts a video conference. For example, A company: A001, B company: B001, C Company: C001 and the like.

  When the conference server 4 receives information (including group code) of the operation instruction terminal and the operation instruction terminal (S51: YES), the still image 2840 of the operation terminal is displayed on the display unit 284 of the display 28 of each terminal 3. Then, the shooting range selection frame 2842 and the current frame 2841 which is the current display range are displayed (S52). Here, when the operation instruction terminal and the operation instruction terminal are in the same group (when having the same group code) (S53: YES), the operation instruction terminal is permitted to operate the selection frame 2842 (S54). If the terminal and the operated instruction terminal are not in the same group (if they do not have the same group code) (S53: NO), the process ends. In the determination process of S51, the process waits until information (including a group code) of the operated instruction terminal and the operation instruction terminal is received (S51: NO).

  Next, referring to FIG. 21, the second case where the operation restriction information (the coordinates (X, Y) of the pointer 2845 and the pointer 2846 on the still image 2840) of the shooting range selection frame 2842 is received from a plurality of terminals. The process will be described. FIG. 21 is a flowchart of the second process of the conference server 4 when the operation restriction information of the shooting range selection frame 2842 is received from a plurality of terminals. In this example, the terminal 3A controls the camera 34 of the terminal 3B, and the terminal 3C transmits the operation restriction information (the coordinates (X1, Y1) of the pointer 2845 on the still image 2840) of the selection frame 2842 of the shooting range. Then, the terminals 3 other than the terminals 3A to 3C transmit the operation restriction information (the coordinates (X2, Y2) of the pointer 2846 on the still image 2840) (see FIG. 18).

  In the conference server 4, as shown in FIG. 18, the operation restriction information (the coordinates (X1, Y1) on the still image 2840 of the pointer 2845 and the still image 2840 of the pointer 2846 on the still image 2840 from a plurality of terminals) When the coordinates (X2, Y2)) are received (FIG. 21: S70: YES), the shooting range selection frame 2842 is expanded to the maximum (S71). Next, when all the coordinates of the operation restriction information (coordinates (X1, Y1), coordinates (X2, Y2)) are entered in the enlarged selection frame 2842 (S72: YES), the processing is ended as it is. If the coordinates (coordinates (X1, Y1), coordinates (X2, Y2)) of all the operation restriction information do not enter the enlarged selection frame 2842 (S72: NO), the coordinates of any operation restriction information In the range including ((X1, Y1) or coordinates (X2, Y2)), the operation of the shooting range selection frame 2842 is permitted (S73). In this case, the terminal 3A operating the selection frame 2842 can remove the coordinates of any operation restriction information by operating the selection frame 2842. However, all the operation restriction information coordinates cannot be removed, and at least one operation restriction information coordinate remains (S73). At this time, instead of manually moving the selection frame 2842, the coordinates of the operation restriction information received earlier may be given priority, or the coordinates of the operation restriction information received later may be given priority. Alternatively, a priority order may be determined for each terminal, and a terminal with a higher priority order may be given priority. Which operation restriction information is given priority may be determined in advance as a rule in the program.

  Next, when a shooting range confirmation signal is received from the operation terminal (S74: YES), all terminals are notified that control of some terminals has been ignored (S75). Thereafter, the process ends. In the determination process of S70, the process waits until operation restriction information of the selection frame 2842 of the shooting range is received from a plurality of terminals (S70: NO). Further, in the determination process of S74, the process waits until a shooting range confirmation signal is received from the operation terminal (S74: NO).

  Next, main control of the video conference system 1 will be described with reference to a flowchart shown in FIG. FIG. 22 is a flowchart showing main control processing by the CPU 40 of the conference server 4.

  As shown in FIG. 22, first, it is determined whether or not user information is received from the terminal 3 (S1). The user information is sent from the communication device 25 of the terminal 3 to the conference server 4 via the network 2 by a participant participating in the conference at the terminal 3 having the card reader 33 read an identification card (not shown). Sent. At this time, the terminal ID of the terminal 3 that transmitted the user information is also transmitted together with the user information. In addition, while not receiving user information (S1: NO), it returns to S1 and reception of user information is monitored continuously.

  Next, when user information is received (S1: YES), a login process is executed (S2). In this process, first, based on the user information received from the terminal 3 at each point, authentication is performed as to whether or not the participant is a registered user. That is, it is determined whether it is registered in the user information table 5111 (see FIG. 5). If the user is a registered user, the user ID and the terminal ID that transmitted the user information are stored in the login table 5121 of the HDD 51 (see FIG. 6). Although not shown in the drawing, if the user is not registered, the user is not authenticated, so the process ends.

  Next, a meeting reception process is executed (S3). In this process, first, a conference in which the logged-in user can participate is searched from the conference table 5131 (see FIG. 7). The meeting organizer decides to hold the meeting. For example, a conference organizer registered in the conference server 4 transmits a conference holding request from the terminal 3 to the conference server 4. Then, the holding of the conference registered in advance by the conference server 4 is determined.

  And user ID is memorize | stored with respect to the corresponding meeting ID registered into the participation reception table 5141 (refer FIG. 8). For example, when “Taro Brothers” with the user ID = 0001 logs in to the conference with the conference ID = 0001, the participation reception table 5141 stores the user ID = A0001 for the conference ID = 001. When “Jiro Jiro” and “Taro Taro” log in to the conference with the conference ID = 001, in addition to the user ID = A0001 with respect to the conference ID = 0001 in the participation reception table 5141. , A0002, and B0001 are stored as needed.

  Next, in the conference in which the user participates, based on the login table 5121, it is determined whether there are two or more terminals that have logged in (S4). A conference cannot be held unless there are at least two terminals 3 that log in. Therefore, if there is only one terminal 3 that logs in for the same conference (S4: NO), it returns to S1 and continues to monitor whether user information has been received in order to wait for the login of another terminal.

  For example, when user IDs = A0001 and B0001 are stored in the login table 5121 (see FIG. 6), these are different terminals. When the conference table 5131 is referred to, both of them have the conference ID = 001. Participating members. In this case, since the conference can be held, a conference start command is transmitted to the terminal IDs “0001” and “0002” of the terminals 3 in which the user IDs = A0001 and B0001 are registered, and these terminals 3 are installed. The points are connected, and the conference is started (S5).

  Subsequently, it is determined whether or not there are participants in the middle of the held meeting (S6). Here, it is determined whether there is a user who has logged in to the same conference from another terminal 3. When there is no midway participant (S6: NO), it is determined whether a conference end request signal is received from the terminal 3 of the conference organizer (S8). When there is a midway participant (S6: YES), the user ID of the midway participant is stored in the participation reception table 5141 for the conference ID (S7). Then, as long as the conference end request signal is not received (S8: NO), the conference is continuously executed and the participants in the middle are monitored (S6). When there is a conference end request from the conference organizer's terminal 3 (S8: YES), a conference end notification is sent to each participant's terminal 3 participating in the conference in order to end the conference. It is transmitted (S9).

  Finally, as a summary, the flow of the shooting range control process when viewed from one terminal 3 will be described with reference to the flowchart shown in FIG. FIG. 23 is a flowchart of the flow of the shooting range control process when viewed from one terminal 3. When the photographing range control process is started, as shown in FIG. 16, a still image 2840 as a composite image of the operation target terminal is displayed on the display unit 284 of the display 28 of the terminal 3 (S81), and then the display unit An angle of view selection frame 2842 and a current frame 2841 are displayed in 284 (S82). Next, when the own terminal instructs the operation of the view angle selection frame 2842 (S83: YES), the operation of the view angle selection frame 2842 becomes possible (S84). Next, when another terminal has created a pointer 2845 for indicating a place to view on the still image 2840 displayed on the display unit 284 of the screen of the display 28 (S85: YES), the pointer 2845 is included. Only within this range, the field angle selection frame 2842 can be operated (S86). Next, the angle of view is determined (S87). If another terminal has not created the pointer 2845 (S85: NO), the angle of view is determined (S87). Next, the conference server 4 is notified of the new coordinates, and a control signal of the camera 34 of the terminal 3 to be controlled is sent from the conference server 4 to the terminal 3 to be controlled (S88). Thereafter, the process ends.

  If the terminal itself has not instructed the operation of the view angle selection frame 2842 (S83: NO), the process ends if all operations are left to other terminals (S89: YES). Further, when the terminal itself does not instruct the operation of the view angle selection frame 2842 (S83: NO), when all the operations are not entrusted to other terminals (S89: NO), the point to be projected is displayed. A pointer 2845 is created (S90). Thereafter, the coordinates of the pointer 2845 are transmitted as operation restriction information to the conference server 4 (S91). Thereafter, the process ends.

  The present invention is not limited to the above embodiment, and various modifications can be made. For example, in the above embodiment, the conference server 4 is used as the imaging range control device, but the function of the conference server 4 may be provided in any terminal 3 so that the conference server 4 is not used. In this case, a plurality of terminals 3 are connected to each other via the network 2, and any one of the terminals 3 functions as an imaging range control device.

  In the above description, the conference server 4 shown in FIG. 1 corresponds to the “shooting range control device” of the present invention, and the terminal 3 and the terminals 3A to 3C correspond to the “remote conference device” of the present invention. The camera 34 shown in FIG. 2 corresponds to the “photographing means” of the present invention, and the electric pan head 37 and the zoom lens 38 correspond to the “switching means” of the present invention. Further, the CPU 20 and the camera control unit 36 correspond to the “shooting control unit” of the present invention. Further, the CPU 20 that executes the process of S11 shown in FIG. 12 corresponds to the “image composition means” of the present invention. Further, the CPU 40 that executes the process of S22 shown in FIG. 13 corresponds to the “current frame information creating unit” and the “selected frame information creating unit” of the present invention. Further, the CPU 40 that executes the process of S21 shown in FIG. 13 and the communication device 25 shown in FIG. 2 correspond to the “operation information receiving means” of the present invention. Further, the CPU 40 that executes the process of S24 shown in FIG. 13 and the communication device 25 shown in FIG. 2 correspond to the “designated coordinate information receiving means” of the present invention. Further, the CPU 40 that executes the process of S25 shown in FIG. 13 corresponds to the “selection frame control means” of the present invention. Further, the CPU 40 that executes the process of S27 shown in FIG. 13 corresponds to the “shooting control information creating unit” of the present invention. Also, the communication device 25 shown in FIG. 2 corresponds to the “transmission unit” of the present invention. Further, the CPU 40 that executes the process of S66 shown in FIG. 19 corresponds to the “weighting unit” of the present invention. The CPU 40 that executes the process of S53 shown in FIG. 20 corresponds to “the communication device 25 corresponds to the“ transmission unit ”of the present invention”. Further, the CPU 40 that executes the process of S66 shown in FIG. 19 corresponds to the “group information receiving means” of the present invention. Further, information (operation restriction information) on coordinates (X, Y) on the still image 2840 displayed on the display unit 284 of the screen of the display 28 at the center of the pointer 2845 and the pointer 2846 is “designated coordinate information” of the present invention. It corresponds to.

1 is a block diagram showing a configuration of a video conference system 1. FIG. 3 is a block diagram showing an electrical configuration of a terminal 3. FIG. 3 is a block diagram showing an electrical configuration of a conference server 4. FIG. 3 is a conceptual diagram showing each storage area of an HDD 51. FIG. It is a conceptual diagram of the user information table 5111. 5 is a conceptual diagram of a login table 5121. FIG. 5 is a conceptual diagram of a conference table 5131. FIG. It is a conceptual diagram of the participation reception table 5141. It is a conceptual diagram of the camera operation participation information table 5151. 3 is a conceptual diagram of a storage area of a RAM 22 of a terminal 3. FIG. It is a figure which shows the display parts 281-284 displayed on the display 28 of the terminal 3A of A point. 4 is a flowchart of main control of the terminal 3. 3 is a flowchart of processing performed in a conference server 4. It is a flowchart of the control performed with the terminal 3 of the non-operation instruction | indication which does not instruct | indicate operation of an imaging range. It is a flowchart of adjustment of the camera 34 in the terminal 3 which receives the instruction | indication of operation of the imaging range from the other terminal 3. 4 is a diagram of an image displayed on a display 28 of the terminal 3. FIG. 4 is a diagram of an image displayed on a display 28 of the terminal 3. FIG. 4 is a diagram of an image displayed on a display 28 of the terminal 3. FIG. FIG. 10 is a flowchart of processing of the conference server 4 when receiving operation restriction information of a shooting range selection frame 2842 from a plurality of terminals. FIG. 6 is a flowchart of processing for permitting operation of the camera 34 of the operated terminal only when the operated terminal and the operation instruction terminal belong to the same group in the conference server 4. 4 is a diagram of an image displayed on a display 28 of the terminal 3. FIG. It is a flowchart which shows the process of the main control by CPU40 of the conference server 4. 5 is a flowchart of a shooting range control process when viewed from one terminal 3;

1 video conference system 2 network 3 terminal 4 conference server 20 CPU
21 ROM
22 RAM
28 Display 34 Camera 36 Camera Control Unit 37 Motorized Head 38 Zoom Lens 40 CPU
41 ROM
42 RAM
51 HDD
221 Self-terminal photographing information storage area 281 Display unit 282 Display unit 283 Display unit 284 Display unit 517 Weighted storage area 2840 Still image 2841 Current frame 2842 Selection frame 2845 Pointer 2846 Pointer

Claims (9)

A plurality of remote conferencing apparatuses, comprising: photographing means for photographing an image to be shown to a conference participant participating in a remote conference; and switching means for switching the photographing range of the photographing means based on photographing range information indicating the photographing range of the photographing means. Each remote conference device is installed at the point of the image, and the image taken by itself is transmitted to another remote conference device via the network, and the image transmitted from the other remote conference device is displayed. In the teleconference implemented by the imaging range control device for controlling the imaging range of the imaging means of each remote conference device,
A photographing control means for controlling the photographing means so as to photograph the entire range of images that can be photographed by the photographing means;
Image synthesizing means for creating a single still image by synthesizing the entire range of images that can be taken by the imaging means;
Current frame information creating means for creating current frame information of the angle of view of the current photographing range of the photographing means in the still image of the photographing possible range of the photographing means of the remote conference device to be operated;
Selection frame information creating means for creating selection frame information which is information of an angle of view to be photographed next in the still image in the shootable range;
Operation information receiving means for receiving operation information for operating the selection frame information from the remote conference device itself or another remote conference device to be operated;
Receives designated coordinate information, which is coordinate information of a location to be photographed next in the still image, from a remote conference device other than the remote conference device that is operating the photographing range of the photographing means to be photographed next. Designated coordinate information receiving means to
When the selection frame information creating unit creates the selection frame information based on the operation information received by the operation information receiving unit, the selection frame information is within a range including the designated coordinate information received by the designated coordinate information receiving unit. Selection frame control means for controlling the selection frame information creation means to create,
Shooting control information creating means for creating shooting control information for controlling the shooting means so that the angle of view indicated by the selection frame information created by the selection frame information creating means is controlled by the selection frame control means;
An imaging range control device comprising: transmission means for transmitting imaging control information created by the imaging control information creation means to a remote conference device that is an operation target.   When the designated coordinate information receiving unit receives a plurality of the designated coordinate information, the shooting control information creating unit is configured to set the center of gravity of the plurality of coordinates to be the center of the angle of view to be shot next. The imaging range control device according to claim 1, wherein: Providing a weight applying means for applying a weight to the plurality of designated coordinate information;
The shooting control information creating means determines the center of gravity of a plurality of coordinates based on the weight given by the weight giving means, and creates the shooting control information so as to be the center of the angle of view to be shot next. The imaging range control device according to claim 2. Group information receiving means for receiving group information indicating that the remote conference device belongs to the same group when participating in the remote conference;
4. The selection frame information transmitting unit, wherein the group information receiving unit transmits the selection frame information only to a remote conference device that has received the group information. Shooting range control device.   When a plurality of the designated coordinate information is received and the plurality of designated coordinate information does not fall within the angle of view, the selection frame information is operated by operation information from a remote conference device that operates the selection frame information. 5. The photographing range control device according to claim 1, wherein the designated coordinate information to be included in the angle of view can be selected.   The imaging range control device according to claim 1, wherein the imaging range control device includes a server that connects each remote conference device.   6. The photographing range control device according to claim 1, wherein the photographing range control device is configured by any one of the remote conference devices. A plurality of remote conferencing apparatuses, comprising: photographing means for photographing an image to be shown to a conference participant participating in a remote conference; and switching means for switching the photographing range of the photographing means based on photographing range information indicating the photographing range of the photographing means. Each remote conference device is installed at the point of the image, and the image taken by itself is transmitted to another remote conference device via the network, and the image transmitted from the other remote conference device is displayed. In the remote conference implemented by the shooting range control method of the shooting range control device for controlling the shooting range of the shooting means of each remote conference device,
The imaging range control device
A photographing control step for controlling the photographing means so as to photograph the entire range of images that can be photographed by the photographing means;
An image synthesis step of synthesizing the entire range of images that can be captured by the imaging unit to create a single still image; and
A current frame information creating step for creating current frame information of the angle of view of the current shooting range of the shooting unit in the still image of the shooting range of the shooting unit of the remote conference device to be operated;
A selection frame information creating step for creating selection frame information which is information of an angle of view to be photographed next in the still image in the photographing range;
An operation information receiving step for receiving operation information for operating the selection frame information from the remote conference device itself or another remote conference device to be operated;
Receives designated coordinate information, which is coordinate information of a location to be photographed next in the still image, from a remote conference device other than the remote conference device that is operating the photographing range of the photographing means to be photographed next. A designated coordinate information receiving step,
When creating a selection frame information in the selected frame information creation step on the basis of the operation information received by the operation information receiving step, the selection frame information in a range including the designated coordinates information received by the specified coordinate information receiving step A selection frame control step for controlling to create,
An imaging control information creating step for creating imaging control information for controlling the imaging means so as to have an angle of view indicated by the selection frame information created by the control of the selection frame control step;
Shooting range control method characterized by performing a transmission step of transmitting to the remote conference apparatus which is an imaging control information created in the imaging control information creating step and operation target. The computer program which makes the computer which controls the said imaging | photography range control apparatus perform each step of the imaging | photography range control method of Claim 8.

Images