JP2005167397A - Server apparatus and control method thereof, and storage medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prevent an increase of the amount of transmission data and an increase of computation complexity in the case that many people use a video communication system employing an omnidirectional camera. <P>SOLUTION: A server apparatus 1-1 used for the video communication system for transmitting a video image from a server apparatus having the camera to a client apparatus comprises: the camera 1-11 capable of carrying out omnidirectional photographing; a video processing section 1-13 for dividing an omnidirectional video image outputted from the camera on the basis of a prescribed rule; a selection section 1-16 for selecting one of the video images divided by the image processing section for transmission to the client apparatus; and a video image distribution section 1-10 for distributing the divided video images in separate streams to the client apparatus. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a video communication technique using an omnidirectional camera.

  In some systems that allow video camera images to be observed from multiple remote locations, the camera pan / tilt angle and zoom magnification can be remotely controlled rather than simply observing the camera video. For example, a system that connects a computer-controllable camera to a WWW (World Wide Web) server on the Internet and not only delivers real-time video shot from the camera to the accessor but also allows control of the camera has been developed. Has been. In such a system, a camera or a pan head having a function of changing the direction by pan / tilt is used.

On the other hand, there are super wide-angle fisheye cameras and cameras that can shoot 360-degree panoramas in the horizontal direction (referred to as omnidirectional cameras) as disclosed in JP-A-11-119303 (Patent Document 1). There are also photographing systems using such a camera. An image taken with such a camera is largely distorted as it is, and is usually presented after converting a necessary portion of the image into a form that is easy to see.
JP 11-119303 A

  When delivering images to a large number of people via the Internet using a physically controllable camera, only one person can control the camera at the same time, and multiple people cannot control it simultaneously. In order to solve this problem, there is a video communication system using an omnidirectional camera. However, the omnidirectional camera simultaneously captures a 360 ° panoramic direction and captures images other than the part that the operator wants to see. Therefore, normally, when delivering a video to a client, the whole video is sent, or only the part that the operator wants to see is cut out and sent.

  When sending all images, the amount of transmission data increases, and when only the part that the operator wants to see is cut out, if many operators access it at the same time, the part to be cut out is calculated for each operator. Occurs and takes time.

  Accordingly, the present invention has been made in view of the above-described problems, and prevents an increase in the amount of transmission data and an increase in calculation amount when a video communication system using an omnidirectional camera is used by a large number of people. is there.

  In order to solve the above-described problems and achieve the object, a server apparatus according to the present invention is a server apparatus used in a video communication system that transmits video from a server apparatus having a camera to a client apparatus, and has all directions. A camera capable of photographing, video processing means for splitting an omnidirectional video output from the camera based on a predetermined rule, and a video divided by the video processing means based on a command from the client device Selection means for selecting which of the videos to be transmitted to the client device, and video delivery means for delivering the divided videos to the client device in separate streams.

  In the server device according to the present invention, the video processing means includes a video input unit that captures an omnidirectional video output from the camera, a video splitting unit that divides the captured omnidirectional video, and a split video And a video compression unit for compressing the video.

  In the server device according to the present invention, the omnidirectional video is divided by a straight line passing through the center of the omnidirectional video.

  In the server device according to the present invention, the omnidirectional video is divided into equiangular sectors around the center of the omnidirectional video.

  In the server device according to the present invention, the omnidirectional video is divided by masking unnecessary portions with a monochrome image.

  In the server device according to the present invention, the single color is black.

  In the server device according to the present invention, the mask has a rectangular shape including a fan shape.

  Further, in the server device according to the present invention, the unnecessary portion is a portion where an image is not displayed, and among the fan shape, an inner portion from the center of the omnidirectional video to the lower end where the image is displayed, and an image It is an outer part from the upper end where is displayed to the outer periphery of the omnidirectional video.

  In the server device according to the present invention, a necessary stream is transmitted to the requested client device in accordance with a requested pan angle from the client device.

  The server device according to the present invention is characterized in that a required stream is transmitted to the client device that has requested it in accordance with the zoom magnification requested from the client device.

  In the server device according to the present invention, there are a plurality of client devices, and a video range corresponding to an arbitrary pan angle or zoom magnification requested from the plurality of client devices is transmitted to the client device. And

  In the server device according to the present invention, when at least two of the plurality of client devices request the same video range, a plurality of streams are transmitted.

  The server device control method according to the present invention is a method for controlling a server device used in a video communication system for transmitting video from a server device having a camera capable of photographing all directions to a client device. A video processing step of dividing the omnidirectional video output from the video processing step based on a predetermined rule, and which video of the video divided in the video processing step to the client device based on a command from the client device A selection step for selecting whether to transmit; and a video delivery step for delivering the divided video to the client device in separate streams.

  A storage medium according to the present invention is characterized in that a program for causing a computer to execute the above control method is stored in a computer-readable manner.

  According to the present invention, when a video communication system using an omnidirectional camera is used by a large number of people, it is possible to prevent an increase in transmission data amount and an increase in calculation amount.

  Hereinafter, a preferred embodiment of the present invention will be described.

In this embodiment, the conventional problem is solved as follows.
(1) Divide an image taken by an omnidirectional camera at a certain angle.
(2) Distribute the divided videos in separate streams.

  The image division of (1) can be simplified by performing a simple shape determined for each direction, and the amount of calculation can be greatly reduced by not calculating a cut-out image for each operator. In addition, the amount of data can be greatly reduced compared to the case where the omnidirectional video is sent as it is by the delivery method (2). When a video server is started up for each stream, the server load is distributed according to angle. For example, when 100 users access at the same time, if the viewing direction is evenly distributed, the load is evenly distributed to each server.

  This embodiment is a system that captures an image from an omnidirectional camera on the server side and displays an image as if the camera was controlled by a client via a network.

  Hereinafter, a video communication system according to an embodiment of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram showing the configuration of the video communication system of this embodiment.

  In FIG. 1, 1-1 is a server apparatus, 1-2, 1-3, and 1-4 are display operation apparatuses, and these are connected to the server apparatus 1-1 by the network 1-5. An omnidirectional camera 1-11 is connected to the server device 1-1, and a video (moving image or still image) captured by the omnidirectional camera 1-11 (imaging device) is displayed via the network 1-5. It can be seen with the operation devices 1-2, 1-3, 1-4. Further, the display operation devices 1-2, 1-3, and 1-4 can be used to change the display range of the omnidirectional camera 1-11. There may be any number of server devices and display operation devices on the network, but in the present embodiment, for convenience of explanation, it is assumed that there is one each (server device 1-1 and display operation device 1-2).

  As for the network 1-5, any digital network such as the Internet or an intranet having a sufficient bandwidth for passing a camera control signal and a compressed video signal described later may be used. Here, the TCP / IP (UDP / IP) protocol is assumed as a network protocol, and the following address indicates an IP address. It is assumed that both the camera server device 1-1 and the display operation device 1-2 are assigned IP addresses. The omnidirectional camera 1-11 is shown in FIG.

  In the omnidirectional camera shown in FIG. 9, when the optical axis of the lens is photographed in the vertical direction, a panoramic image of 360 degrees in the horizontal direction can be simultaneously photographed by the single-sided mirror of the upper hyperboloid. The photographed image is a donut-shaped image as shown in FIG. 10 (a). If the image is left as it is, the distortion is large and it is difficult to intuitively grasp the contents by looking at the image. By panoramic development (geometric image conversion), a horizontally long image as shown in FIG. 10B is obtained. FIG. 10B is a panoramic image with 360 degrees in the horizontal direction, and is easy to grasp intuitively.

  In the present embodiment, it is assumed that the image of the omnidirectional camera is converted into an image corresponding to the angle of view captured by a normal lens and displayed. That is, even if it is wide, the angle of view is converted to approximately 90 degrees or less. However, in this embodiment, it is important not to stick to the numerical value of the angle of view, but to convert the captured raw image into a geometric image that is easy to see as if it was captured with a normal lens.

  In the present embodiment, panning, tilting, and zooming operations can be performed on the image of the omnidirectional camera. For example, an omnidirectional camera image is obtained in the form as shown in FIG. 10- (a). A fan-shaped range as shown in 10-1 is cut out from this, and geometrically transformed into a rectangle. Then, the user actually sees an image like 10-2. By changing the fan-shaped cutout range, virtual (electronic) pan, tilt, and zoom operations can be performed. Of course, in the image obtained by converting the 10- (a) image into 10- (b), the resolution inside the circumference is lower than the outside. That is, at 10 (b), the upper side of the image has a higher resolution than the lower side. However, there is no problem if the resolution at the time of shooting is sufficiently higher than the resolution of the image obtained after geometric conversion. This point is not dealt with in the present embodiment.

  This embodiment is characterized in that an omnidirectional image is divided and distributed.

  The client receives a stream of divided video having a required viewing angle from the server. It may be a single divided video or a plurality of divided videos.

  FIG. 11 is a diagram illustrating a state in which an image of the omnidirectional camera is divided.

In either case, the image is divided into equiangular fans centered on the omnidirectional image center.
(A) is divided into four 90-degree sectors. (B) is divided into two parts by 180 °. (C) is configured by dividing every 90 ° by 90 °. As in this example, the divided image may be covered with another divided image. This is effective when a single image can be used in any direction.

Fan-shaped angle--how many times to configure each element = the maximum angle that only requires one image.

  In the case of (c), 180 ° −90 ° = 90 °, and 90 ° is the maximum angle for one image.

  For explanation, (a) to (c) are prepared, but in this embodiment, as shown in (a), four divisions are performed at 90 °.

  Next, a method for creating a divided image from the entire surrounding image will be described.

  The video server reads the division angle (the angle of the fan-shaped divided image) and the step angle (the angle of how many times the fan-shaped image is formed) set from the registry or file at startup, and each angle for the number of divisions Make a mask according to. Since the number of divisions is 360 ° ÷ step angle, the step angle must be a number obtained by dividing 360 by a positive integer. When the divided angle and the step angle match, the divided images are configured not to overlap each other.

  For example, when the division angle is 60 ° and the step angle is 45 °, the fan shape with an angle of 60 ° is cut out, and eight rectangular masks (360 ° ÷ 45 ° = 8) are formed to exactly fit the fan shape. The mask is a single color, usually black.

  A method for creating a divided image using a mask will be described with reference to FIG.

  (A) Capture an omnidirectional image, (b) Cover the captured omnidirectional image with a mask, and (c) Cut out the masked part and send it to the compression process. Proceed to the process.

  Next, how the divided images selected change according to the change in the pan direction will be described with reference to FIG. In this example, it is divided into a division angle of 90 ° and a step angle of 90 °. For convenience, the vertical upward direction of the omnidirectional image is assumed to be 0 °, and the angle increases in the clockwise direction.

Therefore, the A stream is the upper right part of the 0-90 ° split video, the B stream is the lower right part of the 90-180 ° split video, the C stream is the lower left part of the 180-270 ° split video, The D stream takes charge of the upper left part of the divided video of 270 to 360 °.
(A) The operator is currently facing 45 ° at an angle of view of 30 °. At that time, one A stream is acquired from the server. From this state, it rotates 90 ° clockwise.
(B) When rotating clockwise, a place that spans two divided images of A stream and B stream appears. In the case of the 90 ° direction as an example, two divided videos of A stream and B stream are sent.
(C) When the image is further rotated 45 ° and turned to the 135 ° direction, only the divided video of the B stream is sent.

  FIG. 2 is a diagram for explaining FIG. 1 in more detail.

  The camera server device (server) 1-1 delivers the video imaged by the omnidirectional camera 1-11 to the requested display operation device (client) 1-2, and the camera from the display operation device 1-2. The control command is received, and image selection and image geometric conversion processing (1-13) according to pan, tilt, and zoom operations are performed.

  The server device 1-1 includes a video input unit 1-12 that captures video (moving images and still images) from the omnidirectional camera 1-11, a video dividing unit 1-13 that divides the captured omnidirectional video, and a divided video Video development unit 1-14 for panoramic development when the client's processing capability is insufficient, video compression unit 1-15 for compressing divided video, and video delivery unit 1 for delivering compressed video data to the network 1-5 10. Command interpreting unit 1-17 that interprets commands from the display operation device 1-2 via the network 1-5, calculates the angle of view from the pan, tilt, and zoom commands, and selects a divided video to be transmitted to the client A video selection unit 1-16, a storage device 1-18 used for data transfer of each unit, and an image conversion unit in the video division unit 1-13 are configured. If the panorama development process is unnecessary, the video development unit 1-14 is passed through. Reference numeral 1-19 denotes a camera information setting unit for setting lens parameters (optical parameters) necessary for image geometric transformation of the omnidirectional camera 1-11.

  The video compression unit 1-15 takes in the NTSC video signal from the omnidirectional camera 1-11, performs A / D conversion, compresses it with MotionJEPG, passes it to the video delivery unit 1-10, and sends it to the network 1-5. Here, MotionJEPG compression is used as the video compression format, but MPEG4 or the like may be used, and the present invention is not limited to the compression format.

  Next, the display operation device 1-2 will be described.

  The display operation device 1-2 connects to an arbitrary server device 1-1 by specifying the IP address of each camera server device, and communicates the compressed video data delivered from the server device 1-1. Received through the control unit 1-21, decompressed by the video decompression unit 1-25, performs video segmentation / geometric transformation processing by the video segmentation processing unit 1-26 as necessary, and displays it by the video display unit 1-27. . In addition, camera control, image storage and storage operations, and the like can be performed by user interface (GUI) operations. It is assumed that the display control unit 1-24 performs control of the screen display / operation.

  The video display unit 1-27 includes a bitmap display, and any window system such as Windows (registered trademark) or X-Window that can form a screen as shown in FIG. 3 is installed on the camera client device 1-2. It is assumed that the user interface screen as shown in FIG. 3 is displayed. In FIG. 3, 3-1 is a video display panel on which video is displayed, 3-2 is a camera control panel for camera operation, and the operation of scroll bars 3-22 and 3-23 causes the camera pan, The tilt can be controlled. The buttons 3-24 to 3-27 can also control the pan and tilt of the camera. The scroll bar 3-28 can perform a zoom operation. Note that panning and tilting operations in the display / operation device change the image clipping range. The same applies to the zoom operation.

  FIG. 4 is a process configuration diagram of the present embodiment.

Here, the process means a process of a multitask operating system such as Windows (registered trademark) or UNIX (registered trademark). It is assumed that at least the following processes 4-11, 4-12, and 4-21 are operating on the server device side in the steady state.
Video server process 4-11: Receives video display requests and manages camera video transmission destinations.
Image acquisition process 4-12: Camera image is captured.
Camera control process 4-21: Controls pan, tilt and zoom of the camera.

The following processes 4-13 and 4-22 are dynamically generated in a one-to-one correspondence with the client process according to the connection from the client process.
Video transmission process 4-13: The camera video is compressed and transmitted. Cutout processing is performed as necessary.
Control processing process 4-22: Receives a camera control command from the client process and issues a command to the camera control unit to perform camera control or video segmentation control.

  5 and 6 show details of the operation of these processes.

The camera information register 4-31, the PTZ (pan, tilt, zoom) value register 4-32, the client information register 4-33, and the video buffer 4-34 are shared memories used for data transfer between processes.
Camera information register 4-31: Stores lens parameters (lens characteristics such as focal length). This content is set by the camera information setting unit 1-19.
PTZ value register 4-32: Stores the latest pan, tilt and zoom values as a result of camera control in response to a camera control command from the client.
Client information register 4-33: indicates image development processing capability on the client side. Stores whether the camera can convert the image on the client side (OK) or not (NG).
Video buffer 4-34: It is assumed that writing and reading can be performed simultaneously in units of image frames by the configuration of a ring buffer or the like.

  In the PTZ value register 4-32 and the client information register 4-33, entries are generated in one-to-one correspondence with client processes.

  On the other hand, the client process 4-41 operates on the display operation device 1-2, and the details of the operation are shown in FIG.

  The operation of this embodiment will be described below with reference to the flowcharts of FIGS.

  Note that a packet exchanged between the client process and the server process has a format as shown in FIG. Strictly speaking, a format used in a packet such as TCP / IP or UDP / IP is used, but FIG. 8 shows only a conceptual description of only packet information necessary for the description of the present embodiment. I will do it.

  First, the video server process 4-11, the video acquisition process 4-12, and the video transmission process 4-13 will be described with reference to the flowchart of FIG.

  When the video server process 4-11 is activated (S500), a video acquisition process 4-12 is generated (S502), and an event (command) from the client process 4-41 is awaited (S503). If the event is a video display start request command (FIG. 8 (5)) (S504), the client's image cutout processing capability flag (“flag” in FIG. 8 (5)) included in the video display request packet. (Part) and the transmission source address of the packet are stored as new entries in the client information register 4-33 (S505), and a video transmission process 4-13 is generated (S506). Further, the contents of the camera information register 4-31 indicating the camera information are returned to the client process (S507). If it is a video display end request command (S509), the video transmission process corresponding to the transmission source address of the packet included in the video display end request packet is similarly ended, and an entry in the corresponding client information register 4-33 is entered. Delete (S510).

  When the video acquisition process 4-12 is activated, it captures the camera video as a frame (S522), divides the image according to the direction at a fixed angle (S523), and writes the image into the video buffer 4-34 at a certain cycle. Repeatedly (for example, 30 times per second) (S524).

  The video transmission process 4-13 is generated and activated in response to a video display request from the client, that is, one-to-one corresponding to the client process, and receives the video display end request from the client until the process is terminated. Repeat the operation.

  Basically, the latest video is fetched from the video buffer 4-34 (S532), compressed (S539), and the compressed video data is packetized into the format shown in FIG. (S540). At that time, before compression and transmission, image conversion processing is performed as necessary (S538).

  Specifically, a PTZ (pan, tilt, zoom) value is read (S533), and a divided image corresponding to the PTZ value is selected (S535). Next, the client information is confirmed (S536), and if the conversion processing capability is sufficient (OK), the server side does not perform the conversion process and transmits the captured image as it is to the client. In this case, conversion processing is performed on the client side.

  On the other hand, if the expansion processing capability is insufficient (NG), geometric transformation processing is performed according to the camera type and lens parameter information included in the contents of the camera information register 4-31.

  Next, a camera control process and a control processing process will be described using the flowchart of FIG.

  Since the camera control process 4-21 always accepts a connection request from the client process 4-41, the following process is repeated. That is, when there is a connection request (FIG. 8A) from the client process 4-41, the camera information register 4-31 is first confirmed.

  The control process 4-22 is generated on a one-to-one basis in accordance with the client process that has made the connection request (S602), and the connection acceptance process is performed (S609).

  In the control processing process 4-22, when the process is generated, the process is executed in response to the camera control command (FIG. 8 (3), (4)) transmitted from the connected operation client process 4-41. Appropriate processing is performed until the process ends. When a camera control command (FIG. 8 (3), (4)) is received (S624), the PTZ value in the PTZ value register 4-32 is updated according to the control command (S628), and the transmission image corresponding to the PTZ value is updated. A selection is made (S629).

  The camera control command received from the client process 4-41 performs camera control according to the command through the command interpretation unit 1-17 and the camera control unit. The camera control command is described in this embodiment for convenience of explanation. Therefore, it is assumed that there is the following instruction.

Pan / tilt angle change command
ANG θ φ
Zoom magnification change command
ZOM γ
However, θ, φ, and γ are parameters representing the pan angle, tilt angle, and zoom magnification, respectively. There are various other camera control commands such as backlight correction, autofocus, and manual focus value setting. The description is omitted here.

  FIG. 7 shows the operation of the client process 4-41 operating on the client device 1-2.

  First, when the process is activated by specifying the address (IP address, here “ADDR_C”) of the camera server device 1-1 to be connected, the client's own image cropping capability is confirmed (S702), and video display is performed. A start request is transmitted to the camera server device corresponding to ADDR_C (S703). At that time, the image cropping capability (flag) is set in the video display start request. The packet format is as shown in (5) of FIG. Here, if camera information (FIG. 8 (8)) is not returned from the camera server device corresponding to ADDR_C (S704), the client process is terminated because of an abnormal operation such as an incorrect address.

  Camera Information Notification Packet In FIG. 8 (8), camera includes camera type, and param includes camera parameters necessary for image cutout geometric conversion processing. If the camera information (FIG. 8 (8)) data is returned, the connection is successful, so the camera information is stored in the camera information register 4-51 (S705). The camera information has the same contents as the camera information register 4-31 of the connection destination server. Thereafter, an event, that is, a user interface operation or arrival of various packets from the camera server device 1-1 is repeatedly waited for, and processing corresponding to the event is performed one after another (S706).

  Here, when the user presses the operation start button 3-21 (S707). Whether or not the camera control has already been started (in operation) is confirmed (S708). If already in operation = YES, the process returns to S706. If not in operation = YES, a camera control connection request (FIG. 8) is obtained. (1)) is issued to the camera control server process (S709), and whether or not the connection is accepted is waited (S710). If accepted, a connection is established to the camera control server process, the client state is in operation = YES (S712), the operation of the camera control panel 3-2 is enabled, and the camera control panel 3-2 is activated. It becomes possible to control the camera.

  While the operation of the camera control panel 3-2 is valid, a camera control command corresponding to the operation of the camera control panel 3-2 is generated and issued to the camera server process (S722). At this time, if the clipping process is necessary on the client side, the latest value of the PTZ value in the camera control command is updated and held so that the image clipping process can be performed in S726 when the video data is received.

  In response to a client process termination request (S730) issued by an operation such as a menu, a video display termination request (FIG. 8 (6)) is issued (S731), and the client process is terminated.

  If the packet arrives (S723), if it is video data (FIG. 8 (9)) (S724), the compressed video data in the video data is read and decompressed (S725), and then this video The display image on the image display panel 3-1 is updated using the frame data (S729). If image expansion processing is necessary on the client side, after image expansion processing is performed from the decompressed image data (S727), cutout processing is performed according to PTZ, and the display video is updated (S729). Needless to say, parameters such as a lens included in the camera information are used at that time.

  As described above, according to the above-described embodiment, when images of an omnidirectional camera are divided at a predetermined angle and step and delivered in separate streams, It is not necessary to perform complicated calculations on the angle of view for each client, and it is only necessary to cause the client to select a stream corresponding to the angle of view.

  In addition, since a divided image is created using a circular omnidirectional image using a mask, the creation of the divided image is a simple process, so the load on the server is reduced.

  Also, by dividing and delivering images, the amount of data becomes smaller than when all images are delivered, and the frame rate can be increased.

(Other embodiments)
In addition, an object of each embodiment is to supply a storage medium (or recording medium) on which a program code of software that realizes the functions of the above-described embodiments is recorded to a system or apparatus, and a computer (or CPU) of the system or apparatus Needless to say, this can also be achieved by reading and executing the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention. Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an operating system (OS) running on the computer based on the instruction of the program code. It goes without saying that a case where the function of the above-described embodiment is realized by performing part or all of the actual processing and the processing is included.

  Furthermore, after the program code read from the storage medium is written into a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer, the function is determined based on the instruction of the program code. It goes without saying that the CPU or the like provided in the expansion card or the function expansion unit performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.

  When the present invention is applied to the storage medium, the storage medium stores program codes corresponding to the flowcharts described above.

It is a block diagram which shows the structure of the video communication system of one Embodiment of this invention. FIG. 2 is a detailed block diagram of the system shown in FIG. 1. It is a figure which shows the example of an operation screen in a display operation apparatus. It is a process block diagram in the video communication system of one Embodiment. It is a flowchart which shows operation | movement of a video server process. It is a flowchart which shows operation | movement of a camera control server process. It is a flowchart which shows operation | movement of a client process. It is a figure which shows the list of packet formats. It is a figure which shows an omnidirectional camera. It is a figure which shows the omnidirectional image image | photographed with the omnidirectional camera. It is a figure which shows the example of a division | segmentation of an omnidirectional image. It is a figure which shows the division | segmentation method of an omnidirectional image. It is a figure which shows the selection method of the divided | segmented omnidirectional division | segmentation image.

Explanation of symbols

3-1 Video display panel 3-2 Camera control panel 3-22, 3-23, 3-28 Scroll bar 3-24, 3-25, 3-26, 3-27 Button 10-1, 10-2 Cutout range

Claims (14)

A server device used in a video communication system for transmitting video from a server device having a camera to a client device,
A camera that can shoot all directions,
Video processing means for dividing the omnidirectional video output from the camera based on a predetermined rule;
Selection means for selecting which of the videos divided by the video processing means is to be transmitted to the client device based on a command from the client device;
Video delivery means for delivering the divided video to the client device in separate streams;
A server device comprising:   The video processing means includes a video input unit that captures an omnidirectional video output from the camera, a video division unit that divides the captured omnidirectional video, and a video compression unit that compresses the divided video. The server device according to claim 1.   The server apparatus according to claim 1, wherein the omnidirectional video is divided by a straight line passing through a center of the omnidirectional video.   4. The server device according to claim 3, wherein the omnidirectional video is divided into equiangular fans around the center of the omnidirectional video.   The server apparatus according to claim 1, wherein the omnidirectional video is divided by masking unnecessary portions with a monochrome image.   The server apparatus according to claim 5, wherein the single color is black.   The server apparatus according to claim 5, wherein the mask has a rectangular shape including a fan shape.   The unnecessary portion is a portion in which no image is displayed. Of the sector shape, an inner portion from the center of the omnidirectional video to the lower end where the image is displayed, and an omnidirectional video from the upper end where the image is displayed. The server device according to claim 7, wherein the server device is an outer portion up to an outer periphery of the server.   The server apparatus according to claim 1, wherein a necessary stream is transmitted to the requested client apparatus according to a pan angle requested from the client apparatus.   The server apparatus according to claim 1, wherein a necessary stream is transmitted to the requested client apparatus according to a zoom magnification requested from the client apparatus.   11. The image processing apparatus according to claim 9, wherein there are a plurality of client devices, and an image range corresponding to an arbitrary pan angle or zoom magnification requested from the plurality of client devices is transmitted to the client device. Server device.   12. The server apparatus according to claim 11, wherein, when at least two client apparatuses among the plurality of client apparatuses request the same video range, a plurality of streams are transmitted. A method of controlling a server device used in a video communication system for transmitting video from a server device having a camera capable of photographing all directions to a client device,
A video processing step of dividing the omnidirectional video output from the camera based on a predetermined rule;
A selection step of selecting which of the videos divided in the video processing step is to be transmitted to the client device based on a command from the client device;
A video delivery step of delivering the divided video to the client device in separate streams;
A control method for a server device, comprising:   A storage medium storing a computer-readable program for causing a computer to execute the control method according to claim 13.

Images