JP4405845B2 - Video distribution apparatus and method - Google Patents

Description

  The present invention relates to a video distribution system, and more particularly to a technique for controlling distribution of a video stream in a video distribution system that distributes a video stream having an attribute according to a request from a client.

  With the recent development of streaming technology, it has become common to browse videos distributed from a server on a client. At that time, a stream is selected or subjected to image conversion or the like according to various attributes (encoding / decoding, image quality, bit rate, etc.) requested by the client.

In addition, from the viewpoint of QoS and the like, distribution data is also controlled between the server and the client (see, for example, Patent Document 1).
In addition, there is a video distribution system of an embedded device type camera server, and a desired video stream can be obtained by performing camera control or the like according to a user's request (see, for example, Patent Document 2).

  A typical example of video distribution control is as follows. For example, a video distribution server (video distribution server) is simultaneously connected to a plurality of clients, and distribution of video streams is requested from each client. Here, the processing load on the video distribution server increases as the number of clients that simultaneously perform video distribution increases. When this processing load exceeds the processing capacity of the video distribution server, the overall performance is degraded, and smooth streaming is hindered. Therefore, conventionally, a limit is set on the number of clients that can be connected simultaneously, and when a distribution request is received from another client while the maximum number of clients are connected, the distribution request is rejected. , Was controlled.

JP 2003-009126 A JP-A-10-164420

  However, when a system is considered in which a client can request distribution of a video stream (multi-stream) having an arbitrary attribute and the video distribution server distributes a video stream having an attribute according to a request from the client, Even if the number of clients connected simultaneously is simply limited as in the prior art, effective distribution control cannot be realized. This is because the attributes of the video stream handled for each client are different, and the processing load differs depending on the attribute. Even if the number of clients connected at the same time is small, if each client requests the highest level of quality video, the processing capacity of the video distribution server will be exceeded, and the possibility of a situation where streaming is delayed cannot be excluded. .

  In particular, the camera server of the embedded device type is less powerful than a general video distribution server, and in addition, like the video distribution server, the processing capability is not only used for image processing but also for various device controls. Therefore, the above problem is more remarkable.

  Also, it should be noted that this problem is not only a server processing capability, but the processing capability of the entire system changes depending on the processing capability of the client.

  In addition, in order to select a video stream having an appropriate image attribute that matches the processing capability of the client, the client user needs to search for an optimal video stream by repeating trial and error.

  Furthermore, there is a need for a distribution control technology that can be distributed not only simply but also from various viewpoints when distributing the processing capability of the video distribution server to clients connected simultaneously.

  An object of the present invention is to solve at least one of the above-described problems or other problems.

  According to the video distribution apparatus and method according to an aspect of the present invention, for example, when a distribution request is received from one client, the distribution request is received for each of the other clients connected to distribute the video stream. The processing load at the time is obtained, and the total value is estimated as the current processing load, and the processing load related to the distribution of the video stream according to the distribution request to the one client is estimated. Then, distribution control of the video stream is performed based on at least one of the estimated current processing load and a processing load related to distribution of the video stream according to the distribution request to the one client.

  According to the present invention, optimal distribution control of a video stream is realized according to the processing capabilities of the video distribution server and the client.

  DESCRIPTION OF EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings. In the following, a number of embodiments will be described, but common reference numerals are assigned to configurations and processes common to the embodiments, thereby avoiding redundant description. However, even if a common reference number is assigned, if the content includes a point different from the above-described configuration or processing, the point of difference will be referred to each time.

(Embodiment 1)
FIG. 1 is a block diagram showing a configuration of a video distribution system according to an embodiment of the present invention.

  A video distribution server 100 as a video distribution device includes a data compression device 103 that receives a video signal 102 from an imaging device 101 and compresses data, a memory 104 that stores a boot program and various data, and a central processing that controls operations and processes. The apparatus includes a device (CPU) 105 and a network interface (Network I / F) 106 for connecting to a network.

  The imaging apparatus 101 receives a control signal 107 from the video distribution server 100 and outputs a video signal 102. In the present embodiment, as shown in the figure, the video distribution server 100 and the imaging device 101 are configured independently, but a configuration in which both are integrated may be used.

  The data compression apparatus 103 is realized by, for example, a compression chip, compresses a video signal, outputs data in a format such as JPEG or MPEG, and stores the data in the memory 102.

  The network interface (I / F) 106 is realized by a network chip, for example, and provides a data I / O function between the network 108 and the CPU 105.

  A plurality of clients (information processing terminals) 109 (clients 109-1, 109-2,..., 109-n) each function as a viewer, and receive data distributed from the video distribution server 100 via the network. Receive and display. The client 109 can be realized by a personal computer (PC), a portable computer (PDA), a mobile phone, or the like.

  As shown in FIG. 2, the memory 104 stores a multi-stream creation program 210, a distribution control program 220, stream load data 250, connection information 260, and the like.

  FIG. 3 is a flowchart showing multi-stream creation processing by the video distribution server 100 in the present embodiment. A program corresponding to this flowchart is included in the multi-stream creation program 210 and executed by the CPU 105. Here, an example of handling a JPEG multi-stream having an image size of 640 × 480, 320 × 240, and 160 × 120 is shown.

  First, in step S301, various setting parameters in the memory 104 are initialized. At this time, the load such as the stream load data 250 having the structure shown in FIG. The stream load data 250 has a processing load value for each image size as an attribute based on, for example, the performance of the compression chip (data compression apparatus 103). In the example shown in FIG. 4, the load for processing the stream distribution of the video distribution server 100 of each image size is described using the processing load in the case of the image size 160 × 120 as a reference (= 1). The data shown in FIG. 4 may be a value obtained by experimenting in advance or an estimated value. That is, the processing load at 320 × 240 is 4, the processing load at 640 × 480 is 16, and the processing load at 800 × 600 is 25.

  Next, in step S302, the imaging apparatus 101 is initialized using the imaging parameters set in step S301. This setting can be performed using, for example, a control signal 107 to the imaging apparatus 101.

  Next, in step S303, the data compression apparatus 103 is initialized using the data compression parameters set in step S301. This setting includes, for example, an image size to be compressed.

  In the subsequent step S304, an event such as an interrupt is awaited. The following steps S320, S330, S340, S350, S360, and S370 are steps for determining the type of event received.

  If the received event is a video request event (step S320, yes), the size, which is the attribute of the requested image, is set in the data compression apparatus 103 (step S321), and then the process returns to the event loop of step S304.

  When a video creation completion event is received from the data compression apparatus 103 (step S330, yes), the image is taken out from the memory 104 (step S331), and the video request event is sent according to the connection information 260 having the structure shown in FIG. It is transmitted to the issued client (step S332). As shown in the figure, the connection information 260 describes the currently connected client and the image size requested by the client, and is updated according to a change in the connection status by a process described later.

  If a continuous video request event is received from the client 109 via the network interface 106 (step S340, yes), the video request event is issued and the same processing as step S321 is performed (step S341), and then step S304. Return to the event loop. Here, the continuous video request event is an event for requesting continuous video distribution from the video distribution server 100 to the client 109.

  When a connection end event is received from the client 109 via the network interface 106 (step S350, yes), the corresponding data is deleted from the connection information 260 (step S351), and the event loop of step S304 is performed. Return to. Here, the connection end event is an event requested from the client 109 to the video distribution server 100 in order to disconnect the connection between the video distribution server 100 and the client 109. This connection end event can also be issued when communication from the video distribution server 100 to the client 109 is interrupted for some reason.

  The video distribution mechanism according to the present embodiment is realized by the processes in steps S320 to S351 described above.

  Next, when a connection request event is received from the client 109 via the network interface 106 (step S360, yes), distribution control processing is executed based on the distribution control program 220 (step S361). This connection request event also includes information on the attribute of the video stream requested by the client (hereinafter referred to as “requested image attribute”).

  Hereinafter, the distribution control processing in step S361 by the video distribution server 100 will be described with reference to the flowchart of FIG.

  First, in step S601, a request image attribute included in a connection request event is read, and in step S602, connection information 260 is read. By referring to the connection information 260, all currently connected clients can be specified.

  Next, in step S603, for each connected client, the load corresponding to the client is read and added with reference to the stream load data 250, and the sum is obtained as the total load F. The current processing load is estimated from the overall load F.

  An example will be described based on the connection information 260 in FIG. 5 and the stream load data 250 in FIG. Referring to the connection information 260 in FIG. 5, the client 109-1 and the client 109-2 are currently connected to the video distribution server 100. It can be seen that the client 109-1 requests an image having an image size of 640x480, while the client 109-2 requests an image having an image size of 160x120. Next, referring to the stream load data 250 in FIG. 4, the processing load of the image of 640 × 480 is 16, and the processing load of the image of 160 × 120 is 1, so the total load F (that is, the current processing load) in this case is F = 16 + 1 = 17.

  Next, in step S604, the load f for the currently requested video is obtained with reference to the stream load data 250. For example, when a video with an image size of 320 × 240 is requested, referring to the stream load data 250, the load f for the video requested this time is estimated to be 4.

  When the distribution of the newly requested video is executed, the total processing load at that time is obtained by adding the load f for the currently requested video to the current processing load (= total load F). Value. If this value exceeds the limit of the processing capability of the video distribution server 100, the overall performance may be degraded and smooth streaming may be hindered. Therefore, in step S605, the setting value X (hereinafter referred to as “upper limit value X”) that is the upper limit value of the server process read in step S301 is read, and in step S606, the sum of the total load F and the load f (= current time). It is determined whether or not the distribution of the requested video is as follows (the overall processing load) is below the upper limit value X (that is, whether X ≧ F + f is satisfied). If X ≧ F + f is satisfied, the process advances to step S607 to register the information of the client that issued the connection request event in the connection information 260. Thereafter, a video request event is issued, and processing similar to that in step S321 is performed (step S608). On the other hand, if X ≧ F + f is not satisfied, a rejection response is returned to the issuer of the connection request event (step S609).

  Here, the upper limit value X will be described. This upper limit value X is a value that can be changed by the administrator of the video distribution server 100, and is typically the processing capability permitted by the administrator by the administrator who knows the operation status of the system. Set as the upper limit. The upper limit value X may be set as a fixed value in the memory 104 in advance, but may be set in the memory 104 by a mechanism such as telnet, ftp, or RPC.

  In the conventional method, this upper limit value is specified by the number of client connections and the like, and does not represent the upper limit value of the processing capability permitted by the administrator. When limited by the number of client connections as in the past, when only a stream with a large load is selected, the processing capacity of the video distribution device will be exceeded, and all the stream distribution will be defective. When only these streams are selected, there has been a situation where the distribution is rejected only because the number of connections has reached the upper limit even though the processing capacity of the video distribution apparatus is sufficient.

  On the other hand, the upper limit value X of the present embodiment is not a mere number of client connections, but a value that directly indicates the load on image processing and the like, so that the processing capability of the video distribution server can be utilized more effectively.

  The description returns to the flowchart of FIG. When the distribution control process in step S361 is completed, the process returns to the event loop in step S304.

  If an end event has been received (step S370, yes), the process proceeds to step S371, where all connection information 260 is deleted, and this multi-stream creation process ends.

  According to the first embodiment described above, when the video distribution server receives the connection request, the overall processing load (F + f) when the video stream is created and distributed in response to the request is estimated. Is over a predetermined upper limit (X), the connection relating to the request is rejected. As a result, it is possible to prevent the occurrence of a situation in which the processing load of the video distribution server 100 exceeds the capacity limit, the overall performance is deteriorated, and smooth streaming is hindered.

  In particular, this effect is high in a video distribution server with relatively low processing capability such as an embedded device type camera server. In the case of such a device, the restriction on the processing capacity of the device is greater than the restriction on the transfer path, and the performance of the transfer path exceeds the performance of the equipment. However, appropriate distribution control cannot be performed only by using the processing capability of the video distribution server by performing distribution control according to the processing capability of the device.

  Note that the connection to the video distribution server in this embodiment is for the purpose of video distribution, and the connection for other processing is not considered. Therefore, the expression “reject connection” is equivalent to “reject video distribution” (the same applies hereinafter).

  In the above-described embodiment, an example in which the image attribute indicates the size of the image has been described. However, other than the image attribute, encoding / decoding, image quality, frame rate, or a combination of these, etc. It can be easily understood that the present invention can be applied to these attributes.

(Embodiment 2)
In the above-described first embodiment, image size, image quality, frame rate, and the like are handled as image attributes, and video stream distribution control according to these has been described. However, in this embodiment, not only such image attributes but also image attributes are described. Then, the distribution control of the video stream is performed in consideration of the “processing attribute” indicating the control content of the imaging device (camera).

  The configuration of the video distribution system according to the present embodiment is the same as the configuration shown in FIG. However, in the memory 104, as shown in FIG. 7, in addition to the multi-stream creation program 210, the distribution control program 220, the stream load data 250, and the connection information 260 that are the same as those in the first embodiment, means for controlling the imaging apparatus 101 The camera control program 710 is stored. Whether or not to execute the camera control program 710 can be arbitrarily set by the user. When the camera control program 710 is executed, for example, a patrol process that patrols the orientation of the imaging apparatus 101 or a moving object is performed. Camera control such as moving body tracking processing for moving the direction of the imaging apparatus 101 so as to detect and track the moving body is performed. It is more preferable that the user can set whether or not to execute each of these processes individually, but here, for the sake of simplicity of explanation, it is assumed that only the execution / non-execution setting of the camera control program 710 is performed. To do. Hereinafter, “execution / non-execution of the camera control program 710” is abbreviated as “with / without camera control”.

  In the case of camera control, the processing of the CPU 105 is assigned not only to video stream distribution processing but also to this camera control. In the present embodiment, in consideration of this, the stream load data 250 describes, for each image size, processing loads with and without camera control. FIG. 8 shows an example of the structure of this stream load data 250. In the example shown in the drawing, the processing load in each case is described with the processing load when the image size is 160 × 120 and the camera is not controlled as a reference (= 1). Similarly, in the connection information 260, as shown in FIG. 9, information with / without camera control is described in addition to the image size for each currently connected client.

  The multi-stream creation process in the present embodiment is basically performed in substantially the same manner as the process in the first embodiment (FIGS. 3 and 6).

  In the case of the present embodiment, in step S603 of FIG. 6, for example, the overall load F is obtained as follows. Referring to the connection information 260 in FIG. 9, the client 109-1 and the client 109-2 are currently connected to the video distribution server 100. The client 109-1 requests a video having an image size of 640 × 480, and camera control is set to “none”. On the other hand, the client 109-2 requests an image having an image size of 160 × 120, but the camera control is set to “Yes”. Next, referring to the stream load data 250 in FIG. 8, the processing load when the camera control is not performed for the 640 × 480 image is 16, and the processing load when the camera control is performed for the 160 × 120 image is 2. The total load F is F = 16 + 2 = 18.

  Further, in step S604 for obtaining the load f required to create the video stream requested this time from the stream load data 250, for example, when a video having an image size of 320x240 is requested and setting with camera control is performed, Referring to the stream load data 250 in FIG. 8, it is estimated that the load f = 5.

  The upper limit value X is a setting value that can be changed by the administrator of the video distribution server 100 as in the first embodiment, and is typically set by the administrator who knows the operation status of the system. Is set as the upper limit of the processing capacity allowed. In the first embodiment, the upper limit value X is typically set as an upper limit value for the total load of compression processing. However, in the present embodiment, the upper limit value X corresponds to the total processing load including not only the compression processing but also the load required for camera control. It will be set as the upper limit.

(Embodiment 3)
In the above-described first and second embodiments, when a connection request is received, the overall processing load (F + f) in the case where the creation / distribution of a video stream corresponding to the request is executed is estimated, and this is calculated as a predetermined upper limit value ( When it exceeds X), the connection related to the request is rejected.

  This embodiment does not immediately reject the connection according to the request when the upper limit (X) is exceeded, but provides video stream attribute candidates that can be provided within the capacity of the video distribution server as options. . This aims to reduce the number of cases where the user cannot connect and to provide at least a low-load stream. In the following description, the differences will be mainly described based on the second embodiment, but the present embodiment can be applied to the first embodiment.

  The configuration of the video distribution system according to the present embodiment is the same as the configuration shown in FIG. However, in the memory 104, as shown in FIG. 10, in addition to the multi-stream creation program 210, the distribution control program 220, the camera control program 710, the stream load data 250, and the connection information 260 similar to those in the second embodiment, stream options Information 270 is stored. Further, it is assumed that the stream load data 250 and the connection information 260 according to the present embodiment have the structures as illustrated in FIGS.

  The multi-stream creation processing by the video distribution server 100 in the present embodiment is basically performed in substantially the same manner as the processing in the first and second embodiments (FIG. 3). However, in the present embodiment, the distribution control process in step S361 is performed according to the flowchart of FIG.

  First, in step S1001, the connection information 260 is read. Further, as in step S601 in the first and second embodiments, the request image attribute included in the connection request event is also read.

  Next, in step S1002, the connection information 260 and the stream load data 250 are referred to determine the processing load for each connected client, and the sum is calculated as the total load F (current processing load). In step S1003, the load f for the currently requested video is obtained with reference to the stream load data 250.

  In step S1004, the server processing upper limit value X read in step S301 is read. As described above, the upper limit value X is typically a value that is set as the upper limit value of the processing capability permitted by the administrator by the administrator who knows the operation status of the system. Next, in step S1005, whether or not the sum of the total load F and the load f (= the total processing load when the distribution of the currently requested video is executed) is less than or equal to the upper limit value X (ie, Whether or not X ≧ F + f is satisfied. If X ≧ F + f is satisfied, the process advances to step S1006 to register the information of the client that issued the connection request event in the connection information 260. Thereafter, a video request event is issued, the same processing as step S321 is performed (step S1007), and this processing ends.

  On the other hand, if it is determined in step S1005 that X ≧ F + f is not satisfied, the process proceeds to step S1008 to determine the remaining capacity Y of the video distribution server 100. The remaining capacity Y of the video distribution server 100 is obtained from the upper limit value X-the overall load F.

  Next, in step S1009, based on the obtained surplus Y, candidate video stream attributes that can be provided are read from the stream load data 250, and stream option information 270 having a structure as shown in FIG. For example, assuming that the upper limit value X = 25 and the total load F = 18, the remaining power Y = 25-18 = 7. After that, a set of attributes whose load value is 7 (= reserved power) or less is extracted from the stream load data 250 in FIG. 8, and stream option information 270 with each set as a candidate is configured. Specifically, in this example, “jpeg 320x240, without camera control”, “jpeg 320x240, with camera control”, “jpeg 160x120, without camera control”, “jpeg 160x120, with camera control” are extracted, as shown in FIG. Stream option information 270 is configured.

  Next, in step S1010, the stream option information 270 is notified to the client, and in step S1011 an event is awaited.

  In step S1012, when an attribute specifying request event based on the option is received from the client that has passed the stream option information 270, the process proceeds to step S1013, and the requested image attribute is read. At this time, the attribute may be received as a selection number of the stream option information 270 or the like. Next, in step S1014, as in step S607, the information of the client that issued the attribute specifying request event is registered in the connection information 260. In the subsequent step S1015, a video request event is issued, and the same processing as in step S321 is performed. I do.

  In step S1016, when a connection end event is received from the client that has passed the stream option information 270, or in step S1017, a time-out event has been issued after a certain period of time has passed since the stream option information 270 was passed to the client. In this case, the distribution control process ends.

  According to the third embodiment described above, candidates for a video stream corresponding to the remaining capacity of the video distribution server 100 (that is, a video stream that can be provided within a range in which the processing load of the entire video distribution server 100 does not exceed the upper limit value X). Is notified to the client as an option, and the user can select the next-best video stream accordingly. Thereby, the case where a user cannot connect can be reduced, and at least a low-load stream can be provided.

  In the above processing example, in step S1005, the sum of the total load F and the load f, that is, the total processing load when the distribution of the currently requested video stream is executed exceeds the upper limit value X. In this case, the processing after step S1008 is executed and the option is notified to the client. However, the option is notified to the client regardless of whether the sum of the total load F and the load f exceeds the upper limit value X or not. You may make it do.

(Embodiment 4)
In the above-described third embodiment, the user is notified of the choices of the attributes of the video stream that can be provided within the capacity of the video distribution server 100. However, since the load imposed on the video distribution server changes every moment, the remaining power fluctuates accordingly. Therefore, in the present embodiment, the surplus capacity of the video distribution server is monitored, and when there is a change in the candidates for the attributes of the video stream that can be provided within the surplus capacity, the options are notified again.

  FIG. 13 is a flowchart showing multi-stream creation processing by the video distribution server 100 in the present embodiment. Since many steps in this flowchart are the same as those in the flowchart of FIG. 3, steps having the same contents are denoted by the same reference numerals and description thereof is omitted, and only steps different from those in FIG. 3 are described below. To do.

  As can be seen from the comparison with FIG. 3, the processing of steps S310 to S316 is inserted between the steps S304 and S320 in the flowchart of FIG.

  In step S310, if the received event is an option change event issued when there is a change in the candidate attribute of the video stream that can be provided within the capacity range, the stream load data 250 and the connection information 260 are included. (Step S311), and the option is notified to each client through the following series of operations (step S312).

  That is, referring to the connection information 260 and the stream load data 250, the processing load for each currently connected client is obtained, and the sum is calculated as the total load F (current processing load). Next, a difference between the upper limit value X and the total load F is obtained as a surplus power Y, and based on the surplus power Y, candidates for attributes of video streams that can be provided are read from the stream load data 250 and stored in the memory 104. The stream option information 270 is updated (if the stream option information 270 has not yet been created in the memory 104, it is newly created). Then, a new option is notified for each client, and then the process returns to step S304.

  When the client receives a notification of options, the options are displayed even during video playback. The user of the client can select a desired attribute from the displayed options. In addition, when the surplus capacity of the video distribution server 100 increases, when the choice of higher quality attributes increases, it is convenient that the client is configured to automatically select the high quality attributes. is there.

  When another attribute is selected on the client side, an attribute change request event is issued to the video distribution server 100.

  If an attribute change request event is received in step S313, the requested attribute is read (step S314), and the corresponding portion of the connection information 260 stored in the memory 104 is changed with that attribute (step S315). Subsequently, a video request event and a choice change event are issued (step S316). The option change event is issued when the connection information 260 is changed.

  Further, the flowchart of FIG. 13 differs from the flowchart of FIG. 3 in that step S352 is newly inserted after step S351. When the corresponding data is deleted from the connection information 260 in step S351, an option change event indicating that the option has changed is issued in step S352.

  Also, the distribution control process executed in step S361 is basically executed according to the flowchart shown in FIG. 11, but in this embodiment, information on the client newly connected to the connection information 260 in step S1014 is displayed. After the additional registration is performed and the video request event is issued in step S1015, the option change event is also issued before this distribution control process is completed.

  According to the fourth embodiment described above, when there is a change in video stream attribute candidates that can be provided within the capacity of the video distribution server 100, new options are presented as needed. The person can select the optimum attribute according to the remaining power in a timely manner. For example, a client connected with a minimum number of streams can request a higher quality stream again when the server has enough power.

(Embodiment 5)
In the above-described fourth embodiment, a new option is presented each time there is a change in video stream attribute candidates that can be provided within the capacity of the video distribution server 100.

  However, when the remaining capacity is reduced, the options are merely reduced to a lower quality stream than the currently provided stream, and the high quality stream is not increased. In this case, the client user is bothered to switch to a lower quality than the stream currently being browsed, and may be considered undesirable.

  Therefore, in this embodiment, a new option is presented only when the remaining capacity increases.

  In order to realize this, in the fourth embodiment, the option change event issued in step S316 in the flowchart of FIG. 13 is not issued. Similarly, in the fourth embodiment, the option change event that is issued after step S1015 in the flowchart of FIG. 11 is not issued. Thus, the option change event is issued only in step S352 executed when the number of connections decreases.

  That is, the option change event is not issued when the connection increases, and the option change event is issued only when the connection decreases.

  As a result, only when the number of candidate video stream attributes that can be provided within the range of the capacity of the video distribution server 100 is increased, the client user is notified of a change in choice, so that the client user can receive a useful notification. Can only receive.

(Embodiment 6)
The present embodiment relates to a control process in the client 109 for the video distribution server 100 that provides a choice of candidate video stream attributes described in any of the above-described third to fifth embodiments. Below, it demonstrates based on the video delivery system of Embodiment 3. FIG.

  FIG. 14 is a diagram illustrating a configuration of the client 109 according to the embodiment.

  As shown in the figure, the client 109 includes a network interface (Network I / F) 2001 that receives data from the video distribution server 100 via the network 108, a central processing unit (PU) 2002 that controls operations and processes, and the necessary. A memory 2003 for storing various programs and data, and a display device 2004 constituted by an LCD, a CRT, or the like.

  The compressed data received via the network interface 2001 is temporarily stored in the memory 2003. Thereafter, the compressed data is decompressed by the CPU 2002 and displayed on the display device 2004. Of course, instead of the CPU 2002, a configuration in which a dedicated data processing expansion device (such as a DSP) for performing expansion processing is added may be used.

  As shown in FIG. 15, the memory 2003 implements a video stream receiving program 2100 for receiving a video stream from the video distribution server 100, an option limiting program 2120 for realizing an option limiting process described later, and a viewer function. In addition to storing a program such as a viewer program 2130 for performing the processing, a benchmark program 2140 for measuring the current surplus processing capability of the client, client stream load data 2150 indicating the processing load of the client for each image attribute, video distribution Stream option information 2160, which is information on the options of attribute candidates of video streams that can be provided, notified from the server 100, and stream-limited options in which option data limited by the execution of the option-limiting program 2120 is described. Various data such as distribution 2170 is stored.

  FIG. 16 is a flowchart showing multi-stream reception processing by the client 109 in this embodiment. A program corresponding to this flowchart is included in the stream reception program 2100 and executed by the CPU 2002. This stream reception program 2100 is called, for example, when the viewer program 2130 is executed. As will be described later, the benchmark program 2140 and the option limiting program 2120 are called in the execution of the stream reception program 2130.

  First, in step S2201, various setting parameters in the memory 2003 are initialized. At this time, the client stream load data 2150 having a structure as shown in FIG. 18 is also loaded. The client stream load data 2150 has, for example, a processing load based on the performance of the CPU 2002 for each image size depending on whether or not camera control is performed. In the example illustrated in FIG. 18, the processing load is described for each image size for each presence / absence of camera control, with the processing load when the image size is 160 × 120 and no camera control is used as a reference (= 1).

  In step S2202, a connection request event is issued and transmitted to the video distribution server 100. As described above, the video distribution server 100 executes the distribution control process in response to the reception of the connection request event (FIG. 3, step S361), and the video stream attribute candidate options that can be provided according to the situation. Is transmitted to the client that issued the connection request event (FIG. 11, step S1010).

  When a connection request event is transmitted in step S2202, an initial stream type request may also be transmitted. However, there may be a case where the connection is rejected from the video receiving server 100 or only options below the request stream are sent.

  In the next step S2206, an event is waited for.

  In step S 2203, when information on options of candidate video stream attributes that can be provided is received from the video distribution server 100, the information on the options is stored in the memory 2003 as stream option information 2160, and then the options are limited. By calling the program 2120, an option limiting process for narrowing down the received options is executed (step S2204). An example of the structure of the stream option information 2160 at this time is shown in FIG.

  Hereinafter, the option limiting process in step S2204 will be described with reference to the flowchart of FIG.

  First, in step S2301, after the stream option information 2160 stored in the memory 2003 is read, the current surplus processing capability that the client 109 can allocate to the processing of the viewer (that is, the execution of the viewer program 2130) is measured. Then, the benchmark program 2140 is started (step S2303).

  The benchmark program 2140 is created to make it easier to estimate the processing capacity consumed when the viewer program 2130 is executed, and a program with a relatively light processing load is appropriate. The benchmark program 2140 may be incorporated into the viewer program 2130, and the surplus load may be measured when the viewer program 2130 is activated. Here, it is assumed that the viewer program 2130 includes a program for decompressing compressed video data, and that the load of the decompression processing is the heaviest among the viewer programs 2130. The benchmark program 2140 causes the viewer program 2130 to execute decompression processing of compressed video data having a known processing amount prepared in advance.

  In step S2304, the time T required for the expansion process is measured. Next, from the execution time T, a processing capability P that is a surplus processing capability of the client is calculated (step S2305). Here, the processing capability P is considered to be inversely proportional to the execution time T. For example, assuming that the processing capability P when the execution time T is 100 msec is calculated as 2, the processing capability P when the execution time T is 50 msec is calculated as 4.

  Next, in step S 2306, options having a processing capability of P or less are extracted from the stream option information 2160 and stored in the memory 2003 as stream limited option information 2170. If the processing capacity is P or less, a sufficient video display speed and frame rate can be secured even with surplus processing capacity for the current client. As an example, a case will be described in which the stream option information 2160 is as illustrated in FIG. 19 and the processing capability P calculated in step S2305 is 4. Referring to the stream option information 2160 in FIG. 19, the options presented by the video distribution server 100 include four candidates for the image sizes 320 × 240 and 160 × 120, with and without camera control. is there. Next, referring to the client stream load data 2150 in FIG. 18, the options with a processing capacity (= processing load) of 4 or less are when the image size is 320 × 240 and there is no camera control, when the image size is 160 × 120 and the camera control is performed, And in the case of the same size and no camera control, it is narrowed down to three ways. These three candidates are stored in the memory 2003 as stream limited option information 2170.

  If the options are limited by the above processing, the limited options are displayed in step S2310. In step S2311, it is determined whether the options are limited. If the options are limited, the user is allowed to select any one in step S2312. On the other hand, for example, when the processing capacity P calculated in step S2305 is small and the options are not limited, the process proceeds to step S2313 and issues an end event. Thus, the process of step S2204 is completed.

  As for the processing in step S2312, instead of allowing the user to select a final candidate, a stream with the highest image quality that can be displayed may be automatically selected.

  The description returns to the flowchart of FIG. When the option limiting process in step S2204 is completed, the process advances to step S2205 to issue a video request event, which is transmitted to the video distribution server 100, and returns to the event loop in step S2206.

  When reception of the video stream (see step S332 in FIG. 3) from the video distribution server 100 is completed and a reception completion event is issued (step S2207), the received video stream is sequentially fetched from the memory 2003 (step S2208). The video stream is displayed on the display device 2004 (step S2209). Thereafter, a continuous video request event is issued in step S2211, and the process returns to the event loop in step S2206.

  In step S2214, if a connection rejection response (see step S609 in FIG. 6) is received from the video distribution server 100, a connection end event is issued (step S2212), and the multi-stream reception process ends.

  According to the sixth embodiment described above, when the video distribution server 100 notifies the video stream server 100 of options of candidate video stream attributes that can be provided, the client 109 estimates the processing capability of the client 109 at that time. Then, based on the estimated processing capability, the option candidates are further limited, and the limited candidate options are displayed. As a result, the user can easily specify a video stream having an attribute commensurate with the processing capability on the client side.

(Embodiment 7)
In the present embodiment, video stream attribute candidates that can be provided within the capacity of the video distribution server 100 are selected in consideration of priority information set for each attribute.

  In the following description, the differences will be mainly described based on the third embodiment. However, the present embodiment can also be applied to the fourth and fifth embodiments.

  The configuration of the video distribution system according to the present embodiment is the same as the configuration shown in FIG. However, in the memory 104, as shown in FIG. 20, the same multi-stream creation program 210, distribution control program 220, camera control program 710, stream load data 250, connection information 260, and stream option information 270 as in the second embodiment are stored. In addition, a priority table 280 and priority-specific stream load data 290 are stored. Also in this embodiment, the stream load data 250 and the connection information 260 are assumed to have the same structure as that of the third embodiment (see FIGS. 8 and 9). However, in the present embodiment, it is assumed that the content of the current connection information 260 is as shown in FIG. That is, as can be seen with reference to FIG. 22, four viewers (clients) 109-1 to 109-4 are currently connected to the video distribution server 100.

  The multi-stream creation process by the video distribution server 100 in this embodiment is basically performed according to the flowchart of FIG. 3 described in the third embodiment or the flowchart of FIG. 13 described in the fourth embodiment. However, in the present embodiment, the distribution control process in step S361 is performed according to the flowchart of FIG.

  First, in step S5101, connection information 260 is read. Also, the request image attribute included in the connection request event is read as in step S1001 in the third embodiment.

  Next, in step S5102, the connection information 260 and the stream load data 250 are referred to determine the processing load for each connected client, and the sum is calculated as the total load F (current processing load). A specific example will be described based on the connection information 260 in FIG. Referring to the connection information 260 in FIG. 22, there are connections from the viewers 109-1 to 109-4, which are 800x600 without camera control, 160x120 with camera control, 640x480 with no camera control, and 800x600 with no camera control. It can be seen that an attribute video stream is requested. Here, referring to the stream load data 250, the processing load of each connection is 25, 2, 16, 25. Therefore, the total load F is obtained as a sum 68 of these (an example of this value will be used later).

  In step S5103, the upper limit value X of the server process is read out. As described above, the upper limit value X is typically a value that is set as the upper limit value of the processing capability permitted by the administrator by the administrator who knows the operation status of the system. Next, in step S5104, the remaining power Y of the video distribution server 100 is obtained. The remaining power Y is obtained as a difference between the upper limit value X and the overall load F. Thereafter, in step S5105, the priority-specific option creation program 230 is called.

  An example of the operation by the priority-specific option creation program 230 is shown in FIG.

  First, in step S5301, a priority table 280 having a structure as shown in FIG. 24 is read. In the priority table of FIG. 24, information indicating the priority is described for each case classified in the same manner as the stream load data 250 of FIG. Specifically, in the example shown in FIG. 24, the information indicating the priority is 0 when the image size is 800 × 600 and the camera control is performed and when the camera control is not performed, the image size is 640 × 480 and the camera control is performed and the camera is not controlled. In both cases, information indicating the priority is set to 10. Also, the information indicating the priority when the image size is 320 × 240 and the camera is controlled is 2, the information indicating the priority when the image size is the same size and the camera is not controlled is 0, the image size is 160 × 120 and the camera is controlled, and the camera is not controlled. In both cases, the information indicating the priority is set to 2. Here, it is considered that the priority is highest when the information indicating the priority is 0.

  In step S5302, the value of the information indicating the corresponding priority described in the priority table 280 (FIG. 24) is added to the load of each case described in the stream load data 250 (FIG. 8). Then, the priority-specific stream load assumption data 290 is created. FIG. 25 is a diagram illustrating an example of the obtained priority-specific stream load assumption data 290. If FIG. 8 and FIG. 24 are overlapped, it can be easily understood that the result of FIG. 25 is obtained.

  Next, in step S5303, based on the available capacity Y obtained in step S5104, candidate video stream attributes that can be provided are read from the priority-specific stream load assumption data 290, and stream option information 270 as shown in FIG. 26 is obtained. Write to memory 104. Here, a specific example will be given on the assumption that the total load F is set to 68 and the upper limit value X is set to 93 as described above. In this case, the remaining force Y is 93−68 = 25. Thereafter, a set of attributes having a load value of 25 (= reserved power) or less is extracted from the stream load assumption data 290 by priority, and stream option information 270 with each set as a candidate is configured. In the example of the stream load assumption data 290 by priority in FIG. 25, “jpeg 800 × 600, no camera control”, “jpeg 320x240, with camera control”, “jpeg 320x240, no camera control”, “jpeg 160x120, with camera control”, “jpeg 160x120, “No camera control” is extracted, and stream option information 270 as shown in FIG. 26 is configured.

  In this example, the cases of “jpeg 640x480, without camera control” and “jpeg 640x480, with camera control” are 16 and 17, respectively, even though the load is within the remaining power (= 25) range (see FIG. 8). Reference), while information 10 indicating priority is added, it is removed from the options, while “jpeg 800 × 600, no camera control”, which has a higher load than these, was not added with information indicating priority (priority) Therefore, the result is extracted as an option.

  By setting the priority in this way, it is possible to prioritize a video stream having a high processing load attribute as a choice candidate over a video stream having a low processing load attribute. In other words, the operation is performed so that the stream having the specific attribute of the load that falls within the remaining capacity does not appear in the choices by setting the priority. On the other hand, an operation is performed so that an attribute stream having a higher load than that of the specific attribute stream is selected. Thereby, the remaining power can be turned to a stream having a higher load attribute.

  In addition, by setting the priority, it is possible to create a tendency to increase connections in the stream type intended by the server administrator. For example, when a video with a large stream size is preferentially distributed but a video stream with as large a size as possible is to be distributed preferentially, the priority level may be set to be lowered for a small size stream even if the stream load is small.

  In the above example, as the value that defines the priority, the priority is the highest when the information indicating the priority is 0, and the value becomes smaller as the value increases. This is defined on the assumption that it is added to the load of each attribute. On the other hand, it is also possible to define the priority as a weighting factor by which the load of each attribute is multiplied. In this case, the priority level depends on the numerical value.

  The description returns to the flowchart of FIG. Next, in step S5106, the stream option information 270 is notified to the client, and in step S5107, an event is awaited.

  If it is determined in step S5110 that an attribute specifying request event based on the option is received from the client that has passed the stream option information 270, the process advances to step S5111 to read the attribute of the requested image. At this time, the attribute may be received as a selection number of the stream option information 270 or the like. Similarly to the first and second embodiments, whether or not connection is possible may be controlled based on the upper limit value X, the total load F, and the requested priority stream load f.

  In step S5112, information on the client that issued the attribute identification request event is registered in the connection information 260. In step S5113, a video request event is issued, and an option change event is issued in step S5114.

  In step S5120, when a connection end event is received from the client that has passed the stream option information 270, or in step S5130, a time-out event has been issued after a certain period of time has passed since the stream option information 270 was passed to the client. In this case, the distribution control process ends.

(Embodiment 8)
In the present embodiment, when there is not enough capacity for processing to distribute a video stream that conforms to a connection request, the remaining capacity is generated according to the priority, thereby realizing the distribution related to the request.

  The configuration of the video distribution system according to this embodiment is the same as that of the seventh embodiment. That is, the video distribution system configuration according to this embodiment is as shown in FIG. 1, and the contents stored in the memory 104 are as shown in FIG.

  Also, the multi-stream creation process in the present embodiment is basically performed according to the flowchart of FIG. 3 described in the third embodiment or the flowchart of FIG. 13 described in the fourth embodiment, as in the seventh embodiment, and step S361. The distribution control process is performed according to the flowchart of FIG. However, in this embodiment, the priority-specific option creation processing in step S5105 is performed according to the flowchart of FIG. 27 instead of the flowchart of FIG.

  First, in step S5701, the priority table 280 as shown in FIG. 24 and the requested image attribute read in step S5101 are read. In the next step S5702, as in step S5302 in the seventh embodiment, the corresponding priority described in the priority table 280 (FIG. 24) is assigned to the load of each case described in the stream load data 250 (FIG. 8). Is added, the priority-specific stream load assumption data 290 is created. In the next step S5703, based on the available capacity Y obtained in step S5104, candidate video stream attributes that can be provided are read from the priority-specific stream load assumption data 290, and stream option information 270 as shown in FIG. 26 is created. To do.

  In step S5704, it is determined whether the requested image attribute read in step S5701 is included in the stream option information 270 created in step S5703. If the requested image attribute is included in the stream option information 270, the priority-specific option creation process is exited. On the other hand, if the requested image attribute is not included in the stream option information 270, the process advances to step S5705.

  In step S5705, the priority of the requested image attribute is confirmed with reference to the priority table 280, and stream distribution with a priority lower than the priority of the requested image attribute is referred to with reference to the connection information 260 and the priority table 280. To see if a connection has been established. Here, if a connection for stream delivery with a priority lower than the priority of the requested image attribute has not been established at this time, the selection processing for each priority is exited as it is. On the other hand, if a connection for stream delivery with a priority lower than the priority of the requested image attribute has been established, the process proceeds to step S5706, where a connection rejection event is issued to the connection destination client, and then the step In step S5707, information related to the client that refuses the connection is deleted from the connection information 260.

  Specific examples of steps S5705 and S5706 are shown. The client that issued the connection request event to be processed is 109-5, and another four clients (109-1 to 4) are currently connected. The connection information 260 at that time is as shown in FIG. Assume that there is. Also, assume that the requested image attribute included in the connection request event read in step S5701 is “jpeg 800 × 600, no camera control”.

  First, the priority of the requested image attribute is confirmed with reference to the priority table 280. Referring to the priority table 280 of FIG. 24, it can be seen that the value indicating the priority of the requested image attribute “jpeg 800 × 600, no camera control” is set to “0”, which is set to the highest priority.

  Next, with reference to the connection information 260 and the priority table 280, it is checked whether or not a connection related to a video stream having a priority attribute lower than the priority of the requested image attribute is established. Here, when examining the priority table 260 of FIG. 22, the attribute of the video stream to the clients 109-1 and 109-4 is “jpeg 800 × 600, no camera control”, and the value indicating the priority is “0”. Is. On the other hand, the attribute of the video stream to the client 109-2 is “jpeg 160 × 120, with camera control”, and the value indicating the priority is set to “2” from the priority table 280 of FIG. The attribute of the video stream to the client 109-3 is “jpeg 640 × 480, no camera control”, and the value indicating the priority is set to “10” from the priority table 280 of FIG. Then, it is found that the connection between the clients 109-2 and 109-3 relates to a video stream having a priority attribute lower than the priority of the requested image attribute.

  In step S5706, the connection refusal event may be issued immediately to both of these two clients 109-2 and 109-3, but here, the load assumption value by priority (see FIG. 25) is selected. A connection rejection event is issued only to the highest client (ie, 109-3). As a result, the remaining power increases as a result of the disconnection of the client 109-3. If it is still insufficient to create a video stream having the requested image attribute, a connection refusal event may be issued to the client 109-2.

  Specific examples of steps S5705 and S5706 are as described above.

  In the next step S5708, the stream option information 270 is created again in the same manner as in step S5703. In this case, the stream option information 270 includes the request image attribute of the client that has issued the connection request event to be processed by the deletion in step S5707.

  Thus, the priority-specific option creation process ends.

  According to the above-described option creation process by priority, when a connection request to a stream with a high priority attribute is received, if there is not enough room, distribution of a video stream with a low priority attribute is stopped. It is possible to generate a surplus power and thereby execute the delivery of the above-mentioned high priority stream.

  Further, the connection of the client that is the delivery destination of the low-priority stream is not released (steps S5706 and S5707 are not executed). In step S5708, it is assumed that the connection of the low-priority client is released. After calculating Y and subtracting the load applied to the stream of the requested image attribute having a high priority from the remaining capacity Y, the option is created again, and this is presented to the client related to the stream having the low priority. You may do it. In this case, it is possible to continue the distribution by switching to a stream with a different attribute without disconnecting the client connection relating to the distribution of the low priority stream.

(Embodiment 9)
In the seventh and eighth embodiments described above, priority information is set for each attribute of a video stream. In this embodiment, a priority is set for a client, and a video stream having an attribute that can be provided in consideration of the priority. Create alternatives.

  The configuration of the video distribution system according to the present embodiment is the same as that of the eighth embodiment. That is, the video distribution system configuration according to this embodiment is as shown in FIG. 1, and the contents stored in the memory 104 are as shown in FIG.

  Also, the multi-stream creation process in the present embodiment is basically performed according to the flowchart of FIG. 3 described in the third embodiment or the flowchart of FIG. 13 described in the fourth embodiment, and the distribution control process in step S361 is performed. This is performed according to the flowchart of FIG. Further, as in the eighth embodiment, the priority-specific option creation processing in step S5105 is performed according to the flowchart of FIG. 27 instead of the flowchart of FIG.

  However, the data structure of the priority table 280 is different from that in the eighth embodiment (FIG. 24). FIG. 28 shows a structural example of the priority table 280 in the present embodiment. The priority in the eighth embodiment is set for the attribute of the video stream as shown in FIG. 24, but the priority in the present embodiment is set for the client (viewer) as shown in FIG. .

  The setting value indicating the priority can be changed in accordance with the operation of the administrator in the video distribution server 100, but the setting value indicating the priority of the client is changed according to an instruction from the client. It is also possible.

  For example, each client can include information indicating the priority of the client in addition to the requested video attribute in the connection request event issued to the video distribution server 100. When receiving the connection request event, the video distribution server 100 updates the corresponding part of the priority table 280 with the information indicating the priority included therein.

  In this regard, an operation of the present embodiment that is different from that of the eighth embodiment will be described.

  First, the distribution control process of FIG. In step S5101, in addition to reading the connection information 260, the request image attribute included in the connection request event is read. Further, if the connection request event includes information on the priority of the client, the information is also read.

  Next, the priority-specific option creation process of FIG. In step S5701, the priority table 280 as shown in FIG. 28, the requested image attribute read in step S5101 and the priority information of the client are read. If the client priority information is read out, the corresponding part of the priority table 280 is updated with the information.

  In step S5702, the priority is added by adding a value indicating the priority of the client described in the priority table 280 (FIG. 28) to the load of each case described in the stream load data 250 (FIG. 8). Separate stream load assumption data 290 is created. An example of priority-specific stream load assumption data 290 in this case is shown in FIG.

  In addition, when the priority is not specified on the client side and the priority information of the client is not included in the connection request event, and the client is not registered in the priority table 280. The default value (for example, 20) preset in the video distribution server 100 is used.

  The differences from the eighth embodiment have been described above. Except for this, the same processing as in the eighth embodiment is performed.

  According to the ninth embodiment described above, a priority is set for a client that can connect to the video distribution server 100 (that is, can receive a video stream from the video distribution server 100). Based on the requested image attribute, a video stream option having an attribute that can be provided within the range of the remaining capacity of the video distribution server 100 is created. As a result, the contents of the options presented by the client requesting the connection are different. For this reason, the higher the priority set for the client, the higher the processing load of the attributed video stream will be included in the options, and more capabilities will be distributed within the range of server capabilities. Become.

  In the above example, the priority is set for the client, but with the same concept, the priority is set for the user and the video stream options with attributes that can be provided are created in consideration of this variation. Can also be considered. For example, the video distribution server 100 is configured to execute user authentication processing for a client in response to receiving a connection request event from the client, and the user priority is set through this user authentication. You can do so.

(Embodiment 10)
In the third embodiment, among the distribution control processes shown in FIG. 11, video stream options that can be provided within the capacity of the video distribution server 100 are presented to the client by the processes of steps S1001 to S1010, among others. .

  Further, in the seventh embodiment, among the distribution control processes shown in FIG. 21, the video stream options that can be provided within the range of the video distribution server 100 are presented to the client by the processes of steps S5101 to S5106 in particular. .

  These processes can be considered as an evaluation process for creating options in the video distribution server 100.

  Further, in the sixth embodiment that describes the processing example on the client side, the option limiting process is performed in step S2204 in the multi-stream reception process illustrated in FIG. In the option limiting process, limited options are obtained particularly by the processes of steps S2301 to S2306 shown in FIG. Then, a video request event is issued in step S2205 of FIG. These can be thought of as a selection request process for selectively requesting a received stream.

  In the present embodiment, the means for realizing the evaluation process and the selection request process as described above are configured as independent processing modules, so that it is easy to give flexibility to the evaluation standard for options and the selection standard for received streams. To.

  The configuration of the video distribution system according to the present embodiment is the same as the configuration shown in FIG. The configuration of the client 109 is the same as the configuration of FIG. 14 described in the sixth embodiment. However, the contents stored in the memory 2003 of the client 109 are as shown in FIG. Although this is almost the same as FIG. 15 according to the sixth embodiment, as can be seen from comparison, FIG. 30 according to the present embodiment includes a selection request program 2145 instead of the benchmark program 2140.

  On the other hand, the contents stored in the memory 104 of the video distribution server 100 are as shown in FIG. This is substantially the same as FIG. 20 according to the seventh embodiment, but as can be seen from the comparison, FIG. 31 according to the present embodiment additionally includes an option evaluation program 240.

  FIG. 32 is a flowchart showing option evaluation processing by the video distribution server 100 in the present embodiment. A program corresponding to this flowchart is an option evaluation program 240.

  First, in step S6101, information necessary for evaluation (including connection information 260, a request image attribute extracted from a connection request event, an upper limit value X, and the like) is read. Next, in step S6102, options are obtained by the evaluation function S based on the read information.

  For example, in the third embodiment in which a video stream candidate that can be provided within the range of the available capacity of the video distribution server 100 is notified to the client as an option, the evaluation function S is the step S1002 shown in FIG. This is a function that realizes the respective processes of calculation of load F), step S1008 (calculation of remaining power Y), and step S1009 (generation of options).

  Also, video stream attribute candidates that can be provided within the capacity of the video distribution server 100 are determined in consideration of the priority information set for each attribute, and this is notified to the client as an option. Speaking of the seventh embodiment, the evaluation function S includes each process of step S5102 (calculation of the total load F), step S5104 (calculation of remaining power Y), and step S5105 (option creation process by priority) shown in FIG. It is a function that realizes.

  Since the option evaluation program 240 is a module independent of the distribution control program 220, it is much easier to arbitrarily change the evaluation function S as described above than when the option evaluation program 240 is incorporated in the distribution control program 220. Become. Accordingly, as long as the option evaluation program 240 has such an evaluation function S, the distribution control program 220 itself no longer needs to have a processing part realized by the evaluation function S.

  In step S6103, information about the obtained option is notified to the client that issued the connection request event.

  FIG. 33 is a flowchart showing selection request processing by the client 109 in this embodiment. The program corresponding to this flowchart is the selection request program 2145 and is executed by the CPU 2002.

  First, in step S6201, it is determined whether a connection request event has been issued. If a connection request event has not been issued, the process proceeds to step S6202, where a connection request event is issued and transmitted to the video distribution server 100 in the same process as step S2202 by the multi-stream reception program 2100 of the sixth embodiment. Thereafter, the selection request process is terminated.

  On the other hand, if the connection request event has not been issued in step S6201, the process proceeds to step S6203, and the requested video selection is performed by the evaluation function G. For example, in the case of the sixth embodiment, the evaluation function G corresponds to a process of calling the option limiting program 2120.

  Next, in step S6204, it is determined based on the result of the evaluation function G whether a video request is possible. If it is determined that a video request is not possible, the process advances to step S6205 to determine a connection request according to the result of the evaluation function G, and in step S6202, a connection request event is transmitted and issued, and this is transmitted to the video distribution server 100. Then, this selection request process is terminated. At this time, the connection request itself may be stopped depending on the result of the evaluation function G. On the other hand, if it is determined in step S6204 that a video request is possible based on the result of the evaluation function G, the process advances to step S6206 to issue a video request event in the same manner as in step S2205 by the multi-stream reception program 2100 of the sixth embodiment. Is transmitted to the video distribution server 100, and then the selection request process is terminated.

  The present embodiment is characterized in that, in the video distribution server 100, a processing portion related to option generation is configured by a module independent of the distribution control program 220. Thereby, an arbitrary evaluation function S can be described easily.

  Similarly, in the client 109, the selection request processing part related to the issuance of the connection request event and the video request event is configured by a module independent of the stream reception program 2100. Thereby, an arbitrary evaluation function G can be described easily.

(Embodiment 11)
Hereinafter, variations of the evaluation function S or G described in the tenth embodiment will be described.

  For example, it is possible to realize the update of the priority according to the payment of the price for the multi-stream distribution.

  First, in the same process as step S6201 by the selection request program 2145 of the tenth embodiment, the client 109 obtains user information by user authentication and consideration (amount) information for the current connection when making a first connection request. This is included in the connection request event and issued in step S6202.

  In response to this, the video distribution server 100 reads the information of the consideration paid for the connection request in the same process as step S6101 of the option evaluation program 240 of the tenth embodiment.

  The evaluation function S in step S6102 is almost equivalent to that in the tenth embodiment. However, the present embodiment includes a process of updating the priority related to the connection request according to the read consideration (the priority table 280 is updated). For example, the priority can be increased by 1 for every 100 yen of consideration. As a specific example, for example, a viewer with a priority of 10 can set the priority to 9 by paying 100 yen. In this way, the user can connect with a higher priority by paying a consideration.

  In addition, for example, it is possible to implement priority update by so-called point addition according to the number of connections.

  First, in the same process as step S6201 by the selection request program 2145 of the tenth embodiment, the client 109 includes user information by user authentication in a connection request event and issues it in step S6202 when making a first connection request. To do.

  In response to this, the video distribution server 100 reads the user authentication information for the connection request in the same process as step S6101 of the option evaluation program 240 of the tenth embodiment.

  The evaluation function S in step S6102 is substantially the same as in the tenth embodiment. However, according to the present embodiment, points corresponding to the number of connections are managed for each user, and the priority is updated according to the number of points (the priority table 280 is updated). For example, the priority of the user A is increased by 1 when the accumulated point of the user A reaches 100 points. To give a specific example, for example, a user of a viewer with a priority of 10 can set the priority to 9 by making 100 connections. In this way, the user can connect by increasing the priority by overlapping the connections.

  Further, for example, it is possible to realize priority update by bidding for multi-stream distribution.

  First, in the same process as step S6201 by the selection request program 2140 of the tenth embodiment, the client 109 receives user information based on user authentication and bid amount (auction) information for the current connection when making a first connection request. This is included in the connection request event and issued in step S6202.

  In response to this, the video distribution server 100 reads the bid amount information related to the connection request in the same process as step S6101 of the option evaluation program 240 of the tenth embodiment.

  The evaluation function S in step S6102 is substantially the same as in the tenth embodiment. However, this embodiment includes a process of updating the priority related to the connection request according to the read bid amount (the priority table 280 is updated.) For example, each of a plurality of users A, B, and C When a connection request is made at 100 yen, 200 yen, and 300 yen, the priority of C is set to 0, and the priority of A and B is set to 100, so that only the user C is substantially preferentially connected. it can.

  Further, for example, when the video distribution server 100 has no remaining capacity and gives up the connection, it is possible to realize the priority update associated with issuing points, coupons, etc. that can be used later.

  First, in the same process as step S6201 of the selection request program 2140 of the tenth embodiment, when the client 109 makes a first connection request, the client 109 sends user information based on user authentication and a specific stream connection request to the connection request event. This is issued in step S6202.

  In response to this, the video distribution server 100 reads the user information and the stream attribute in the same process as step S6101 of the option evaluation program 240 of the tenth embodiment.

  The evaluation function S in step S6102 is almost equivalent to that in the tenth embodiment. However, in the present embodiment, when a connection request to the received specific stream cannot be realized (when the request cannot be included in the option), a predetermined number of points are issued to the user, and the same user within a certain period of time is issued. Includes a process of updating the priority according to the point information when the connection is made (the priority table 280 is updated). For example, the priority can be increased by 1 per point. As a result, the connection can be realized in such a way that the user whose initial connection is rejected is rewarded.

  As described above, the specific priority is raised by consideration, the number of connections, points, etc. Of course, of course, the priority of clients (or users) other than the corresponding client (or user) is lowered. The same effect can be obtained also. In this case, as described in step S5705 by the priority-specific option creation program 230 in FIG. 27, the subtracted user may become a connection refusal target. It is possible to increase the degree. Note that when the connection is rejected in step S5706, the priority that is the object of connection rejection can be returned to the initial value. When not returning to the initial value, it is realized that the user returns to the original value by paying some consideration, points, etc., and when returning to the initial value, equality is realized in the initial connection.

Embodiment 12
Still another variation regarding the evaluation functions S and G will be described.

  FIG. 34 shows an example of stream load data in the present embodiment. This data structure is the same as the stream load data shown in FIG. 8, but as a variation of the image size 640 × 480, a video stream of 640 × 480 is added by expanding the video of 320 × 240. According to the figure, for a normal 640x480 video stream, the load with camera control is 17 and the load without camera control is 16, whereas the 320x240 video is enlarged to 640x480. As for the video stream, the load when the camera control is performed is 9 and the load when the camera control is not performed is 8. It can be seen that the load is greatly reduced. This is an example of a video stream having an image size of 640 × 480, and similar settings can be made for video streams of other sizes. However, those descriptions are omitted here.

Now, the client that issued the connection request event to be processed is 109-5, and another four clients (109-1 to 4) are currently connected. The connection information 260 at that time is shown in FIG. Assuming that The upper limit value X is set to 70. Referring to the connection information 260 in FIG.
(1) 800 × 600, no control (2) 160 × 120, with control (3) 640 × 480, no control (4) 800 × 600, it is understood that a video having an image attribute without control is requested. Referring to the stream load data 250 in FIG. 34, the load applied to each request is 25, 2, 16, 25. The total load F is F = 25 + 2 + 16 + 25 = 68. Here, it is assumed that the client 109-5 issues a connection request “640 × 480, with control”. The load f applied to the request of the client 109-5 is f = 17 with reference to the stream load data 250 in FIG. Then, in this case, F + f = 68 + 17 = 85, which exceeds the upper limit value X = 70, and the connection as requested by the client 109-5 must be refused as it is.

  Therefore, in the present embodiment, the normal “640 × 480, no control” is removed from the options once created in step S1009 in FIG. 11 based on the stream load data 250 in FIG. An extended version of "640x480, no control" is added. Further, for “640 × 480, with control” requested by the client 109-5, “640 × 480, without control”, which is an expanded version of 320 × 240, is used instead of the usual “640 × 480, without control”. If this is realized, the total load F + f when the connection of the client 109-5 is established becomes F + f = 25 + 2 + 8 + 25 + 9 = 69, and all connections can be made without exceeding the upper limit 70. After such adjustment, a connection request option change event may be issued.

  Therefore, in the present embodiment, the following processing is performed. First, the current overall load F (load applied to the clients 109-1 to 109-4) and the load for the connection request to be processed (load applied to the client 109-5) are estimated, and the total of both loads (that is, the client 109). It is determined whether the total load (when the connection of −5 is established) exceeds the upper limit value X. If the upper limit value X is exceeded, at least one of the candidates extracted once based on the stream load data is a load having an approximate attribute (enlarged image having a size smaller than the request). It asks other applicable clients and / or clients 109-5 to substitute by another small candidate. If another client and / or the client 109-5 responds to this request, and the overall load when the connection of the client 109-5 is established falls below the upper limit value X, the initial request from the client is satisfied. A close connection can be realized.

  Also, on the client side, if an evaluation function G as shown in step S6203 of FIG. 33, for example, when “640 × 480, no control” is requested, and it cannot be selected, “640 × 480, control of an expanded version of 320 × 240” An embodiment in which “None” is used instead and the selection is performed again in step S6205 is also possible.

(Embodiment 13)
In each of the above-described embodiments, the load on the video distribution server 100 is estimated based on a table (stream load data) in which the load for each case is described in advance. In this case, there is a difference between the estimated and actual depending on the connection status of the client. In fact, even if the server has sufficient processing capacity, the client connection may be refused, or the stream with a small amount of data In some cases, only options are generated.

  Therefore, in the present embodiment, by actually measuring the load state of the CPU of the video distribution server 100, it is possible to use the capacity of the server more efficiently by using this actually measured value.

  The configuration of the video distribution system according to the present embodiment is the same as the configuration shown in FIG. However, in the memory 104, as shown in FIG. 35, in addition to the multi-stream creation program 210, the distribution control program 220, the camera control program 710, and the connection information 260, the CPU load measurement program 1310 and the CPU load measurement program 1310 are stored. CPU load data 1320 generated by execution is stored. The CPU load data 1320 represents an index of CPU load related to data distribution per predetermined data transmission rate (for example, 1 kbps).

  FIG. 36 is a diagram illustrating a structure example of the CPU load data 1320. 3401 is a value indicating the processing capacity of the CPU in the initial state of the system (no-load data), 3405 is a value indicating the necessary processing capacity of the CPU when compressing an image of 800 × 600 size, and 3402 is an image of a size of 640 × 480. A value indicating the required processing capacity of the CPU when compressing, 3403 is a value indicating the required processing capacity of the CPU when compressing an image having a size of 320 × 240, and 3404 is a required process of the CPU when compressing an image having a size of 160 × 120 It is a value indicating the ability. Reference numeral 3406 denotes a value indicating the necessary processing capacity of the CPU at the time of distribution.

  Further, the connection information 260 in the present embodiment has a data structure as shown in FIG. In the connection information 260 in the present embodiment, as shown in the figure, the currently connected client, the image size requested from the client, and the image transmission rate related to the connection of the client are described.

  FIG. 37 is a flowchart showing multi-stream creation processing in the present embodiment. This flowchart is almost the same as that in FIG. 3 except that in this embodiment, the CPU load measurement program 1310 is started as step S1301 between steps S301 and S302, and the distribution control in step S361 is performed. Processing is performed according to the flowchart of FIG.

  In step S1301, when the CPU measurement program 1310 is activated, first, the CPU load in the initial state in which the compression process is not operating and the client is not connected is measured. Here, the CPU processing capacity is measured based on the number of times specific processing has been performed within a predetermined time, and the obtained value is stored in the memory 104 as upper limit data (no-load data) 3401 of the CPU processing capacity. Is done.

  Next, in order to measure the CPU load for the compression process, the compression process is started. At this time, the CPU load for each image size is measured by the same method as in the no-load state. As a result, the CPU load required for the compression processing is measured for each of four sizes of 160 × 120, 320 × 240, 640 × 480, and 800 × 600, and is stored in the memory 104 as compressed load reference value data 3404, 3403, 3402, and 3405, respectively. Is done.

  Next, the distribution control processing in step S361 will be described using the flowchart of FIG.

  First, in step S3301, the requested image attribute included in the connection request event is read, and in step S3302, connection information 260 as shown in FIG. 39 is read.

  Next, in step S3303, the load data obtained by multiplying the image data size transmitted for each connection of the connection information 260 and the CPU load data 250, and the CPU load data for compression for each attribute of the requested image are added. Thus, the total load F30 is obtained as a sum.

For example, in the case of the connection information 260 shown in FIG. 39, there is a connection between the clients (viewers) 109-1 and 109-2, requesting a video having an attribute of the image transmission rate 30 with images of 640x480 and 160x120 respectively. I understand that. Here, the image data size of 640 × 480 is D3001 kbyte, the image data size of 160 × 120 is D3003 kbyte, the distribution load data of CPU per unit transmission rate (3406), the load data (3402) for data compression of F3006, 640 × 480 is F3402, 160 × 120 Assuming that the load data (3404) for data compression is F3404, the overall load F30 is:
F30 = D3001 × 8 × F3006 × 30
+ D3003 × 8 × F3006 × 30
+ F3402 + F3404
It becomes.

Next, in step S3304, the load f30 for the attribute currently requested is obtained by the same method. For example, when an image having an image attribute of 320 × 240 is requested, if the image size is D3002 bytes and the load data (3403) required for data compression of 320 × 240 is F3403,
f30 = D3002 × 8 × F3006 × 30 + F3403
It becomes.

  In step S3305, the actually measured no-load data, that is, the upper limit value (3401) is read as X30, and in step S3306, this X30 is compared with F30 + f30.

  As a result of the comparison, if X30 is larger than F30 + f30, the upper limit of the server processing is not exceeded even if the currently requested connection is permitted. In this case, the process advances to step S3307, and the requested connection is additionally registered in the connection information 260 and stored in the memory 104. Thereafter, a video request event is generated in step S3308, and this distribution control process is terminated.

  On the other hand, as a result of the comparison in step S3306, if X30 is F30 + f30 or less, if the currently requested connection is permitted, the upper limit of the processing capacity of the server will be exceeded. Therefore, in this case, a rejection response to the connection request is returned in step S3309, and then this distribution control control process is terminated.

Here, when the image size of the requested connection is the same as that of the connection that has already been distributed, it is not necessary to add an additional CPU load for image compression. For example, when an image having an image size of 160 × 120 that has already been distributed is requested, if the image size is D3003 bytes and the image transmission rate is 30,
f30 = D3002 × 8 × F3006 × 30
It becomes.

  In the case of a system having a plurality of network interfaces, for example, a wired LAN connection, a wireless LAN connection, and a modem connection have large differences in the CPU load for data distribution. In such a case, it is possible to know a more accurate CPU load status by having distribution load information for each interface.

  In this embodiment, the load data of the CPU related to a specific data transmission rate is treated as constant, but by actually transmitting and measuring the load data of the CPU, it is possible to obtain a more accurate load status for the CPU. More suitable multi-stream delivery can also be performed.

  By doing as described above, multi-stream distribution becomes possible with an upper limit value of the processing capability of the video distribution system.

(Other embodiments)
As mentioned above, although embodiment of this invention was explained in full detail, this invention may be applied to the system comprised from several apparatuses, and may be applied to the apparatus which consists of one apparatus.

  In the present invention, a software program that realizes the functions of the above-described embodiments is directly or remotely supplied to a system or apparatus, and the computer of the system or apparatus reads and executes the supplied program code. Is also achieved. In that case, as long as it has the function of a program, the form does not need to be a program.

  Therefore, in order to realize the functional processing of the present invention with a computer, the program code itself installed in the computer and the storage medium storing the program also constitute the present invention. In other words, the claims of the present invention include the computer program itself for realizing the functional processing of the present invention and a storage medium storing the program.

  In this case, the program may be in any form as long as it has a program function, such as an object code, a program executed by an interpreter, or script data supplied to the OS.

  As a storage medium for supplying the program, for example, flexible disk, hard disk, optical disk, magneto-optical disk, MO, CD-ROM, CD-R, CD-RW, magnetic tape, nonvolatile memory card, ROM, DVD (DVD-ROM, DVD-R).

  As another program supply method, a client computer browser is used to connect to an Internet homepage, and the computer program of the present invention itself or a compressed file including an automatic installation function is downloaded from the homepage to a storage medium such as a hard disk. Can also be supplied. It can also be realized by dividing the program code constituting the program of the present invention into a plurality of files and downloading each file from a different homepage. That is, a WWW server that allows a plurality of users to download a program file for realizing the functional processing of the present invention on a computer is also included in the claims of the present invention.

  In addition, the program of the present invention is encrypted, stored in a storage medium such as a CD-ROM, distributed to users, and key information for decryption is downloaded from a homepage via the Internet to users who have cleared predetermined conditions. It is also possible to execute the encrypted program by using the key information and install the program on a computer.

  In addition to the functions of the above-described embodiments being realized by the computer executing the read program, the OS running on the computer based on the instruction of the program is a part of the actual processing. Alternatively, the functions of the above-described embodiment can be realized by performing all of the processes.

  Furthermore, after the program read from the storage medium is written to a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function expansion board or The CPU or the like provided in the function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

FIG. 1 is a configuration diagram of a video distribution system according to an embodiment of the present invention. FIG. 2 is a diagram illustrating programs and data stored in the memory of the video distribution server according to the first embodiment. FIG. 3 is a flowchart showing multistream creation processing by the video distribution server in the first embodiment. FIG. 4 is a diagram illustrating a structure example of stream load data according to the first embodiment. FIG. 5 is a diagram illustrating a structure example of connection information in the first embodiment. FIG. 6 is a flowchart showing the distribution control process in the first embodiment. FIG. 7 is a diagram illustrating programs and data stored in the memory of the video distribution server according to the second embodiment. FIG. 8 is a diagram illustrating a structure example of stream load data according to the second embodiment. FIG. 9 is a diagram illustrating a structure example of connection information in the first embodiment. FIG. 10 is a diagram illustrating programs and data stored in the memory of the video distribution server according to the third embodiment. FIG. 11 is a flowchart illustrating distribution control processing according to the third embodiment. FIG. 12 is a diagram illustrating a structure example of stream option information according to the third embodiment. FIG. 13 is a flowchart illustrating multistream creation processing according to the fourth embodiment. FIG. 14 is a diagram showing a configuration of a viewer (client) in the embodiment of the present invention. FIG. 15 is a diagram illustrating programs and data stored in the client memory according to the sixth embodiment. FIG. 16 is a flowchart illustrating multistream reception processing according to the sixth embodiment. FIG. 17 is a flowchart illustrating the option limiting process according to the sixth embodiment. FIG. 18 is a diagram illustrating a structure example of client stream load data according to the sixth embodiment. FIG. 19 is a diagram illustrating a structure example of stream option information according to the sixth embodiment. FIG. 20 is a diagram illustrating programs and data stored in the memory of the video distribution server according to the seventh embodiment. FIG. 21 is a flowchart showing the distribution control process in the seventh embodiment. FIG. 22 is a diagram illustrating an example of connection information in the seventh embodiment. FIG. 23 is a flowchart showing option creation processing by priority according to the seventh embodiment. FIG. 24 is a diagram illustrating a structure example of a priority table in the seventh embodiment. FIG. 25 is a diagram illustrating an example of priority-based stream load assumption data according to the seventh embodiment. FIG. 26 is a diagram illustrating an example of stream option information according to the seventh embodiment. FIG. 27 is a flowchart showing option creation processing by priority according to the eighth embodiment. FIG. 28 is a diagram illustrating a structure example of a priority table in the ninth embodiment. FIG. 29 is a diagram illustrating an example of priority-specific stream load assumption data according to the ninth embodiment. FIG. 30 is a diagram illustrating programs and data stored in the client memory according to the tenth embodiment. FIG. 31 is a diagram illustrating programs and data stored in the memory of the video distribution server according to the tenth embodiment. FIG. 32 is a flowchart illustrating option evaluation processing by the video distribution server according to the tenth embodiment. FIG. 33 is a flowchart illustrating selection request processing by a client according to the tenth embodiment. FIG. 34 is a diagram illustrating an example of stream load data according to the twelfth embodiment. FIG. 35 is a diagram illustrating programs and data stored in the memory of the video distribution server according to the thirteenth embodiment. FIG. 36 is a diagram illustrating a structure example of CPU load data according to the thirteenth embodiment. FIG. 37 is a flowchart illustrating multistream creation processing according to the thirteenth embodiment. FIG. 38 is a flowchart illustrating distribution control processing according to the thirteenth embodiment. FIG. 39 is a diagram illustrating a structure example of connection information in the thirteenth embodiment.

Claims (6)

A video distribution apparatus for distributing movies image stream,
Control means for controlling the imaging device;
A determination means,
Processing relating to distribution of video streams to other clients, whether or not to permit distribution of video streams together with control of the imaging apparatus to one client that has transmitted a request including a control request of the imaging device and a distribution request of the video stream A determination is made based on the total load, the processing load of the control means for controlling the imaging device, and the processing load related to the delivery of the video stream requested from the one client,
Whether or not it is possible to permit the delivery of the video stream without allowing the one client to control the imaging device, and the total value of the processing load related to the delivery of the video stream to the other client, And determining based on the processing load related to the delivery of the video stream requested by the one client.
Determination means capable of
The determination means determines that distribution of the video stream is not permitted together with the control of the imaging device to the one client, and the distribution of the video stream can be permitted without permitting the control of the imaging device. A notification means for notifying the one client of permission information indicating permission to distribute the video stream without allowing control of the imaging device;
A video distribution apparatus comprising: Whether the determination means can permit distribution of a second video stream having an image size smaller than that of the first video stream in response to the distribution request in conjunction with the control of the imaging device. It is possible to make a determination based on the total processing load related to the delivery of the video stream to the client, the processing load of the control means for controlling the imaging device, and the processing load related to the delivery of the second video stream And
The notification means determines that the determination means does not permit the delivery of the first video stream together with the control of the imaging device to the one client, and the first video stream together with the control of the imaging device. When it is determined that the distribution of the second video stream smaller than the image size of the second video stream can be permitted, the second video stream is distributed to the one client together with the control of the imaging device. The video distribution apparatus according to claim 1, wherein permission information indicating permission is notified . The notification means includes
Responding to a request from the one client according to a change in at least one of a processing load relating to the delivery of the video stream to the other client and an upper limit value of the processing load set in the video delivery device When the determination unit determines that distribution of the video stream is permitted together with the control of the imaging apparatus, permission information indicating that distribution of the video stream is permitted is notified together with the control of the imaging apparatus. Item 2. The video distribution device according to Item 1 . Having an actual measurement means for actually measuring the processing load related to the distribution of the video stream ;
The determination unit determines whether to permit the one client to distribute the video stream together with the control of the imaging device, based on a total value of the distribution load of the video stream to the other client measured by the actual measurement unit. The video distribution apparatus according to claim 1, wherein the determination is performed . A video distribution method for distributing movies image stream,
Control steps for controlling the imaging device;
A determination step,
Processing relating to distribution of video streams to other clients, whether or not to permit distribution of video streams together with control of the imaging apparatus to one client that has transmitted a request including a control request of the imaging device and a distribution request of the video stream A determination is made based on the total load value, the processing load of the control step for controlling the imaging device, and the processing load related to the delivery of the video stream requested by the one client;
Whether or not it is possible to permit the delivery of the video stream without allowing the one client to control the imaging device, and the total value of the processing load related to the delivery of the video stream to the other client, And determining based on the processing load related to the delivery of the video stream requested by the one client.
A determination step capable of
The determination step determines that distribution of the video stream is not permitted with the control of the imaging device to the one client, and distribution of the video stream can be permitted without permitting the control of the imaging device. A notification step for notifying the one client of permission information indicating that the distribution of the video stream is permitted without permitting the control of the imaging device;
A video distribution method comprising: To the computer that delivers the video stream,
A control procedure for controlling the imaging device;
Judgment procedure,
Processing relating to distribution of video streams to other clients, whether or not to permit distribution of video streams together with control of the imaging apparatus to one client that has transmitted a request including a control request of the imaging device and a distribution request of the video stream And determining based on the total load, the processing load of the control procedure for controlling the imaging device, and the processing load related to the delivery of the video stream requested from the one client,
Whether or not it is possible to permit the delivery of the video stream without allowing the one client to control the imaging device, and the total value of the processing load related to the delivery of the video stream to the other client, And determining based on the processing load related to the delivery of the video stream requested by the one client.
Judgment procedures that can be
According to the determination procedure, it is determined that distribution of the video stream is not permitted with the control of the imaging device for the one client, and distribution of the video stream can be permitted without permitting the control of the imaging device. A notification procedure for notifying the one client of permission information indicating that distribution of the video stream is permitted without permitting control of the imaging device;
A program characterized by having executed .

Images