JP2007325109A - Distribution server, network camera, distribution method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a system which increases the number of streams by minimizing the effect on the image quality of other effective stream video images even at the arrival of video data distribution requests in excess of a permissible band of network in the case that a distribution server connected to the network distributes different video data to a plurality of clients at the same time. <P>SOLUTION: The distribution server distributes a plurality of video data subjected to coding and compression to one or more apparatuses connected to the network at the same time, and is provided with: a means for classifying the plural video stream data into groups; and a control means for decreasing a bit rate of a particular stream data in one and same group. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a distribution server that can simultaneously transmit a plurality of video data through a network.

  Equipment that distributes video between devices connected to a network, such as a network camera that performs video compression on video captured by a camera and distributes video streams in response to requests from clients on the network, is practical It has become.

  Video data distributed from these devices will be distributed with higher definition and higher image quality as cameras and playback devices become more sophisticated, and high-speed network bandwidth will be required. I came.

  On the other hand, in the network infrastructure as well, broadband networks have become common in the home, and high-bandwidth network lines have been used for both wired LANs and wireless LANs.

  However, when data is transmitted from a remote location via the Internet network, a sufficient bandwidth may not be ensured depending on the communication path, or when a wireless LAN is used, the communication bandwidth depends on the surrounding environment. There are often cases where this is reduced.

  In such a case, a sufficient throughput cannot be obtained in response to a distribution request from a client, resulting in a significant deterioration in image quality.

  In order to solve this problem, as a technique for minimizing the degradation of image quality when the communication state deteriorates in video and audio distribution using a wireless LAN, a communication error is disclosed as in the invention disclosed in Patent Document 1. There has been proposed a method for variably controlling the bit rate according to the state.

  On the other hand, in a video distribution device such as a network camera, a system for simultaneously distributing different video data to a plurality of clients is being put into practical use.

For example, video distribution through broadcast radio waves uses a method of distributing video and audio programs as multiplexed data using the transmission band assigned to each channel, and each client (receiver) selects a channel. Thus, desired video and audio are reproduced.
As a method for reducing deterioration in image quality in these broadcasting systems, there is a method for controlling the bit rate distribution of a plurality of programs included in multiplexed data, as in the inventions disclosed in Patent Document 2 and Patent Document 3. Proposed.

Japanese Patent Laid-Open No. 2005-6286 JP-A-9-149388 JP 2001-136139 A

  However, in a system that distributes different video data from a server to a plurality of clients at the same time, the maximum bit rate that can be transmitted and received is limited depending on the inherent bandwidth and network state of the network to which each server and client are connected. For this reason, an increase in the number of connected clients and the number of request streams causes a reduction in image quality due to a communication error.

  This degradation in image quality occurs evenly in the stream distributed to all clients connected to the network that exceeds the bandwidth limit, regardless of whether the user operating the client wants to view high quality video. . As a result, the video quality is unstable for the user.

  In addition, when a new stream request is made from the client, if the existing stream that is currently distributed already uses the full communication bandwidth, the server may refuse to distribute the requested stream. appear. Alternatively, there is a case where only a specific stream is preferentially assigned a band according to a fixed priority, and the quality of the other streams is significantly degraded.

  An object of the present invention is to distribute different video data from a server to a plurality of clients at the same time, even if there is a video data distribution request exceeding the allowable bandwidth of the network, the effect on the image quality of other effective stream video is affected. It is to minimize and increase the stream.

The distribution server of the present invention is a distribution server capable of simultaneously distributing a plurality of encoded and compressed video data to one or a plurality of devices connected to a network, and means for classifying the plurality of video stream data into groups And control means for lowering the bit rate of specific stream data in the same group.
The network camera of the present invention is a network camera capable of simultaneously delivering encoded compressed stream data of an image obtained by imaging a subject at a wide angle and an image obtained by zooming to one or a plurality of devices connected to the network. Provided with means to classify video stream data into groups, and wide-angle shooting within the same group when the distribution bit rate exceeds the network bandwidth or when the packet drop rate exceeds a certain value due to changes in communication status Control means for lowering the bit rate of the video stream data or the zoomed video stream data is provided.
The distribution method of the present invention is a distribution method capable of simultaneously distributing a plurality of encoded and compressed video data to one or a plurality of devices connected to a network, wherein the plurality of video stream data are classified into groups. And a control step of reducing the bit rate of specific stream data in the same group.

  According to the present invention, even when there is a video data distribution request to the video distribution server exceeding the allowable bandwidth of the network, the number of streams can be increased while minimizing the influence on the image quality of other effective stream videos. . Therefore, the stream that the user wants to see can always be reproduced with high quality.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments to which the invention is applied will be described in detail with reference to the accompanying drawings.

(First embodiment)
FIG. 1 is a block diagram showing the overall configuration of the first embodiment of the present invention.
In the first embodiment, a system for simultaneously distributing different video stream data to a plurality of clients in a network camera will be described.

  In FIG. 1, reference numeral 101 denotes a part representing the entire network camera of the first embodiment. The network camera 101 includes an imaging unit 102, an image processing unit 103, an encoding unit 104, a network protocol processing unit 105, a MAC (Media Access Control) 106, a rate control unit 107, and a connection management information storage unit 108. Furthermore, the network camera 101 has a stream distribution information storage unit 109 and a CPU 110.

  The imaging unit 102 includes a lens for capturing a moving image, a CCD, a CCD control unit, and the like. The captured video is converted into RGB digital data by the imaging unit 101 and sent to the image processing unit 103.

  The image processing unit 103 performs processing for converting RGB digital data into digital data in the YCbCr 4: 2: 0 format, processing for converting the requested transmission video to an image size, various types of filter processing, and the like, and then processing to the encoding unit 104. send.

  The image processing unit 103 can output digital data having a plurality of different pixel sizes and image quality with respect to an input single captured video.

  The encoding unit 104 is an encoding processing unit for performing compression processing to the MPEG4 format. The encoding unit 104 performs encoding and compression processing on a plurality of digital data transmitted from the image processing unit 103 in a time division manner. MPEG4 stream data is generated.

  The generated MPEG4 stream data is sent to the network protocol processing unit 105, where it is converted into TCP / IP packet data, and distributed to the network 111 through the MAC 106.

  As shown in 114, the stream indicated by a solid line as the data to be delivered is a stream delivered with the same image quality and bit rate as the client request, and as shown by 115, the stream indicated by the broken line is a stream with a reduced bit rate. Show.

  The bit rate of the encoded data output from the encoding unit 104 is controlled by the rate control unit 107 according to an instruction from the CPU 110.

  The rate control unit 107 performs control for changing the MPEG4 quantization step and scheduling control for changing the frame rate to the encoding unit 104 so as to satisfy the target bit rate.

  The connection management information storage unit 108 is an area in which static network information related to the connection client is stored for each distribution stream, and details will be described later with reference to FIG.

  The information stream distribution information storage unit 109 is an area for storing dynamically changing information such as stream meta information, reproduction state, and communication state for each distribution stream, and details will be described later with reference to FIG.

  These storage units are referred to and updated by the CPU 110 or the network protocol processing unit 105.

  As described above, the CPU 110 refers to data in the connection management information storage unit 108 and the stream distribution information storage unit 109 to manage connection clients. In addition, the CPU 110 performs processing for updating data in these storage units according to changes in the network status and client status, and processing for instructing the target bit rate to the rate control unit 107.

  In addition, various network server protocols are executed. Details of processing contents of the CPU 110 will be described later with reference to flowcharts.

  Reference numeral 112 denotes a client A which is a device for receiving and reproducing video stream data distributed from the network camera 101, and 113 denotes a client B.

  These clients are terminals having a video playback function connected to a network, such as a dedicated viewer, a PC application, and a network compatible digital TV.

  Reference numeral 111 denotes a network to which the network camera 101 and each of the clients 112 and 113 are connected, and includes a local network and the Internet.

Next, FIG. 2 shows details of the connection management information storage unit 108.
In FIG. 2, 201 represents an information storage area related to stream 1, 202 represents an information storage area related to stream 2, and 203 represents an information storage area related to stream 3. Management information is stored.

  Reference numeral 201a denotes a transmission destination IP address for the stream 1, in which an IP address of a transmission destination terminal is stored.

  201b is a user name for the stream 1, and is an ID for identifying a user who is using the distribution stream.

As this user name, for example, a login name managed by the OS of the playback terminal is assumed.
Reference numeral 201c denotes a transmission destination port number for the stream 1, and stores the port number on the client side when the connection is established by the TCP / IP protocol.

  Reference numeral 201d denotes a protocol number for the stream 1, which stores a number for identifying a protocol (TCP or UDP, etc.) used in the connection.

  201e is an application ID for the stream 1, and is an ID for identifying an application using the distribution stream.

  This application ID is acquired in the stream distribution request negotiation with the client. These pieces of information are similarly stored for stream 2 (202) and stream 3 (203).

FIG. 3 shows details of the stream distribution information storage unit 109.
In FIG. 3, reference numeral 301 denotes an information storage area related to stream 1. Reference numeral 302 denotes an information storage area related to the stream 2. Reference numeral 303 denotes an information storage area related to the stream 3, and similarly to the connection management information storage unit 108, management information is stored for each distributed stream.

  301a is an average bit rate for the stream 1, and stores the value of the target bit rate instructed from the CPU 110 to the rate control unit 107 in correspondence with the image quality requested by the client.

  301b is a frame rate for the stream 1, and similarly stores a target frame rate value set in the encoding unit 104 by the CPU 110 in correspondence with the frame rate of the video requested by the client.

  301c is the pixel size for the stream 1, and stores the value of the output pixel size set by the CPU 110 in the image processing unit 103 and the encoding unit 104 corresponding to the pixel size of the video requested by the client. .

  301d represents the playback state for stream 1, and indicates whether the video stream being distributed is active on the client side.

  For example, when the distribution video is displayed in a window on the client side, it can be considered active if the window is selected, and inactive if the window is not selected. In picture-in-picture display, the video displayed on the main screen may be active and the sub-screen display may be inactive. A screen example will be described later.

  This playback state is notified to the network camera side when the playback state changes on the client side.

  Reference numeral 301e denotes a packet loss rate for the stream 1, and by monitoring this packet loss rate, the state of the network up to the client can be known.

  A high missing rate indicates that packets are congested or lost due to lack of bandwidth or network disconnection somewhere in the network.

  This missing rate can be obtained, for example, by using a reception quality report defined by the RTCP protocol.

  These pieces of information are stored in the same way for stream 2 (302) and stream 3 (303).

  Next, in FIG. 4 to FIG. 6, software control executed by the CPU 110 shown in FIG. 1 will be described in detail with reference to flowcharts.

  FIG. 4 is a control flow when a new video stream distribution request is made from a client.

  In FIG. 4, in step S401, the network camera executes a process of waiting for a stream distribution request from a client.

  If a distribution request is accepted in step S402, processing for checking the margin of the communication band of the network to which the network camera is connected is executed in step S403.

  This margin check compares the total sum of the average bit rates (301a) of each stream stored in the stream distribution information storage unit 109 described in FIG. 3 with the value of the effective bandwidth of the connection network. Is done.

  Here, if it is exceeded in step S404, that is, if the total value of the bit rates exceeds a certain safety factor with respect to the effective band, the process proceeds to step S405.

  If not, in step S415, a request stream generation instruction from the client, that is, the video stream data having the required pixel size, frame rate, and required image quality is generated for the image processing unit 103 and the encoding unit 104. Set to

  In step S416, the network protocol processing unit 105 executes processing for distributing the stream generated in S415 to the network.

  If the connection network bandwidth is exceeded in S404, first, in step S405, a grouping policy is acquired.

  The grouping policy is a unit in which each stream that is distributed is grouped.

This assumes the following case. That is,
-Streams distributed to terminals connected to the same subnet as a group-Streams used by the same user as a group-Delivered to the same terminal, that is, the same IP address A stream that is used by the same application on the same terminal as a group

  Each stream can be grouped by referring to information stored in the connection management information storage unit 108.

  If the acquired grouping policy is the same subnet in step S406, all inactive streams are extracted from the streams distributed to the same subnet as the distribution destination of the new request stream in step S407. Process.

  Subsequently, in step S414, processing for reducing the bit rate of the extracted stream is performed.

  This process is realized by instructing the rate control unit 107 to lower the target bit rate of the target stream.

  If the acquired grouping policy is the same user in S408, all inactive streams are extracted from the streams distributed to the same user as the user who uses the new request stream in step S409. Process. Then, the process proceeds to step S414.

  If the acquired grouping policy is the same terminal in S410, all inactive streams are extracted from the streams distributed to the same terminal as the distribution destination terminal of the new request stream in step S411. Perform the process. Then, the process proceeds to step S414.

  If the acquired grouping policy is the same application in S412, the process proceeds to step S413. In step S413, a process of extracting all inactive streams from the streams distributed to the same application as the application using the new request stream is performed. Then, the process proceeds to step S414.

  When the bit rate lowering process in S414 ends, the process proceeds to a stream generation instruction for a reduced bit rate in S415 and a distribution instruction in S416, and the process ends.

  FIG. 5 illustrates a control flow in the case where a distribution stop request for a certain stream among video streams that have already been distributed is made from the client, contrary to FIG. 4.

  In FIG. 5, in step S501, the network camera executes a process of waiting for a stream distribution stop request from the client.

  If a distribution stop request is accepted in step S502, the process proceeds to step S503. In step S503, processing for stopping the generation of the stop request stream, that is, processing for setting the image processing unit 103 and the encoding unit 104 so as to stop generation of the requested stream data is executed.

  In step S504, the network protocol processing unit 105 is subjected to processing for stopping distribution.

  Next, in step S505, a process of checking the margin after stopping the distribution of the communication band of the network to which the network camera is connected is executed.

  This margin check is performed by a process similar to the process described in step S403 in FIG.

  Here, in step S506, if there is a margin in the band, that is, if the total value of the bit rates is below a certain safety factor with respect to the effective band, the process proceeds to step S507. If it is still over, the process is terminated as it is.

  In S507, the grouping policy is acquired. This acquisition of the grouping policy is realized by the same processing as in S405.

  If the acquired grouping policy is in the same subnet, in step S508, processing for extracting all inactive streams from the streams distributed to the same subnet as the distribution destination of the new request stream is performed. Subsequently, in step S516, processing is performed to return the bit rate of the extracted stream to the standard state, that is, the bit rate of the specification first requested by the client.

This process is performed by an instruction to the rate control unit 107 as in S414.
If the acquired grouping policy is the same user in S510, the process proceeds to step S511. In step S511, a process for extracting all inactive streams from the streams distributed to the same user as the user who uses the new request stream is performed, and after the process proceeds to step S516, the process is terminated.

  If the acquired grouping policy is the same terminal in S512, the process proceeds to step S513. In step S513, processing for extracting all inactive streams from the streams distributed to the same terminal as the distribution destination terminal of the new request stream is performed, and the process ends after proceeding to step S516. .

  If the acquired grouping policy is the same application in S514, the process proceeds to step S515. In step S515, a process of extracting all inactive streams from the streams distributed to the same application as the application using the new request stream is performed, and the process proceeds to step S516. finish.

Next, FIG. 6 illustrates a control flow when the reproduction state of the client changes.
In FIG. 6, in step S601, processing for monitoring the reproduction state of the client is performed. In this method, a control command connection is established with respect to a client, and a process of waiting for a command for notifying a change in a reproduction state through a network is executed.
The change in the reproduction state means that the reproduction state 301d in the stream distribution information storage unit 109 described with reference to FIG. 3 changes from active to inactive or from inactive to active.

  Here, in step S602, when a change in the playback state is detected, that is, when a command indicating that the change has been received is received, a grouping policy is acquired in step S603. This acquisition of the grouping policy is realized by the same processing as in S405.

  If the acquired grouping policy is in the same subnet, in step S605, processing is performed to extract a stream whose playback state has been changed from streams distributed to the same subnet as the distribution destination of the new request stream.

  Subsequently, in step S612, it is checked whether or not the bit rate control is performed on the streams in the group. If the bit rate control is performed, the process proceeds to the bit rate reset processing in steps S613 and S614. move on. If the bit rate control is not performed, the process ends.

  If the acquired grouping policy is the same user in S606, the process proceeds to step S607. In step S607, processing for extracting a stream whose playback state has been changed from streams distributed to the same user as the user who uses the newly requested stream is performed, and the flow proceeds to step S612.

  If the acquired grouping policy is the same terminal in S608, the process proceeds to step S609. In step S609, processing for extracting a stream whose playback state has been changed from the streams distributed to the same terminal as the distribution destination terminal of the new request stream is performed, and the process proceeds to step S612.

  If the acquired grouping policy is the same application in S610, the process proceeds to step S611. In step S611, processing for extracting a stream whose playback state has been changed from streams distributed to the same application as the application using the new request stream is performed, and the process proceeds to step S612.

  In the bit rate resetting process of S613 and S614, first, in step S613, a process for instructing the rate control unit 107 to reduce the bit rate of the extracted stream that has changed to inactive is executed.

  Next, in step S614, a process for instructing the rate control unit 107 to return the bit rate of the actively changed stream to the standard state, that is, the bit rate of the specification first requested by the client is executed.

  FIG. 7 shows an example of a display screen when the client is displaying a video stream in the control process described with reference to the flowchart of FIG.

  FIG. 7 shows an example of multi-window display. In FIG. 7, reference numeral 701 denotes a display area of the entire client screen. Three display windows 702, 703, and 704 are opened in this display area, and stream data of video 1, video 2, and video 3 are displayed independently in each window.

Reference numerals 702a and 704a denote title bars of the respective windows 702 and 704, which display title character strings of stream names “video 1”, “video 2”, and “video 3”, respectively.
Since the window 704 is in an active state at this time, the title bar 704a is displayed in the selected color while being displayed in the foreground.

  On the other hand, FIG. 8 shows an example of picture-in-picture display. In FIG. 8, similarly to FIG. 7, reference numeral 801 denotes a display area of the entire client screen, in which a main screen display video 802 (video 1) and a sub screen display video 803 (video 2) are reproduced. It is displayed. In this example, the main screen is in an active state.

(Second Embodiment)
In the first embodiment, the embodiment of the distribution bit rate control process when the request stream from the client is increased or decreased has been described, but the present invention is not limited to this.

  In a network in which a network camera and a video playback client are connected, there are cases where the distribution route passes through a network with a narrow band, or communication quality may be deteriorated due to a change in the surrounding environment. In such a case, a packet error or transmission delay of the distribution video data may occur.

  In such a situation, for example, when the video data is compressed by an encoding method using the difference between the preceding and subsequent frames and the peripheral pixel data such as MPEG2 and MPEG4, the video quality is particularly deteriorated.

  Therefore, in the second embodiment, an example will be described in which a decrease in video quality is reduced by performing bit rate control in response to changes in these communication conditions.

  The change in the communication status can be realized by monitoring an error rate using a reception report defined by RTCP, which is an RTP control protocol, for example. The configuration of the RTCP reception report will be described later.

  FIG. 9 is a flowchart showing control of software executed by the CPU 110 in the second embodiment of the present invention.

  In FIG. 9, in step S901, processing for checking all packet loss rates of each stream currently being distributed is performed.

  This packet loss rate is realized by reading the packet loss rate field 301e stored in the stream distribution information storage unit 109, and this field is updated every time the above-described RTCP reception report is received. Therefore, it is possible to always know the latest missing rate.

  If it is determined in step S902 that there is a stream having a missing rate higher than a certain threshold, a grouping policy is acquired in step S903. This acquisition of the grouping policy is realized by the same processing as in S405.

  When the acquired grouping policy is the same subnet, in step S905, all the streams that are inactive among the streams that are distributed to the same subnet as the distribution destination of the stream whose drop rate exceeds the threshold value are all determined. Perform the extraction process.

  Subsequently, in step S912, it is checked whether or not bit rate control is being performed on the streams in the group. If bit rate control is being performed, the process proceeds to a bit rate reduction process in step S913. If the bit rate control is not performed, the process ends.

  If the acquired grouping policy is the same user in S906, the process proceeds to step S907. In step S907, processing is performed to extract all inactive streams from streams distributed to the same user as the stream user whose drop rate exceeds the threshold, and the process proceeds to step S912.

  If the acquired grouping policy is the same terminal in S908, the process proceeds to step S909. In step S909, a process of extracting all inactive streams from the streams distributed to the same terminal as the distribution destination terminal of the stream whose drop rate exceeds the threshold is performed, and the process proceeds to step S912.

  If the acquired grouping policy is the same application in S910, the process proceeds to step S911. In step S911, a process of extracting all inactive streams among streams distributed to the same application as the application using the stream whose drop rate exceeds the threshold is performed, and the process proceeds to step S912. move on.

  In the bit rate reduction processing of S913, processing for instructing the rate control unit 107 is executed to reduce the bit rate of the stream that has changed to inactive among the extracted streams.

  Next, FIG. 10 shows a packet structure of an RTCP reception report used for collecting the packet drop rate.

  In FIG. 10, reference numeral 1001 denotes V (version number field), which stores the RTP version number.

  Reference numeral 1002 denotes P (padding field), and reference numeral 1003 denotes RC (reception report block number field), which stores the number of reception reports included in this packet.

  Reference numeral 1004 denotes a PT (packet type field), which stores 201 indicating a reception report (RR).

  A length field 1005 stores the length of the content after the packet common header.

  Reference numeral 1006 denotes an SSRC (identifier) of the report creator, and reference numeral 1007 denotes an SSRC (identifier) field of the report subject.

  Reference numeral 1008 denotes a missing rate field, which stores a value obtained by dividing the number of missing packets by the number of packets to be received.

  Further, reference numeral 1009 denotes a cumulative lost packet number field, which stores the cumulative total number of packets lost in the RTP session.

  Reference numeral 1010 denotes a maximum extension sequence number field, 1011 denotes a packet interval jitter field, 1012 denotes a time stamp field of the latest transmission report, and 1013 denotes a latest transmission report elapsed time field, and a detailed description thereof is omitted.

  The field from the report target person's SSRC (1007) to the latest transmission report elapsed time (1013) becomes one unit of the reception report block, and then continues to the reception report block 2 (1014).

(Third embodiment)
In the third embodiment, an application example to a network camera capable of simultaneously distributing a wide-angle video stream and a zoom video stream in the present invention will be described.

  When the distribution video from the network camera is played back and displayed on the client, a wide-angle video for grasping the entire subject being photographed by the camera and a zoom video for viewing the target portion in detail are arranged on the display screen. Sometimes you want to watch at the same time.

  In the first and second embodiments, when video distribution exceeding the allowable bandwidth of the network occurs, the bit rate of the distribution stream is set according to the playback state (active or inactive) of the client in order to prevent the video quality from degrading. The case of controlling was explained. In contrast, in the third embodiment, an embodiment will be described in which the bit rate of a wide-angle video is controlled in a network camera that can simultaneously deliver a wide-angle video and a zoom video.

  FIG. 11 is a diagram illustrating detailed blocks of the imaging unit 102 and the image processing unit 103 according to the third embodiment in the overall configuration of the network camera illustrated in FIG. 1.

  In FIG. 11, reference numeral 1101 denotes a wide-angle imaging unit which is a camera unit for taking a wide-angle video.

  The wide-angle imaging unit 1101 includes a lens group 1 (1102), a CCD 1 (1103), a CCD control unit 1 (1104), and an image processing unit 1 (1105). The lens group 1102 is composed of a plurality of lenses for optically projecting a subject image onto the CCD 1102, and has a characteristic of capturing a subject at a wide angle.

  A CCD (photoelectric conversion element) 1103 is an element for converting a captured image projected by the lens group 1102 into an analog electric signal, and has a VGA size (640 × 480 dots) pixel count.

  Data can be read out at a rate of 30 frames / second. The CCD control unit 1104 includes a timing generator for supplying a transfer clock signal and a shutter signal to the CCD 1103, a circuit for performing noise removal and gain processing of the CCD output signal. Further, the CCD control unit 1104 includes an A / D conversion circuit for converting an analog signal into a 10-bit digital signal. The image processing unit 1105 performs image processing such as gamma conversion, color space conversion, white balance, AE, and flash correction on the 10-bit digital signal output from the CCD control unit 1104. The image processing unit 1105 outputs an 8-bit digital signal in the YCbCr (4: 2: 0) format.

  This wide-angle video image is output at a cycle of 30 frames / second with the number of pixels of VGA size (640 × 480 dots).

  Reference numeral 1106 denotes a zoom imaging unit which is a camera unit for photographing at a desired zoom angle of view.

  As with the wide-angle imaging unit 1101, the zoom imaging unit 1106 includes a lens group 2 (1107), a CCD 2 (1108), a CCD control unit 2 (1109), and an image processing unit 2 (1110). Further, the zoom imaging unit 1106 includes a control motor 1111 for panning and tilting operations that move the shooting direction.

  The lens group 1107 is configured as a zoom lens having characteristics capable of photographing at a normal angle of view.

  The CCD 1108 has an SXGA size (1280 × 1024) pixel count, and can be read out at 30 frames / second.

  The CCD control unit 2 (1109) and the image processing unit 2 (1110) perform the same operation as each block of the wide-angle imaging unit 1101.

  The pan / tilt control motor 1111 includes two motors, a motor for rotating the camera unit in the horizontal direction and a motor for rotating the camera unit in the vertical direction, and respective drive circuits.

  As described above, the network camera according to the present embodiment is equipped with two image capturing units capable of capturing images at different angles of view, and digital data output from each image capturing unit is sent to the encoding unit 104. It is done.

  Next, FIG. 12 shows a control flow when a new video stream distribution request is made from a client in the third embodiment.

  In FIG. 12, in step S1201, the network camera executes a process of waiting for a stream distribution request from a client.

  If a distribution request is accepted in step S1202, processing for checking the margin of the communication band of the network to which the network camera is connected is executed in step S1203.

  This margin check is performed in the same procedure as the process described in S403 of FIG. Here, if it is exceeded in step S1204, that is, if the total value of the bit rates exceeds a certain safety factor with respect to the effective band, the process proceeds to step S1205.

  If not, in step S1215, a request stream generation instruction from the client, that is, video stream data having the required pixel size, frame rate, and required image quality is generated for the image processing unit 103 and the encoding unit 104. Set up.

  In step S1216, the network protocol processing unit 105 executes processing for distributing the stream generated in S1215 to the network.

  If the connection network bandwidth is exceeded in S1204, first, in step S1205, a grouping policy is acquired. The acquisition of the grouping policy is performed in the same procedure as in FIG. 4 and S405.

In step S1206, if the acquired grouping policy is the same subnet, in step S1207, processing for extracting all wide-angle shot video streams from the streams distributed to the same subnet as the distribution destination of the new request stream is performed. Do.
Subsequently, in step S1214, processing for lowering the bit rate of the extracted stream is performed.

  This process is realized by instructing the rate control unit 107 to lower the target bit rate of the target stream.

  In step S1208, if the acquired grouping policy is the same user, the process advances to step S1209. In step S1209, a process of extracting all wide-angle shot video streams from the streams distributed to the same user as the user who uses the new request stream is performed, and the process proceeds to step S1214.

  If the obtained grouping policy is the same terminal in S1210, the process proceeds to step S1211. In step S1211, a process of extracting all wide-angle shot video streams from the streams distributed to the same terminal as the distribution destination terminal of the new request stream is performed, and the process proceeds to step S1214.

  If the acquired grouping policy is the same application in step S1212, the process advances to step S1213. In step S1213, a process of extracting all wide-angle shot video streams from the streams distributed to the same application as the application using the new request stream is performed, and the process proceeds to step S1214.

  When the bit rate lowering process in S1214 ends, the process proceeds to a stream generation instruction for a reduced bit rate in S1215 and a distribution instruction in S1216, and the process ends.

  FIG. 13 shows an example of a display screen when the client is displaying a video stream in the third embodiment.

  FIG. 13 shows an example of multi-window display as in FIG. 7. In FIG. 13, reference numeral 1301 denotes a display area of the entire client screen.

  In this display area, two display windows of a zoom video screen shown at 1302 and a wide-angle video screen shown at 1303 are opened. In each window, a zoom video stream captured and encoded by the zoom imaging unit 1106 of the network camera and a wide-angle video stream captured and encoded by the wide-angle imaging unit 1101 are independently displayed and reproduced.

(Other embodiments)
In the first and second embodiments, the active stream in each group is a stream distributed to the active window and a stream of the main screen in the picture-in-picture screen. In the third embodiment, the zoom video stream is used, but the present invention is not limited to this. For example, in the case where a stream for performing display reproduction in real time and a stream for storing and storing are distributed together, an embodiment in which the display reproduction stream is subject to bit rate control is also conceivable. Needless to say.

  Further, the embodiment has been described in which the RTCP reception report is used as means for knowing the quality of the communication band. However, the present invention is not limited to this. For example, in the case of stream distribution using TCP, the network protocol processing unit 105 monitors the frequency of TCP retransmission requests for each connection of each stream. Can be realized.

Alternatively, the communication quality can be monitored by the network camera and each client negotiating with a unique application.
Further, in the first to third embodiments, the bit rate control range is described as an embodiment limited to a group to which a newly added stream belongs or a group to which a stream with an increased communication error rate belongs. did. However, the present invention is not limited to this, and if a sufficient communication band cannot be secured even if bit rate control is performed within the same group, the same bit rate control is performed for other groups. It goes without saying that such an embodiment is also conceivable.

  In the first to third embodiments, the bit rate control method has been described as changing the MPEG quantization step or changing the frame rate by the control from the rate control unit 107. However, the present invention is not limited to this, and it is needless to say that changes in the pixel size of the video and switching to a compression method with higher encoding efficiency are included.

  Further, as a group of video stream data to be distributed, streams configured by multiplexing a plurality of video data are classified as one group. An embodiment in which the bit rate is controlled to be variable for each video data unit that is multiplexed and stored is also conceivable.

  Another object of the present invention is to supply a storage medium storing software program codes for realizing the functions of the above-described embodiments to a system or apparatus, and a computer such as the system reads and executes the program codes from the storage medium. Is also achieved.

  In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the program code itself and the storage medium storing the program code constitute the present invention.

  As a storage medium for supplying the program code, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.

  In addition, the case where the functions of the above-described embodiment are realized by performing part or all of the actual processing by an OS or the like running on the computer based on the instruction of the program code read by the computer. It is.

  Further, after the program code read from the storage medium is written in a memory provided in a function expansion unit connected to the computer, the CPU or the like performs actual processing based on the instruction of the program code, and the above-described processing is performed. The case where the functions of the embodiment are realized is also included.

It is a block diagram showing the whole structure by the 1st Embodiment of this invention. It is a figure which shows the detail of the connection management information storage part by the 1st Embodiment of this invention. It is a figure which shows the detail of the stream delivery information storage part by the 1st Embodiment of this invention. It is a flowchart showing the control when the video stream delivery request | requirement is newly made from the client by the 1st Embodiment of this invention. It is a flowchart showing the control when the delivery stop request | requirement of the stream is performed from the client by the 1st Embodiment of this invention. It is a flowchart showing the control when the reproduction | regeneration state of a client changes by the 1st Embodiment of this invention. It is a figure which shows an example of the multi-window display screen when the client by the 1st Embodiment of this invention is displaying the video stream. It is a figure which shows an example of the picture in picture display screen in case the client by the 1st Embodiment of this invention is displaying the video stream. It is a flowchart showing control of the software which CPU performed by the 2nd Embodiment of this invention. It is a figure which shows the packet structure of the RTCP reception report by the 2nd Embodiment of this invention. It is a figure showing the detailed block of the imaging part by the 3rd Embodiment of this invention, and an image process part. It is a flowchart showing the control when the video stream delivery request | requirement is newly made from the client by the 3rd Embodiment of this invention. It is a figure which shows an example of the multi-window display screen when the client by the 3rd Embodiment of this invention is displaying the video stream.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 Network camera 102 Image pick-up part 103 Image processing part 104 Coding part 105 Network protocol processing part 106 MAC
107 rate control unit 108 connection management information storage unit 109 stream distribution information storage unit 110 CPU
111 Network 112 Client A
113 Client B

Claims (14)

In a distribution server capable of simultaneously distributing a plurality of encoded and compressed video data to one or a plurality of devices connected to a network,
Means for classifying the plurality of video stream data into groups;
A distribution server comprising: control means for lowering a bit rate of specific stream data in the same group.   Video stream data distributed to the same subnet, video stream data used by the same user, video stream data distributed to the same device, or video stream data used by the same application 2. The distribution server according to claim 1, wherein a plurality of pieces of video data stored in a multiplexed manner are classified as the same group.   2. The distribution server according to claim 1, wherein the specific stream data for lowering the bit rate is determined by a reproduction state of a terminal that receives the stream.   The playback state of the terminal that receives the stream is the state of the active window or the inactive window in the multi-window display, and includes control means for lowering the bit rate of the video stream that is played back and displayed in the inactive window. The distribution server according to claim 3, wherein   The playback state of the terminal that receives the stream is the state of the main screen or the sub screen in picture-in-picture display, and includes control means for reducing the bit rate of the video stream that is played back and displayed on the sub screen. The delivery server according to claim 3.   In the case where video stream data for playback display and video stream data for storage are distributed independently, the specific stream data for lowering the bit rate should be video stream data for playback display. The distribution server according to claim 1, wherein:   2. The distribution according to claim 1, wherein when the distribution bit rate exceeds the network bandwidth due to an increase in the number of requested video streams, control is performed to lower the bit rate of specific stream data in the same group. server.   2. The distribution server according to claim 1, wherein when the packet loss rate exceeds a certain value due to a change in communication state, control is performed to lower the bit rate of specific stream data in the same group.   The distribution server according to claim 1, wherein the control for lowering the bit rate is performed by changing a quantization step, changing a frame rate, or changing a pixel size. In a network camera capable of simultaneously delivering encoded compressed stream data of an image captured at a wide angle and a zoomed image to one or more devices connected to the network,
Means for classifying the plurality of video stream data into groups;
If the delivery bit rate exceeds the network bandwidth, or if the packet drop rate exceeds a certain value due to a change in communication status, the bit of wide-angle video stream data or zoom video stream data within the same group A network camera comprising control means for reducing a rate.   Video stream data distributed to the same subnet, video stream data used by the same user, video stream data distributed to the same device, or video stream data used by the same application The network camera according to claim 10, wherein a plurality of video data stored in a multiplexed manner is classified as the same group.   The network camera according to claim 10, wherein the control for lowering the bit rate is performed by changing a quantization step, changing a frame rate, or changing a pixel size. In a distribution method capable of simultaneously distributing a plurality of encoded and compressed video data to one or a plurality of devices connected to a network,
Classifying the plurality of video stream data into groups;
And a control step of reducing the bit rate of specific stream data in the same group.   The program for making a computer perform each process of the delivery method of Claim 13.

Images