JP2008079061A - Image streaming system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To obtain an image streaming system for smoothly streaming an image data. <P>SOLUTION: In the image streaming system for streaming image data from a transmission server to a reception client, a transmission server 2 includes a moving image coding unit 22 for generating the image data of an intra-frame coding data and inter-frame prediction coding data, and a server controller 24 for streaming the generated image data to a reception client 3 and for storing the intra-frame coding data among the image data to be streamed to the reception client 3. When the data to be streamed to the reception client already connected to the self-server is in the timing for other than the intra-frame coding data, the server controller 24 starts streaming to a new reception client, using the stored intra-frame coding data. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image streaming system for streaming a compression-coded image from a transmission server to a reception client.

  2. Description of the Related Art In recent years, with the expansion of networks such as the Internet, intranets, and mobile phone networks, an environment where voice and images can be communicated in real time is being prepared. For example, in a streaming system (image streaming system) that streams compression-encoded images, image data sent from a transmission server is streamed to a receiving client via a network.

  A conventional image streaming system will be described with reference to FIGS. FIG. 11 is a diagram for explaining a first example of conventional image streaming, FIG. 12 is a flowchart showing a processing procedure of a first example of conventional image streaming, and FIG. 13 is a conventional image streaming. FIG. 14 is a flowchart illustrating a processing procedure of a second example of conventional image streaming. 12 and 14 show a processing procedure of moving image streaming control (moving image streaming control) in a conventional transmission server.

  First, moving image streaming control (first example) to a connected receiving client will be described as conventional image streaming. As shown in FIG. 12, the transmission server of the image streaming system first performs image input processing from a camera or the like (S50). Then, the transmission server determines whether a new distribution request (new distribution request) has been received from the receiving client (S51).

  If a new distribution request has not been received from the receiving client (S51, NO), the transmission server determines whether or not to send (stream) a key frame (S52).

  If it is determined that no key frame is to be transmitted (S52, NO), inter-frame predictive coding (inter-coding or motion compensation predictive coding) processing is performed (S53). On the other hand, if it is determined that a key frame is to be transmitted (S52, YES), intra-frame coding (intra coding) processing is performed (S54).

  After the process of S54 or S53, the moving image encoded data is subjected to server processing (streaming the moving image data to the receiving client via the network) (S55).

  After streaming to the network, image input processing from the camera or the like is performed again (S50), and the above-described series of processing (S50 to S55) is repeatedly executed continuously. Thereby, streaming from the transmission server to the reception client is continued. Note that the image input process shown in S50, the encoding process shown in S53 and S54, and the server process shown in S55 may be processed in parallel in the transmission server.

  Here, the above-described intraframe coding, interframe prediction coding, and key frame will be described. For example, as represented by MPEG-4 (Moving Picture Experts Group phase 4), generally, intraframe encoding and interframe predictive encoding are used in compression encoding of moving images. Intraframe coding divides input image data into a plurality of blocks and performs discrete cosine transform (DCT), quantization, entropy coding, etc. on all the block data to increase compression efficiency.

  Interframe predictive coding uses the property that the current frame of a moving image and the immediately preceding frame are similar to improve the compression efficiency more than intraframe coding, and the difference between the input image and the predicted image. This is a method for encoding data. When inter-frame predictive coding is used, since it is coding by prediction and compression efficiency is good, an error from the actual input image is gradually accumulated in the receiving client. In order to avoid this problem, it is a general method to perform intra-frame encoding periodically. The encoded data generated by the intra-frame encoding performed periodically is referred to as the above-described key frame.

  In order to start image display in the receiving client after connecting to the transmission server, a frame obtained by encoding all block data which are key frames, not inter-frame predictive encoded data (P frame) obtained by encoding difference data. It is necessary to receive the inner encoded data (I frame). By receiving the key frame, the transmission server can update the image display by receiving the inter-frame predictive encoded data thereafter.

  Here, the operation when a new receiving client is connected during moving image streaming control will be described. If there is a newly connected receiving client (new client) during streaming to the connected receiving client (existing client), it is determined in S51 that there is a new distribution request, and a new distribution identification setting ( A setting for connecting to a new receiving client is executed (S56).

  Then, after setting the new distribution identification, the transmission server determines whether or not the data to be transmitted is a key frame (S52). The transmission server performs either the inter-frame prediction encoding process (S53) or the intra-frame encoding process (S54), and then performs the server process (S55).

  In the server process of S55, the encoded data generated in the process of S53 or S54 is streamed to an existing client. Further, when the new distribution identification is set and the intra-frame encoded data is generated in the process of S54, streaming to the new client is started and the new distribution identification is canceled.

  FIG. 11 shows a first example of a moving image streaming frame configuration in a conventional transmission server. In the case of the processing shown in FIG. 12, for example, the moving image data shown in FIG. 11 is sent from the transmission server to the network. As shown in FIG. 11, streaming to the existing client is performed sequentially with the P frame 50y, the P frame 50z, the I frame 51, the P frame 51a, and the P frame 51b. Note that the I frame is moving image data that is intra-frame encoded, and the P frame is moving image data that is inter-frame prediction encoded.

  Here, when a new client connects at the timing of the new distribution request shown in the flowchart of FIG. 12, the P frame 51b is streamed to the existing client, and after that, the P frame 51c and the P frame 51d are continuously streamed sequentially.

  During this period, the new delivery identification described in the processing of S56 in FIG. 12 is set, but streaming to the existing client is performed in the server processing of S55, but streaming to the new client is performed. Is not started.

  Although streaming to the existing client is continuously performed, when the I frame 52 is streamed to the existing client, streaming of the I frame 52 is also started to the new client. Then, after the I frame 52, the P frame 52a and the P frame 52b are sequentially streamed to the existing client and the new client at the same timing (see page 4 of Patent Document 1 and FIG. 14).

  Next, moving image streaming control (second example) to a connected receiving client will be described as conventional image streaming. As shown in FIG. 14, the transmission server of the image streaming system first performs image input processing from a camera or the like (S50). The transmission server determines whether a new distribution request has been received from the reception client (S51).

  If a new distribution request has not been received from the receiving client (S51, NO), the transmission server determines whether or not to send a key frame (S52). If it is determined not to send a key frame (S52, NO), an inter-frame predictive encoding process is performed (S53). On the other hand, if it is determined that a key frame is to be transmitted (S52, YES), intraframe encoding processing is performed (S54).

  After the processing of S54 or S53, the moving image encoded data is subjected to server processing (S57). After streaming to the network, image input processing from the camera or the like is performed again (S50), and the above-described series of processing (S50 to S54, S57) is repeatedly executed continuously. Thereby, streaming from the transmission server to the reception client is continued. Note that the image input process shown in S50, the encoding process shown in S53 and S54, and the server process shown in S57 may be processed in parallel in the transmission server.

  Here, the operation when a new receiving client is connected during moving image streaming control will be described. If there is a newly connected new client during streaming to an existing client that is connected, it is determined that there is a new distribution request in the processing of S51, and a new distribution identification is set and the data to be transmitted is used as a key frame. Setting is forcibly (S58).

  Then, after setting the new distribution identification, the transmission server determines whether or not the data to be transmitted is a key frame (S52). Since the transmission server here has a new client and is set as a key frame in S57, the server process is performed after executing the intra-frame encoding process (S54) (S58).

  In the server process of S58, the intra-frame encoded data generated in the process of S54 is streamed to the existing client. Further, since the new distribution identification is set, streaming of the intra-frame encoded data to the new client is started and the new distribution identification is canceled.

  FIG. 13 shows a second example of a moving image streaming frame configuration in a conventional transmission server. In the case of the processing shown in FIG. 14, for example, the moving image data shown in FIG. 13 is sent from the transmission server to the network. As shown in FIG. 13, streaming is sequentially performed to an existing client with a P frame 50 y, a P frame 50 z, an I frame 51, a P frame 51 a, and a P frame 51 b.

Here, when a new client connects at the timing of the new distribution request shown in the flowchart of FIG. 14, the P frame 51b is streamed to the existing client, but thereafter, the P frame 51c and the frames after the P frame 51d are streamed. Not. And then. New key frames re-calculated when the new I frame 61, P frame 61a, and P frame 61b are sequentially streamed to the new client and the existing client, and the first I frame 61 after the new distribution request is generated. The I frame 62 is streamed at intervals (see page 11 of FIG. 1 and FIG. 1).
JP 2002-344932 A

  However, in the former conventional image streaming system (conventional technology described as the first example), when a distribution request is made from a new client, the time until the new client receives the I frame from the distribution request is the transmission server. Therefore, there is a problem that it takes a long time for a moving image to be displayed on a new client.

  In the latter conventional image streaming system (the conventional technology described as the second example), the transmission server immediately streams an I frame after detecting a new distribution request. Frames must be generated and streamed as key frames. For this reason, there is a problem that the control of the generated code amount to the existing receiving client is out of order, and the existing receiving client cannot obtain a stable image. In particular, when new connections frequently occur or when the timing of key frame transmission cannot be changed due to network restrictions, the problem that a stable image cannot be obtained by an existing receiving client becomes significant.

  The present invention has been made in view of the above, and an object of the present invention is to provide an image streaming system that smoothly streams to an existing client and a new client when a new connection is made from the new client to the transmission server. To do.

  In order to solve the above-mentioned problems, the present invention provides a transmission server that compresses and encodes moving image data and streams the compression-encoded data that has been compression-encoded via a network, and a network that transmits the compression-encoded data streamed by the transmission server to the network. And a plurality of receiving clients that display the image by decompressing and decoding the received compressed encoded data, and generating the compressed encoded data from the transmission server to the receiving client by intra-frame encoding In an image streaming system that streams inter-frame predictive encoded data generated by inter-frame predictive encoding of intra-frame encoded data and compressed encoded data, a transmission server stores intra-frame encoded data and inter-frame predictive encoded data. Coded data generation unit to generate, code A streaming unit for streaming the intra-frame encoded data and the inter-frame prediction encoded data generated by the data generation unit to the receiving client and storing the intra-frame encoded data to be streamed to the receiving client. When the data to be streamed to the connected client that is a receiving client already connected to the own server is the timing of the compression encoded data other than the intra-frame encoded data, it is a new receiving client different from the connected client. When a connection request is received from a new client, streaming to the new client is started using the stored intra-frame encoded data.

  According to the present invention, a series of streaming can be maintained without any change to a connected client connected to a transmission server, and intraframe encoded data can be streamed to a newly connected new client without delay. An image streaming system capable of displaying an image in a short time with a new client while maintaining stable streaming to a middle client is obtained.

  An image streaming system according to a first aspect of the present invention includes a transmission server that compresses and encodes moving image data and streams the compressed encoded data through a network, and compressed encoded data that is streamed by the transmission server. A plurality of receiving clients that display the image by decompressing and decoding the received compressed encoded data, and transmitting the encoded encoded data from the transmitting server to the receiving client by intra-frame encoding. In an image streaming system for streaming inter-frame predictive encoded data generated by inter-frame predictive encoding of generated intra-frame encoded data and compressed encoded data, the transmission server includes intra-frame encoded data and inter-frame predictive encoded Encoded data that generates data A streaming unit that stores the intra-frame encoded data and the inter-frame prediction encoded data generated by the encoded data generation unit to the reception client and stores the intra-frame encoded data to be streamed to the reception client; The streaming unit is different from the connected client when the data to be streamed to the connected client that is a receiving client already connected to the server is at the timing of the compressed encoded data other than the intra-frame encoded data. When a connection request is received from a new client that is a new receiving client, streaming to the new client is started using the stored intra-frame encoded data, and the connected client connected to the transmission server is started. Is a series of stories Be sustained without changing the timing any, is to a new client newly connected has an effect of being able to stream the intraframe coded data without delay.

  An image streaming system according to a second aspect is the image streaming system according to the first aspect, wherein the streaming unit starts streaming to a new client, and then the encoded data generation unit generates and streams to the connected client. The encoded data and inter-frame predictive encoded data are streamed to a new client at the same timing as streaming to the connected client, and can be easily streamed to the connected client and the new client at the same timing. Has an effect.

  The image streaming system according to a third aspect is the image streaming system according to the first aspect, wherein the encoded data generation unit stores the intra-frame encoded data after storing the intra-frame encoded data. Inter-frame predictive encoded data with the same number of frames as the inter-frame predictive encoded data streamed to the connected client before streaming to When the streaming unit starts streaming to the new client, the intra-frame encoded data stored and the same number of inter-frame prediction codes as the number of frames for which the header is set in the encoded data generation unit are stored. New data and stored intra-frame encoded data Intra-frame encoded data and inter-frame predictive encoded data are streamed to a new client in the order of inter-frame predictive encoded data that has been streamed to the connected client and then streamed to the connected client. Streaming can be easily performed at different timings.

  The image streaming system according to a fourth aspect of the present invention is the image streaming system according to the third aspect of the present invention, wherein the encoded data generation unit performs inter-frame predictive encoding for streaming to the new client next to the stored intra-frame encoded data. As the data, inter-frame predictive encoded data in which a header indicating an elapsed time since storing the intra-frame encoded data is set is generated, and the streaming unit stores the data when the streaming to the new client is started. The intra-frame encoded data and the inter-frame predictive encoded data are streamed to the new client in the order of the intra-frame encoded data and the inter-frame predictive encoded data in which the header is set in the encoded data generation unit. After storing the intra-frame encoded data An effect that can be recognized over time.

  The image streaming system according to a fifth aspect of the present invention is the image streaming system according to any of the first to fourth aspects of the present invention, wherein the encoded data generation unit sets a control parameter for displaying an image in the header and sets the encoded data in the frame. When streaming or storing the intra-frame encoded data, the streaming unit streams or stores the intra-frame encoded data together with the header set by the encoded data generation unit, and displays the image. It has the effect that the new client can acquire the control parameters when performing it.

  According to a sixth aspect of the present invention, there is provided an image streaming system that compresses and encodes moving image data and streams the compression-encoded data that has been compression-encoded via a network; And a plurality of receiving clients that display the image by decompressing and decoding the received compressed encoded data, and generating the compressed encoded data by intra-frame encoding from the transmission server to the receiving client. In an image streaming system that streams inter-frame predictive encoded data generated by intra-frame predictive encoding of intra-encoded data and compressed encoded data, the transmission server generates intra-frame encoded data and inter-frame predictive encoded data Encoded data generator When a connection request is received from a new client that is a new receiving client different from the connected client that is already connected to the server, the still image data is compressed and encoded. Stream the still image data generation unit to be generated, the intra-frame encoded data and inter-frame prediction encoded data generated by the encoded data generation unit, and the still image encoded data generated by the still image data generation unit to the receiving client. A streaming unit, and the streaming unit receives a connection request from a new client when the data to be streamed to the connected client is at the timing of the compressed encoded data other than the intra-frame encoded data. The still image encoded data generated by the After being issued, the encoded data generator generates and streams the intra-frame encoded data and the inter-frame predictive encoded data to the connected client at the same timing as the streaming to the connected client. A sequence of streaming can be maintained without any change to an existing client that is connected, and still image encoded data can be sent to a newly connected new client without delay.

  An image streaming system according to a seventh aspect is the image streaming system according to the sixth aspect, wherein the still image data generation unit generates a plurality of still image encoded data and receives the streaming when receiving a connection request from a new client. The unit is to continuously send a plurality of still image encoded data generated by the still image data generation unit to a new client, and can continue to a connected existing client without any change, This has the effect that the still picture encoded data can be continuously sent to the newly connected new client without delay.

  The image streaming system according to an eighth aspect of the present invention is the image streaming system according to the seventh aspect of the present invention, wherein the still image data generation unit is stationary until the first intra-frame encoded data is streamed to a new client that has received a connection request. When it is time to generate image encoded data, it is determined whether or not to generate still image encoded data based on the accumulated code amount of the still image encoded data sent to the new client and the predetermined pattern data. When it is determined that the encoded data is to be generated, the still image encoded data is generated and sent to the new client. The occupation of the transmission band by the still image encoded data is reduced, and the transmission band can be used efficiently. It has the action.

  An image streaming system according to a ninth aspect is the image streaming system according to any of the sixth to eighth aspects, wherein the encoded data generation unit sets a control parameter for displaying an image in a header to set intra-frame encoded data When streaming the intra-frame encoded data, the streaming unit streams the intra-frame encoded data together with the header set by the encoded data generation unit, and controls when displaying the image. It has the effect that the new client can acquire the parameters.

  The image streaming system according to a tenth aspect of the invention is the image streaming system according to the fifth or ninth aspect of the invention, wherein the header in which the encoded data generation unit sets the control parameter is a GOV header 93 or MPEG-4 specified in MPEG-4. This is a header corresponding to a GOV header defined by another video encoding method different from the above, and has a function that the receiving client can acquire information included in the GOV header or the header corresponding thereto.

  An image streaming system according to an eleventh aspect of the invention is the image streaming system according to the fifth, ninth or tenth aspect of the invention, wherein a header in which the encoded data generation unit sets a control parameter is a GOV header or MPEG specified in MPEG-4 Specified by another moving image encoding method different from MPEG-4, and a VOL header specified by MPEG-4 or specified by another moving image encoding method different from MPEG-4 This is a header corresponding to the VOL header, and the receiving client can acquire both the information included in the GOV header or the header corresponding thereto and the information included in the VOL header or the header corresponding thereto. Have.

  The image streaming system of the twelfth invention is the image streaming system of the fifth, ninth to eleventh inventions, wherein the header for setting the control parameter by the encoded data generator is a GOV header or MPEG specified in MPEG-4. A header corresponding to a GOV header defined by another moving image coding scheme different from -4, a VOL header defined by MPEG-4, or a VOL defined by another moving image coding scheme different from MPEG-4 The header corresponding to the header, the VO header specified in MPEG-4, or the header corresponding to the VO header specified in another moving image encoding method different from MPEG-4, and the receiving client receives the GOV header Or information included in a header corresponding to this and information included in a VOL header or a header corresponding thereto Has the effect that all can be obtained with the information contained in the VO header or header corresponding thereto.

  The image streaming system according to a thirteenth aspect is the image streaming system according to any of the fifth and ninth to twelfth aspects, wherein the header in which the encoded data generation unit sets the control parameter is a GOV header or MPEG defined in MPEG-4 A header corresponding to a GOV header defined by another moving image coding scheme different from -4, a VOL header defined by MPEG-4, or a VOL defined by another moving image coding scheme different from MPEG-4 A header corresponding to a header, a VO header defined in MPEG-4 or a header corresponding to a VO header defined in another moving image encoding system different from MPEG-4, and a VOS header or MPEG defined in MPEG-4 -4 is a header corresponding to a VOS header defined by another moving image encoding method different from -4, The trusted client includes information contained in the GOV header or the header corresponding thereto, information contained in the VOL header or the header corresponding thereto, information contained in the VO header or the header corresponding thereto, and the VOS header or It has the effect that all the information contained in the corresponding header can be acquired.

  An image streaming system according to a fourteenth aspect is the image streaming system according to any one of the first to thirteenth aspects, wherein the encoded data generation unit is composed of a plurality of units, and when there is a connection request from a new client, the connection request from the new client If there is an encoded data generator that is generating intra-frame encoded data and inter-frame predictive encoded data in the connected client at the same resolution as the moving image resolution, the encoded data generator has requested connection Generates encoded data with the same resolution even when all encoded data generation units are generating encoded data, by causing a new client to generate intra-frame encoded data and inter-frame predictive encoded data. It has the effect | action that it can stream to a new client in the inside encoded data production | generation part.

  Embodiments of the present invention will be described below.

(Embodiment 1)
FIG. 1 is a diagram showing a schematic configuration of an image streaming system according to Embodiment 1 of the present invention. In the image streaming system, 2 is a transmission server, 1 is a communication network such as the Internet, and 3 is a reception client. The transmission server 2 and the reception client 3 are connected via the network 1. The reception client 3 connects to the transmission server 2 as necessary, and acquires image data from the transmission server 2. That is, the image data transmitted from the transmission server 2 is streamed to the reception client 3 via the network 1.

  The transmission server 2 includes an image (moving image) input function such as a camera, an image encoding function, an image transmission function, and the like. The reception client 3 is, for example, a personal computer or a mobile phone, and has an image reception function, an image decoding function, an image output function such as a display monitor, and the like.

  Hereinafter, for convenience of explanation, the receiving client 3 connected to the transmitting server 2 may be referred to as an existing client (connected client), and the receiving client 3 newly connected to the transmitting server 2 and 3 may be referred to as a new client.

  FIG. 2 is a block diagram showing the configuration of the image streaming system according to Embodiment 1 of the present invention. In the transmission server 2 of FIG. 2, 21 is an image input unit, 22 is a moving image encoder, 23 is an application processing unit, and 24 is a server control unit. In the receiving client 3 of FIG. 2, 31 is an image output unit, 32 is a moving image decoder, 33 is an application processing unit, and 34 is a client control unit.

  The image input unit 21 captures an image from a built-in or externally connected CCD (Charge Coupled Devices) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor or the like, and adjusts the resolution to an appropriate size as necessary. Convert. The image input unit 21 inputs the captured image (moving image data) to the moving image encoder 22.

  The moving image encoder (encoded data generation unit) 22 performs moving image compression encoding on the image data from the image input unit 21 in accordance with the definition of MPEG-4 or the like, and inputs it to the application processing unit 23. The application processing unit 23 controls all the internal controls of the transmission server 2 and executes a new distribution request from the receiving client 3, control for inserting a key frame, and the like.

  The server control unit (streaming unit) 24 performs protocol control such as connection control and disconnection control for the connection request and disconnection request from the receiving client 3, and start and stop of streaming, and is generated by the video encoder 22. The encoded data is streamed to the receiving client 3 via the network 1.

  The client control unit 34 performs protocol control such as connection request or disconnection request to the transmission server 2, start or stop of streaming, and reception control of encoded data streamed from the transmission server 2 via the network 1. . The application processing unit 33 controls all internal controls of the reception client 3 and executes user interface control.

  The moving picture decoder 32 decompresses and decodes moving picture encoded data such as MPEG-4 received from the transmission server 2. The image output unit 31 converts the image data generated by the moving image decoder 32 to an appropriate resolution as necessary, and outputs it to a display monitor such as an LCD (Liquid Crystal Monitor) or a display.

  Next, a processing procedure for streaming control in the transmission server 2 will be described.

  FIG. 3 is a flowchart showing a processing procedure of streaming control in the transmission server. First, moving picture streaming control to the receiving client 3 connected to the transmitting server 2 will be described.

  The image input unit 21 of the transmission server 2 performs image input processing from a camera or the like (S10). The application processing unit 23 determines whether a new distribution request has been received from the receiving client 3 (S11).

  When a new distribution request has not been received from the receiving client 3 (S11, NO), the application processing unit 23 determines whether or not to send a key frame (whether or not it is time to stream the key frame) (S12). ).

  When it is determined that the key frame is not transmitted (S12, NO), the moving picture encoder 22 performs inter-frame predictive coding (inter-coding or motion-compensated predictive coding) (S13). In the first embodiment, after the moving image encoder 22 performs the inter-frame predictive encoding process, the process after detecting a new distribution request (change of time information in a header described later) (the process of S14) is performed. Not performed. On the other hand, when it is determined that a key frame is to be transmitted (S12, YES), the moving image encoder 22 performs an intra-frame encoding (intra encoding) process (S15).

  When the moving image encoder 22 performs the intra-frame encoding process, the intra-frame encoded data generated by the moving image encoder 22 is stored (S16). The intra-frame encoded data generated by the moving image encoder 22 is stored by a predetermined storage unit provided in the transmission server 2 or the like. The intraframe encoded data generated by the moving image encoder 22 may be stored in the application processing unit 23 or the server control unit 24.

  The moving image encoded data (intra-frame encoded data and inter-frame predictive encoded data) generated by the moving image encoder 22 is subjected to server processing (moving image data via a network) under the control of server processing by the server control unit 24. Streamed to the receiving client) (S17).

  In the transmission server 2, after streaming on the network 1, image input processing from the camera or the like is performed again (S10), and the series of processing (S10 to S17) described above is repeatedly executed. Thereby, streaming from the transmission server 2 to the reception client 3 is continued. Note that the image input process shown in S10, the encoding process shown in S13 and S15, and the server process shown in S17 may be processed in parallel in the transmission server 2.

  Here, the operation when a new receiving client is connected during moving image streaming control will be described. If there is a newly connected reception client (new client) 3 during streaming to the connected reception client (existing client) 3, it is determined that there is a new distribution request in the processing of S11, and the application processing unit 23 Performs connection setting with the newly connected receiving client 3 and processing when a new distribution request is detected (S18).

  First, the application processing unit 23 determines whether or not the data to be transmitted is a key frame. The application processing unit 23 does nothing when the encoded data to be streamed next to the existing client is a key frame (when the key frame is sent).

  On the other hand, when the encoded data to be streamed next to the existing client is not a key frame (when the compressed encoded data other than the intra-frame encoded data is transmitted), the key frame saved in the process of S16 ( Server processing is performed on the new client under the control of the server control unit 24 (intra-frame encoded data) (flow indicated by a broken line in FIG. 3). Thereby, streaming to the new client is started.

  Thereafter, the transmission server 2 proceeds to the process of S12. That is, the same moving image data is streamed to the new client and the existing client at the same timing. The subsequent processing is the same as the moving image streaming control to the connected receiving client 3 described above, but when the application processing unit 23 determines that the data to be transmitted in the processing of S12 is a key frame (for example, the last Intra-frame encoded data that is encoded to a new client in S15 for a new distribution request between a P frame (P frame 11z, which will be described later) and the next I frame (I frame 12, which will be described later). Streaming. For example, after the stored n-th (n is a natural number) I-frame is transmitted to the new client, the (n + 1) -th I-frame encoded in S15 is immediately transmitted to the new client.

  In the reception client 3 (existing client or new client), the client control unit 34 performs reception control of the encoded data streamed from the transmission server 2 via the network 1 and receives the encoded data. The application processing unit 33 performs user interface control, and sends encoded data (moving image encoded data) to the moving image decoder 32.

  The moving picture decoder 32 decompresses and decodes moving picture encoded data such as MPEG-4 received from the transmission server 2. The image output unit 31 converts the image data generated by the moving image decoder 32 to an appropriate resolution as necessary, and outputs the image data to a display monitor (displays an image).

  FIG. 4 is a diagram illustrating a frame configuration of moving image streaming. As shown in FIG. 4, streaming is sequentially performed to an existing client with a P frame 10y, a P frame 10z, an I frame 11, a P frame 11a, and a P frame 11b.

  Here, when the new client connects to the transmission server 2 while streaming the P frame 11b to the existing client (generation of a new distribution request) (S11 in FIG. 3, YES), the P frame 11c, Streaming is continued sequentially with the P frame 11d. The I frame 11 that is the stored key frame is streamed to the new client, and thereafter, the P frame 11c, the P frame 11d, and the P frame 11e are sequentially streamed to the existing client and the new client at the same timing.

  As described above, according to the first embodiment, a series of streaming can be maintained without any change to an existing client that is connected, and a key frame can be streamed without delay to a newly connected new client. The new client can display images in a short time while maintaining stable streaming.

(Embodiment 2)
Next, a second embodiment of the present invention will be described with reference to FIGS. 1 to 3, 5, and 6. In the second embodiment, when server processing a stored key frame to a new client, data corresponding to the number of inter-frame predictive encoded data streamed to an existing client after the key frame is generated and transmitted to the new client. . Note that the image streaming system of the second embodiment has the same configuration as the image streaming system of FIGS. 1 and 2 described in the first embodiment, and thus the description thereof is omitted.

  First, a processing procedure of streaming control in the transmission server 2 will be described. Here, the processing procedure of the streaming control in the transmission server 2 will be described using the flowchart of FIG. Note that in the image streaming system of the second embodiment, the moving image streaming control to the receiving client 3 being connected is the same as the processing described in the first embodiment, and a description thereof will be omitted. Here, as an operation different from the image streaming system of the first embodiment, an operation when a new receiving client is connected during moving image streaming control will be described.

  If there is a newly connected reception client (new client) 3 during streaming to the connected reception client (existing client) 3, it is determined that there is a new distribution request in the processing of S11, and the application processing unit 23 Performs connection setting with the newly connected receiving client 3 and processing when a new distribution request is detected (S18).

  First, the application processing unit 23 determines whether or not the data to be transmitted is a key frame. The application processing unit 23 does nothing when the encoded data to be next streamed to the existing client is a key frame.

  On the other hand, if the encoded data to be streamed next to the existing client is not a key frame, the server frame is stored in the process of S16 and the server process to the new client is performed by the server control unit 24. (The flow shown by the broken line in FIG. 3). Further, data corresponding to the number of inter-frame predictive encoded data streamed to the existing client after the stored key frame (from when the existing client stores the key frame to when the key frame is transmitted to the new client) (to be described later) Empty inter-frame predictive encoded data) (the same number of data as the number of frames that have been streamed) is generated and server-processed to a new client under the control of the server control unit 24.

  Here, the internal configuration of the intra-frame encoded data and the inter-frame predictive encoded data will be described with reference to FIG.

  FIG. 5 is a diagram for describing an internal configuration of intra-frame encoded data and inter-frame predictive encoded data. FIG. 5 shows the internal structure of the moving image frame 9. In the moving image frame 9 shown in FIG. 5, 90 is a VOS (Visual Object Sequence) header, 91 is a VO (Visual Object Layer) header, 92 is a VOL (Video Object Layer) header, and 93 is a GOV (Group). Of Video object Plane) header, 94 is a VOP header, and 95 is encoded data.

  In the moving image frame 9, intra-frame encoded data and inter-frame predictive encoded data are composed of a VOP (Video Object Plane) header 94 and encoded data 95.

  In the VOP header 94 of the moving image frame 9, 94a is vop_start_code, 94b is vop_coding_type, 94c is modulo_time_base, 94d is vop_time_increment, 94e is vop_coded_flag, and 95f is a plurality of other header elements.

  For example, intra-frame encoded data and inter-frame predictive encoded data are identified by vop_coding_type 94b. Modulo_time_base94c and vop_time_increment 94d are time information used in pairs and are updated according to a reference frequency, and are used for display timing control on the moving image frame receiving side. The vop_coded_flag 94e is information for identifying whether or not data continues thereafter. Here, the internal structure defined by MPEG-4 is shown as an example of the structure of the moving image frame 9, but similar structures are also used in other moving image encoding systems, and other moving images are used. An image coding method may be used.

  The above-described empty inter-frame predictive encoded data sets time information (modulo_time_base94c and vop_time_increment94d) according to inter-frame predictive encoded data (number of frames) streamed to an existing client after the stored key frame, This is inter-frame predictive encoded data in which vop_coded_flag 94e indicating that data is not continuous thereafter is set.

  After the process of S18 (streaming of the intra-frame encoded data stored in the new client and the empty inter-frame predictive encoded data described later), the transmission server 2 proceeds to the process of S12. The subsequent processing is the same as the moving image streaming control to the connected receiving client described in the first embodiment.

  In the second embodiment, as in the first embodiment, when the application processing unit 23 determines that the data to be transmitted in the process of S12 is a key frame, the new client immediately includes the frame encoded in S15. Stream encoded data.

  Also, in the second embodiment, as in the first embodiment, after the moving image encoder 22 performs the inter-frame predictive encoding process, the process after the detection of a new distribution request (such as changing the time information of the header described later) ) (Processing of S14) is not performed.

  Further, the reception client 3 (existing client or new client) receives the encoded data 95 streamed from the transmission server 2 and displays an image, as in the processing in the first embodiment.

  FIG. 6 is a diagram illustrating a frame configuration of moving image streaming. As shown in FIG. 6, streaming to the existing client is sequentially performed with the P frame 10 y, the P frame 10 z, the I frame 11, the P frame 11 a, and the P frame 11 b.

  Here, when the new client connects to the transmission server 2 while streaming the P frame 11b to the existing client (generation of a new distribution request) (S11 in FIG. 3, YES), the P frame 11c, Streaming is continued sequentially with the P frame 11d.

  To the new client, the I frame 11 that is the stored key frame is streamed, and then the P frame 11a1 and the P frame 11b1 that are empty intraframe prediction encoded data are streamed. Thereafter, the P frame 11c, the P frame 11d, and the P frame 11e are sequentially streamed to the existing client and the new client.

  As described above, according to the second embodiment, a series of streaming can be maintained without any change to an existing client that is connected, and a key frame and an empty P frame can be streamed without delay to a newly connected new client. Therefore, it is possible to display a smoother image than in the first embodiment on the new client side in a short time while maintaining stable streaming to the existing client.

(Embodiment 3)
Next, a third embodiment of the present invention will be described with reference to FIGS. In the third embodiment, the application processing unit 23 targets only the inter-frame predictive encoded data to be streamed to the new client, and calculates the number of inter-frame predictive encoded data streamed to the existing client after the stored key frame. Time information after the saved key frame is calculated. Then, inter-frame prediction encoded data in which the time information of the header is changed to the calculated time information is generated and transmitted to the reception client 3. Since the image streaming system of the third embodiment has the same configuration as the image streaming system of FIGS. 1 and 2 described in the first embodiment, the description thereof is omitted.

  Next, a processing procedure for streaming control in the transmission server 2 will be described. Here, the processing procedure of the streaming control in the transmission server 2 will be described using the flowchart of FIG. Note that, in the image streaming system of the third embodiment, the moving image streaming control to the receiving client 3 being connected is the same as the processing described in the first embodiment, and thus the description thereof is omitted.

  Here, as a different operation from the image streaming system of the first and second embodiments, an operation when a new receiving client is connected during moving image streaming control will be described. If there is a newly connected reception client (new client) 3 during streaming to the connected reception client (existing client) 3, it is determined that there is a new distribution request in the processing of S11, and the application processing unit 23 Performs connection setting with the newly connected receiving client 3 and processing when a new distribution request is detected (S18).

  First, the application processing unit 23 determines whether or not the data to be transmitted is a key frame. The application processing unit 23 does nothing when the encoded data 95 to be next streamed to the existing client is a key frame.

  On the other hand, if the encoded data 95 to be streamed next to the existing client is not a key frame, the application processing unit 23 performs setting of new distribution identification (setting of a flag for determining whether or not a new client has been connected). The key frame (intra-frame encoded data) saved in the process of S16 is server-processed to a new client under the control of the server control unit 24 (flow indicated by a broken line in FIG. 3) (S18).

  After the process of S18 (streaming of the intra-frame encoded data stored in the new client), the transmission server 2 proceeds to the process of S12. The subsequent processing is the same as the moving image streaming control to the connected receiving client described in the first embodiment.

  In the third embodiment, as in the first and second embodiments, if the application processing unit 23 determines that the data to be transmitted in the process of S12 is a key frame, the frame to be encoded to the new client in S15. Stream the encoded data.

  In the third embodiment, when the new distribution identification is set, the process after the detection of the new distribution request is performed after the process of S13. Specifically, the application processing unit 23 targets only the inter-frame predictive encoded data to be streamed to the new client, after the key frame saved in the process of S16 (after the existing client saved the key frame). The time information after the stored key frame is calculated from the number of inter-frame predictive encoded data streamed to the existing client (until the key frame is transmitted to the new client). Then, the calculated time information is set in modulo_time_base 94c and vop_time_increment 94d, inter-frame prediction encoded data in which the time information of the header is changed is generated, and new distribution identification is canceled (S14). In other words, the inter-frame predictive encoded data in which the time information in the header of the moving image frame is changed so that the next inter-frame predictive encoded data indicates the elapsed time from the time when the stored intra-frame encoded data is generated. Stream.

  Thereafter, the server process is performed on the interframe prediction encoded data whose time information has been changed to a new client under the control of the server control unit 24 (S18). Thereby, the inter-frame prediction encoded data whose time information is changed is streamed only to the new client.

  The receiving client 3 (existing client or new client) receives the encoded data streamed from the transmission server 2 (interframe predictive encoded data whose time information has been changed) as in the processing in the first embodiment. To display an image.

  The frame configuration of moving image streaming according to the third embodiment has the same configuration as the frame configuration of FIG. 4 described in the first embodiment. The difference between the frame of the first embodiment and the frame of the third embodiment is that the time information in the header of the P frame 11c for the existing client and the time information in the header of the P frame 11c for the new client are the same as those in the third embodiment. So it is different. The time information in the header of the P frame 11c streamed to the new client indicates information related to the elapsed time from the generation time of the key frame (intra-frame encoded data) saved in the process of S16.

  In this way, the inter-frame predictive encoded data that is streamed to the existing client after the stored key frame by the application processing unit 23 of the transmission server 2 only for the inter-frame predictive encoded data to be streamed to the new client. Time information after the saved key frame is calculated from the number, and inter-frame prediction encoded data in which the time information of the header is changed based on the calculated time information is generated. For this reason, the receiving client 3 can display a smooth image using the time information in the header.

  As described above, according to the third embodiment, a series of streaming can be maintained without any change to an existing client that is connected, and a key frame and a frame in which time information is changed without delay to a newly connected new client. Since the inter prediction encoded data can be streamed, it is possible to display a smoother image than the first embodiment in a short time on the new client side while maintaining stable streaming to the existing client.

  Further, even when the time until the transmission of the next I frame is long after a new client connects, it is not necessary to transmit an empty P frame, so that a smoother image is easier than in the second embodiment. Can be displayed.

(Embodiment 4)
Next, a fourth embodiment of the present invention will be described with reference to FIGS. In the fourth embodiment, when the GOV header 93 defined in MPEG-4 is generated when the moving image encoder 22 performs the intra-frame encoding process, the GOV header 93 together with the intra-frame encoded data is generated. Is transmitted to the receiving client 3. Since the image streaming system of the fourth embodiment has the same configuration as the image streaming system of FIGS. 1 and 2 described in the first embodiment, the description thereof is omitted.

  Next, a processing procedure for streaming control in the transmission server 2 will be described. Here, similarly to Embodiments 1 to 3, the processing procedure of streaming control in the transmission server 2 will be described using the flowchart of FIG. Note that, in the image streaming system of the fourth embodiment, the moving image streaming control to the receiving client 3 being connected is the same as the processing described in the first to third embodiments, and thus the description thereof is omitted.

  Here, the processing operation (key frame storage) of S16 in FIG. 3 will be described as an operation different from the image streaming system of the first to third embodiments. In the fourth embodiment, when the moving image encoder 22 performs the intra-frame encoding process (S15), when the GOV header 93 defined in MPEG-4 is generated, the moving image encoder 22 The GOV header 93 is stored together with the generated intraframe encoded data (S16).

  For example, when MPEG-4 is used, the GOV header 93 to be saved is the GOV header 93 shown in FIG. 5, and the key frame to be saved is composed of the VOP header 94 and the encoded data 95 shown in FIG. Here, when it is determined that there is a new distribution request in the process of S11 of FIG. 3, the intra-frame encoded data is server-processed together with the GOV header 93 stored in the process of S16 (S17).

  Specifically, if the encoded data 95 to be streamed next is not a key frame, the intra-frame encoded data that is the key frame (intra-frame encoded data having the GOV header 93) saved in the process of S16 is stored in the server. Process. On the other hand, if the encoded data to be streamed next is a key frame, the intra-frame encoded data that has been intra-frame encoded in the process of S15 is subjected to server processing. At this time, the GOV header 93 is also server-processed together with the intra-frame encoded data generated by the moving image encoder 22. As a result, the intra-frame encoded data together with the saved GOV header 93 is streamed to the new client.

  In the fourth embodiment, the GOV header 93 defined in MPEG-4 is stored together with the intraframe encoded data generated by the moving image encoder 22 and is streamed to a new client. The header that is stored together with the data and streamed to the new client is not limited to the GOV header 93 defined in MPEG-4. That is, together with the intra-frame encoded data generated by the video encoder 22, a header (a header equivalent to the GOV header 93) defined by another video encoding method (H.263, H264, etc.) is stored. You may stream to a new client.

  For example, headers such as the VOS header 90, the VO header 91, the VOL header 92, and the GOV header 93 shown in FIG. 5 (layer-specific headers defined in MPEG-4) are stored together with the intra-frame encoded data and sent to the new client. You may stream.

  The VOS header 90 here stores a start code, profile information, level information, and the like. The VO header 91 stores a start code, information for identifying a visual object, priority information for a visual object, various types of information for each type of visual object, and the like. The VOL header 92 stores a start code, information for identifying a short video header, information for identifying a visual object layer, image aspect information, a VOL control parameter group, and the like. The GOV header 93 stores a start code, a time code, and the like.

  As described above, according to the fourth embodiment, since the new client can acquire the information included in the GOV header 93 or a header equivalent thereto, the receiving client does not need to operate with a predetermined control parameter. It is possible to provide an image streaming system having a higher flexibility than the image streaming systems according to the first to third embodiments.

(Embodiment 5)
Next, a fifth embodiment of the present invention will be described with reference to FIGS. In the fifth embodiment, when the VOL header 92 and the GOV header 93 defined in MPEG-4 are generated when the moving image encoder 22 performs the intra-frame encoding process, the intra-frame encoded data is generated. At the same time, the VOL header 92 and the GOV header 93 are stored and transmitted to the receiving client 3. Since the image streaming system of the fifth embodiment has the same configuration as the image streaming system of FIGS. 1 and 2 described in the first embodiment, the description thereof is omitted.

  Next, a processing procedure for streaming control in the transmission server 2 will be described. Here, similarly to Embodiments 1 to 3, the processing procedure of streaming control in the transmission server 2 will be described using the flowchart of FIG. Note that, in the image streaming system of the fifth embodiment, the moving image streaming control to the receiving client 3 being connected is the same as the processing described in the first to third embodiments, and thus the description thereof is omitted.

  Here, the processing operation (key frame storage) of S16 in FIG. 3 will be described as an operation different from the image streaming system of the first to third embodiments. In the fifth embodiment, when the video encoder 22 performs the intra-frame encoding process (S15), if the VOL header 92 and the GOV header 93 defined in MPEG-4 are generated, the video code The VOL header 92 and the GOV header 93 are stored together with the intra-frame encoded data generated by the encoder 22 (S16).

  For example, when MPEG-4 is used, the VOL header to be stored is the VOL header 92 shown in FIG. 5, the GOV header 93 to be stored is the GOV header 93 shown in FIG. 5, and the key frame to be stored is shown in FIG. A VOP header 94 and encoded data 95 shown in FIG.

  Here, when it is determined that there is a new distribution request in the process of S11 of FIG. 3, the intra-frame encoded data is server-processed together with the VOL header 92 and the GOV header 93 stored in the process of S16 (S17).

  Specifically, if the encoded data 95 to be streamed next is not a key frame, the intra-frame encoded data that is the key frame saved in the process of S16 is subjected to server processing. On the other hand, if the encoded data 95 to be streamed next is a key frame, the intra-frame encoded data that has been intra-frame encoded in the process of S15 is processed by the server. As a result, the intra-frame encoded data is streamed to the new client together with the plurality of saved headers (VOL header 92 and GOV header 93).

  In the fifth embodiment, the VOL header 92 and the GOV header 93 defined in MPEG-4 are stored together with the intra-frame encoded data generated by the moving image encoder 22 and streamed to a new client. The header that is stored together with the intra-frame encoded data and is streamed to the new client is not limited to the VOL header 92 and GOV header 93 defined in MPEG-4. That is, together with the intra-frame encoded data generated by the video encoder 22, a plurality of headers defined by other video encoding methods (a header equivalent to the VOL header 92 and a header equivalent to the GOV header 93) are displayed. You may save and stream to a new client.

  As described above, according to the fifth embodiment, the new client is included in the information included in a plurality of headers (information included in the VOL header 92 or a header equivalent thereto, and the GOV header 93 or a header equivalent thereto). Information) can be acquired, and the receiving client does not need to operate with a predetermined control parameter, and can provide an image streaming system having a higher flexibility than the image streaming systems of the first to fourth embodiments. .

(Embodiment 6)
Next, a sixth embodiment of the present invention will be described with reference to FIGS. 1 to 3 and FIG. In the sixth embodiment, when the moving image encoder 22 performs the intra-frame encoding process, if the VO header 91, the VOL header 92, the GOV header 93, etc. defined in MPEG-4 are generated, the frame The VO header 91, the VOL header 92, and the GOV header 93 are stored together with the inner encoded data, and transmitted to the receiving client 3. Since the image streaming system of the fifth embodiment has the same configuration as the image streaming system of FIGS. 1 and 2 described in the first embodiment, the description thereof is omitted.

  Next, a processing procedure for streaming control in the transmission server 2 will be described. Here, similarly to Embodiments 1 to 3, the processing procedure of streaming control in the transmission server 2 will be described using the flowchart of FIG. Note that, in the image streaming system of the fifth embodiment, the moving image streaming control to the receiving client 3 being connected is the same as the processing described in the first to third embodiments, and thus the description thereof is omitted.

  Here, the processing operation (key frame storage) of S16 in FIG. 3 will be described as an operation different from the image streaming system of the first to third embodiments. In the sixth embodiment, when the moving picture encoder 22 performs the intra-frame encoding process (S15), when the VO header 91, the VOL header 92, and the GOV header 93 defined in MPEG-4 are generated, The VOL header 92 and the GOV header 93 are stored together with the intra-frame encoded data generated by the moving picture encoder 22 (S16).

  For example, when MPEG-4 is used, the VO header 91 to be stored is the VO header 91 shown in FIG. 5, the VOL header is the VOL header 92 shown in FIG. 5, and the GOV header to be stored is the GOV shown in FIG. The key frame to be saved is a header 93 and includes a VOP header 94 and encoded data 95 shown in FIG.

  Here, when it is determined that there is a new distribution request in the process of S11 of FIG. 3, the intra-frame encoded data is server-processed together with the VO header 91, the VOL header 92, and the GOV header 93 stored in the process of S16 (S17). ).

  Specifically, if the encoded data 95 to be streamed next is not a key frame, the intra-frame encoded data that is the key frame saved in the process of S16 is subjected to server processing. On the other hand, if the encoded data 95 to be streamed next is a key frame, the intra-frame encoded data that has been intra-frame encoded in the process of S15 is processed by the server. As a result, the intra-frame encoded data is streamed to the new client together with the plurality of saved headers (VO header 91, VOL header 92 and GOV header 93).

  In the sixth embodiment, the VO header 91, the VOL header 92, and the GOV header 93 defined in MPEG-4 are stored together with the intra-frame encoded data generated by the moving image encoder 22, and are streamed to a new client. However, the headers stored together with the intra-frame encoded data and streamed to the new client are not limited to the VO header 91, the VOL header 92, and the GOV header 93 defined in MPEG-4. That is, together with the intra-frame encoded data generated by the video encoder 22, a plurality of headers defined by other video encoding methods (a header equivalent to the VO header 91, a header equivalent to the VOL header 92, GOV) Header equivalent to the header 93) may be stored and streamed to a new client.

  As described above, according to the sixth embodiment, the new client can include information included in a plurality of headers (information included in the VO header 91 or a header equivalent thereto, information included in the VOL header 92 or a header equivalent thereto). , The information contained in the GOV header 93 or an equivalent header thereof) can be acquired, so the receiving client does not need to operate with a predetermined control parameter, and is more flexible than the image streaming systems of the first to fifth embodiments. High image streaming system can be provided.

(Embodiment 7)
Next, a seventh embodiment of the present invention will be described with reference to FIGS. 1 to 3 and FIG. In the seventh embodiment, when the moving image encoder 22 performs the intra-frame encoding process, the VOS header 90, the VO header 91, the VOL header 92, the GOV header 93, and the like defined in MPEG-4 are generated. In this case, the VOS header 90, the VO header 91, the VOL header 92, and the GOV header 93 are stored together with the intra-frame encoded data, and transmitted to the receiving client 3. Since the image streaming system of the seventh embodiment has the same configuration as the image streaming system of FIGS. 1 and 2 described in the first embodiment, the description thereof is omitted.

  Next, a processing procedure for streaming control in the transmission server 2 will be described. Here, similarly to Embodiments 1 to 3, the processing procedure of streaming control in the transmission server 2 will be described using the flowchart of FIG. Note that, in the image streaming system of the fifth embodiment, the moving image streaming control to the receiving client 3 being connected is the same as the processing described in the first to third embodiments, and thus the description thereof is omitted.

  Here, the processing operation (key frame storage) of S16 in FIG. 3 will be described as an operation different from the image streaming system of the first to third embodiments. In the seventh embodiment, when the moving image encoder 22 performs the intra-frame encoding process (S15), the VOS header 90, the VO header 91, the VOL header 92, and the GOV header 93 defined in MPEG-4 are generated. If so, the VOS header 90, the VO header 91, the VOL header 92, and the GOV header 93 are stored together with the intra-frame encoded data generated by the moving image encoder 22 (S16).

  For example, when MPEG-4 is used, the VOS header 90 to be saved is the VOS header 90 shown in FIG. 5, the VO header to be saved is the VO header 91 shown in FIG. 5, and the VOL header is the VOL shown in FIG. The GOV header 93 to be stored is the GOV header 93 shown in FIG. 5, and the key frame to be stored includes the VOP header 94 and the encoded data 95 shown in FIG.

  Here, when it is determined that there is a new distribution request in the process of S11 in FIG. 3, the intra-frame encoded data is stored in the server together with the VOS header 90, the VO header 91, the VOL header 92, and the GOV header 93 stored in the process of S16. Process (S17).

  Specifically, if the encoded data 95 to be streamed next is not a key frame, the intra-frame encoded data that is the key frame saved in the process of S16 is subjected to server processing. On the other hand, if the encoded data 95 to be streamed next is a key frame, the intra-frame encoded data that has been intra-frame encoded in the process of S15 is processed by the server. As a result, the intra-frame encoded data is streamed to the new client together with the plurality of saved headers (VOS header 90, VO header 91, VOL header 92, and GOV header 93).

  In the seventh embodiment, the VOS header 90, the VO header 91, the VOL header 92, and the GOV header 93 defined in MPEG-4 are stored together with the intra-frame encoded data generated by the moving image encoder 22, and a new Although streaming to the client is performed, the header to be stored together with the intra-frame encoded data and streamed to the new client is not limited to the VOS header 90, VO header 91, VOL header 92, and GOV header 93 defined in MPEG-4. That is, together with the intra-frame encoded data generated by the moving image encoder 22, a header defined by another moving image encoding method (a header equivalent to the VOS header 90, a header equivalent to the VO header 91, and a VOL header 92 May be stored and streamed to a new client.

  As described above, according to the seventh embodiment, a new client can receive information included in a plurality of headers (information included in a VOS header 90 or a header equivalent thereto, information included in a VO header 91 or a header equivalent thereto). , Information contained in the VOL header 92 or an equivalent header thereof, information contained in the GOV header 93 or an equivalent header thereof) can be obtained, and the receiving client does not need to operate with predetermined control parameters, It is possible to provide an image streaming system that is more flexible than the image streaming systems of the first to sixth embodiments.

(Embodiment 8)
Next, an eighth embodiment of the present invention will be described with reference to FIGS. In the eighth embodiment, when a new client connects to the transmission server 2 during streaming to the existing client connected to the transmission server 2, still image encoded data is generated and transmitted to the new client, and then to the existing client. From the time of transmitting the next key frame (I frame), the same moving image data as that of the existing client is transmitted to the new client.

  FIG. 7 is a block diagram showing a configuration of an image streaming system according to Embodiment 8 of the present invention. Among the constituent elements in FIG. 7, constituent elements that achieve the same functions as those in the image streaming system of the first embodiment shown in FIG. 2 are assigned the same numbers, and redundant descriptions are omitted.

  The transmission server 2 of the image streaming system according to the eighth embodiment includes a still image encoder 25 in addition to the image input unit 21, the moving image encoder 22, the application processing unit 23, and the server control unit 24. . The image input unit 21 is connected to the moving image encoder 22 and the still image encoder 25, and the moving image encoder 22 and the still image encoder 25 are connected to the application processing unit 23.

  The still image encoder (still image data generation unit) 25 performs still image compression encoding on the image data from the image input unit 21 in accordance with the definition of JPEG (Joint Photographic Experts Group) and the like, and inputs it to the application processing unit 23 To do.

  The server control unit 24 here performs connection control and disconnection control for connection requests and disconnection requests from the receiving client 3, protocol control such as start and stop of streaming, and the moving image encoder 22 and still image. The encoded data generated by the encoder 25 is streamed to the receiving client 3 via the network 1.

  In addition, the receiving client 3 of the image streaming system according to the eighth embodiment includes a still image decoder 35 in addition to the image output unit 31, the moving image decoder 32, the application processing unit 33, and the client control unit 34. ing. The image output unit 31 is connected to the moving image decoder 32 and the still image decoder 35, and the moving image decoder 32 and the still image decoder 35 are connected to the application processing unit 33.

  The still image decoder 35 decompresses and decodes still image encoded data such as JPEG-4 received from the transmission server 2. The image output unit 31 converts the image data generated by the moving image decoder 32 and the image data generated by the still image decoder 35 to an appropriate resolution as necessary, and displays them on an LCD or a display. Output to a monitor (not shown).

  Next, a processing procedure for streaming control in the transmission server 2 will be described. Here, similarly to Embodiments 1 to 3, the processing procedure of streaming control in the transmission server 2 will be described using the flowchart of FIG. Note that, in the image streaming system of the eighth embodiment, the moving image streaming control to the receiving client 3 being connected is the same as the processing described in the first to third embodiments, and thus the description thereof is omitted.

  Here, as a different operation from the image streaming systems of the first to third embodiments, an operation when a new receiving client is connected during moving image streaming control will be described.

  If there is a newly connected reception client (new client) 3 during streaming to the connected reception client (existing client) 3, it is determined that there is a new distribution request in the processing of S11, and the application processing unit 23 Performs connection setting with the newly connected receiving client 3 and processing when a new distribution request is detected (S18).

  First, the application processing unit 23 determines whether or not the data to be transmitted is a key frame. The application processing unit 23 does nothing when the encoded data to be next streamed to the existing client is a key frame.

  On the other hand, if the encoded data to be streamed next to the existing client is not a key frame, still image encoded data is generated by the still image encoder 25 and server processing is performed on the new client by the control of the server control unit 24 (see FIG. 3 is a flow indicated by a broken line). As a result, the encoded still image data is sent to the new client and streaming is started.

  Thereafter, the transmission server 2 proceeds to the process of S12. The subsequent processing is the same as the moving image streaming control to the connected receiving client 3 described in the first embodiment, but the application processing unit 23 determines that the data transmitted in the processing of S12 is a key frame. In this case, the intra-frame encoded data encoded in S15 is also streamed to the new client.

  After the application processing unit 23 sets the new distribution identification, when the application processing unit 23 detects that the new distribution identification is set (S14), the next encoding process is performed within the frame for creating the key frame. Only in the case of encoding, the new distribution identification is canceled, and thereafter the same moving image encoded data as that of the existing client is streamed to the new client.

  The reception client 3 (existing client or new client) receives the encoded data streamed from the transmission server 2 and displays the image, as in the processing in the first embodiment. In the eighth embodiment, after the moving image encoder 22 performs the intra-frame encoding process, the intra-frame encoded data generated by the moving image encoder 22 is not stored (S16).

  FIG. 8 is a diagram illustrating a frame configuration of image streaming according to the eighth embodiment. As shown in FIG. 8, the existing client is sequentially streamed with the P frame 10y, the P frame 10z, the I frame 11, the P frame 11a, and the P frame 11b.

  Here, when the new client connects to the transmission server 2 while streaming the P frame 11b to the existing client (S11 in FIG. 3, YES), the existing client is then sequentially streamed with the P frame 11c and the P frame 11d. to continue.

  The still image frame 20 (still image encoded data) generated by the still image encoder 25 is transmitted to the new client, and thereafter the same as the existing client from the I frame 12 that is the next key frame to the existing client. Streaming is performed sequentially with the P frame 12a and the P frame 12b at the timing.

  As described above, according to the eighth embodiment, a series of streaming can be maintained without any change to an existing client that is connected, and still image encoded data can be sent to a newly connected new client without delay. While maintaining stable streaming to the existing client, the new client can display a still image in a short time and then display a moving image without waiting for the start of moving image display.

(Embodiment 9)
Next, a ninth embodiment of the present invention will be described with reference to FIGS. In the ninth embodiment, still image encoded data for a new client is continuously generated and transmitted to the new client. Thereafter, the same moving image data as that of the existing client is transmitted from the time when the next key frame (I frame) is transmitted to the existing client. Since the image streaming system of the ninth embodiment has the same configuration as the image streaming system of FIGS. 1 and 7 described in the first and eighth embodiments, the description thereof is omitted.

  Next, a processing procedure for streaming control in the transmission server 2 will be described. Here, the processing procedure of the streaming control in the transmission server 2 will be described using the flowchart of FIG. 3 as in the first to third and eighth embodiments. Note that, in the image streaming system of the ninth embodiment, the moving image streaming control to the receiving client 3 being connected is the same as the processing described in the first to third embodiments, and thus the description thereof is omitted.

  Here, as an operation different from the image streaming systems of the first to third and eighth embodiments, an operation when a new receiving client is connected during moving image streaming control will be described.

  If there is a newly connected reception client (new client) 3 during streaming to the connected reception client (existing client) 3, it is determined that there is a new distribution request in the processing of S11, and the application processing unit 23 Performs connection setting with the newly connected receiving client 3 and processing when a new distribution request is detected (S18).

  First, the application processing unit 23 determines whether or not the data to be transmitted is a key frame. The application processing unit 23 does nothing when the encoded data to be next streamed to the existing client is a key frame.

  On the other hand, if the encoded data to be streamed next to the existing client is not a key frame, still image encoded data is generated by the still image encoder 25 and server processing is performed on the new client by the control of the server control unit 24 (see FIG. 3 is a flow indicated by a broken line). As a result, the encoded still image data is sent to the new client and streaming is started.

  Thereafter, the transmission server 2 proceeds to the process of S12. The subsequent processing is the same as the moving image streaming control to the connected receiving client 3 described in the first and eighth embodiments, but the application processing unit 23 determines that the data to be transmitted in the processing of S12 is a key frame. If it is determined, the intra-frame encoded data encoded in S15 is also streamed to the new client.

  After the application processing unit 23 sets the new distribution identification, when the application processing unit 23 detects that the new distribution identification is set (S14), the still image encoder 25 performs still image encoded data (2 The first and subsequent still image encoded data) is generated, and server processing is performed on the new client under the control of the server control unit 24. As a result, the still image encoded data is sent to the new client. Still image encoded data is generated by the still image encoder 25 until the next encoding process is an intra-frame encoding for generating a key frame, and server processing is performed on a new client by control of the server control unit 24. . The application processing unit 23 cancels the new distribution identification only when the next encoding process is intraframe encoding for creating a key frame, and thereafter, the same moving image encoded data as the existing client is streamed to the new client. To do.

  As described above, the still image encoded data is continuously sent to the new client until the next key frame by the server processing in S17. At this time, the still image encoding is performed using a method such as M-JPEG (Motion-JPEG). Data may be sent out.

  The reception client 3 (existing client or new client) receives the encoded data streamed from the transmission server 2 and displays the image, as in the processing in the first embodiment. In Embodiment 9, after the moving image encoder 22 performs the intra-frame encoding process, the intra-frame encoded data generated by the moving image encoder 22 is not saved (S16).

  FIG. 9 is a diagram illustrating an image streaming frame configuration according to the ninth embodiment. As shown in FIG. 9, the existing client is sequentially streamed with the P frame 10y, the P frame 10z, the I frame 11, the P frame 11a, and the P frame 11b.

  Here, when the new client connects to the transmission server 2 while streaming the P frame 11b to the existing client (S11 in FIG. 3, YES), the existing client is then sequentially streamed with the P frame 11c and the P frame 11d. to continue.

  The still image frame 30a (still image encoded data) generated by the still image encoder 25 is sent to the new client. Thereafter, the P frame 11c, the P frame 11d, and the P frame 11e are sequentially streamed to the existing client. While performing, the still image frame 30b, the still image frame 30c, and the still image frame are sequentially transmitted to the new client.

  Thereafter, streaming to the new client is performed sequentially from the I frame 12, which is the next key frame to the existing client, with the P frame 12a and the P frame 12b at the same timing as the existing client.

  As described above, according to the ninth embodiment, a series of streaming can be maintained without any change to existing connected clients, and still image encoded data is continuously transmitted to newly connected new clients without delay. Therefore, while maintaining stable streaming to the existing client, it is not necessary for the new client to wait for the start of moving image display, and it is possible to continuously display still images in a short time and then display moving images.

(Embodiment 10)
Next, a tenth embodiment of the present invention will be described with reference to FIGS. 1, 3 and 7 to 9. In the tenth embodiment, when it is determined whether or not to execute still image compression encoding based on the accumulated code amount of still image encoded data sent to a new client, and it is determined to execute still image compression encoding Only generate still image encoded data and send it to a new client. Since the image streaming system of the tenth embodiment has the same configuration as the image streaming system of FIGS. 1 and 7 described in the first, eighth, and ninth embodiments, the description thereof is omitted.

  Next, a processing procedure for streaming control in the transmission server 2 will be described. Here, the processing procedure of the streaming control in the transmission server 2 will be described using the flowchart of FIG. Note that, in the image streaming system of the tenth embodiment, the moving image streaming control to the receiving client 3 being connected is the same as the processing described in the first to third embodiments, and thus the description thereof is omitted. Here, a processing procedure of streaming control different from the ninth embodiment will be described.

  After the application processing unit 23 sets the new distribution identification, when the application processing unit 23 detects that the new distribution identification is set (S14), the application processing unit 23 selects a predetermined pattern (for example, A pattern image in which a predetermined character or picture is written on a blue background) (an image connected until the first image display is performed by the receiving client 3) or an accumulated code amount of still image encoded data transmitted to a new client ( It is determined whether or not to perform still image compression coding based on the amount of data that has been streamed. When it is time to generate still image encoded data, the application processing unit 23 determines whether or not to perform still image compression encoding at each timing to generate still image encoded data.

  When still image compression encoding is executed, still image encoded data is generated by the still image encoder 25, and server processing is performed on the new client under the control of the server control unit 24. As a result, the still image encoded data is sent to the new client. The new distribution identification is canceled only when the next encoding process is intra-frame encoding for creating a key frame, and thereafter, the same moving image encoded data as that of the existing client is streamed to the new client.

  The frame structure of the image streaming according to the tenth embodiment has the same structure as the frame structure of the image streaming according to the ninth embodiment shown in FIG. The tenth embodiment is different from the ninth embodiment in that the transmission of a still image frame to a new client (timing) may or may not occur.

  As described above, according to the tenth embodiment, a series of streaming can be maintained without any change to an existing client connected. In addition, still image encoded data can be sent to a newly connected new client without delay according to a predetermined pattern, and generally occupies a transmission band with still image data having a larger amount of data than moving image data. There is no. As a result, while maintaining stable streaming to existing clients, it is not necessary for the new client to wait for the start of moving image display, and it is possible to continuously display still images in a short time and then display moving images in a short time. Become.

(Embodiment 11)
Next, an eleventh embodiment of the present invention will be described with reference to FIGS. In the eleventh embodiment, when the GOV header 93 defined in MPEG-4 is generated when the moving image encoder 22 performs the intraframe encoding processing in the eighth to tenth embodiments, The GOV header 93 and the like are stored together with the encoded data and transmitted to the receiving client 3.

  The image streaming system of the eleventh embodiment has the same configuration as the image streaming system of FIG. 1 described in the first embodiment and the image streaming system of FIG. 7 described in the eighth to tenth embodiments. Description is omitted.

  Next, a processing procedure for streaming control in the transmission server 2 will be described. Here, similarly to Embodiments 1 to 3 and 8 to 10, a processing procedure of streaming control in the transmission server 2 will be described using the flowchart of FIG. Note that in the image streaming system of the eleventh embodiment, the moving image streaming control to the receiving client 3 being connected is the same as the processing described in the first to third and eighth to tenth embodiments, and the description thereof is omitted. To do.

  The processing in the case where there is a new client that is a newly connected receiving client during streaming to an existing client that is a connected receiving client is the content described in Embodiments 8 to 10 except for the content described below. It is the same.

  First, in the eleventh embodiment, when the GOV header 93 defined in MPEG-4 is generated when the moving image encoder 22 performs the intra-frame encoding process (S15), the moving image encoder The GOV header 93 is stored together with the intra-frame encoded data generated by the terminal 22 (S16).

  Furthermore, the difference from Embodiments 8 to 10 is that the new distribution identification is not canceled in S14, and if the new distribution identification is set when the intraframe encoded data is generated in S15, the new distribution is set. The identification is canceled and the intra-frame encoded data is server-processed together with the GOV header 93 stored in S16 (S17). As a result, when a new delivery request is issued from a new client, if the data sent from the transmission server 2 to the existing client is a key frame, the intra-frame encoded data is stored together with the GOV header 93 stored in the new client. Can be streamed to new clients. Note that the transmission server may send the intra-frame encoded data having the GOV header 93 to a new client or an existing client without saving.

  In the eleventh embodiment, the GOV header 93 defined in MPEG-4 is stored together with the intra-frame encoded data generated by the moving image encoder 22 and is streamed to a new client. The header that is stored together with the data and streamed to the new client is not limited to the GOV header 93 defined in MPEG-4. That is, together with the intra-frame encoded data generated by the video encoder 22, a header (a header equivalent to the GOV header 93) defined by another video encoding method may be stored and streamed to a new client. .

  As described above, according to the eleventh embodiment, in addition to the effects of the eighth to tenth embodiments, the new client can acquire information included in the GOV header 93 or a header equivalent thereto. Therefore, it is possible to provide an image streaming system that is more flexible than the image streaming systems according to the eighth to tenth embodiments.

(Embodiment 12)
Next, a twelfth embodiment of the present invention will be described with reference to FIGS. In the twelfth embodiment, the VOL header 92 and the GOV header 93 stipulated in MPEG-4 are generated when the moving picture encoder 22 performs the intra-frame encoding process in the eighth to tenth embodiments. Stores the VOL header 92 and the GOV header 93 together with the intra-frame encoded data, and transmits them to the receiving client 3.

  The image streaming system of the twelfth embodiment has the same configuration as the image streaming system of FIG. 1 described in the first embodiment and the image streaming system of FIG. 7 described in the eighth to tenth embodiments. Description is omitted.

  Next, a processing procedure for streaming control in the transmission server 2 will be described. Here, similarly to Embodiments 1 to 3 and 8 to 10, a processing procedure of streaming control in the transmission server 2 will be described using the flowchart of FIG. In the image streaming system according to the twelfth embodiment, the moving image streaming control to the receiving client 3 being connected is the same as the processing described in the first to third and eighth to tenth embodiments, and thus the description thereof is omitted. To do.

  The processing in the case where there is a new client that is a newly connected receiving client during streaming to an existing client that is a connected receiving client is the content described in Embodiments 8 to 10 except for the content described below. It is the same.

  First, in the twelfth embodiment, when the moving image encoder 22 performs the intra-frame encoding process (S15), when the VOL header 92 and the GOV header 93 defined in MPEG-4 are generated, The VOL header 92 and the GOV header 93 are stored together with the intra-frame encoded data generated by the image encoder 22 (S16).

  Furthermore, the difference from Embodiments 8 to 10 is that the new distribution identification is not canceled in S14, and if the new distribution identification is set when the intraframe encoded data is generated in S15, the new distribution is set. The identification is canceled, and the intra-frame encoded data is server-processed together with the VOL header 92 and the GOV header 93 stored in S16 (S17). Thereby, the intra-frame encoded data can be streamed to the new client together with the stored VOL header 92 and GOV header 93.

  In the twelfth embodiment, the VOL header 92 and the GOV header 93 defined in MPEG-4 are stored together with the intraframe encoded data generated by the moving image encoder 22 and streamed to a new client. The header that is stored together with the intra-frame encoded data and is streamed to the new client is not limited to the VOL header 92 and GOV header 93 defined in MPEG-4. That is, together with the intra-frame encoded data generated by the moving image encoder 22, the headers defined by other moving image encoding methods (the header equivalent to the VOL header 92 and the header equivalent to the GOV header 93) are stored. You may stream to a new client.

  As described above, according to the twelfth embodiment, in addition to the effects of the eighth to tenth embodiments, the new client can include information included in a plurality of headers (information included in a VOL header 92 or a header equivalent thereto, Information included in the GOV header 93 or the header equivalent thereto), the receiving client does not need to operate with a predetermined control parameter, and is more flexible than the image streaming systems of the eighth to eleventh embodiments. It is possible to provide a high image streaming system.

(Embodiment 13)
Next, a thirteenth embodiment of the present invention will be described with reference to FIGS. In the thirteenth embodiment, when the moving image encoder 22 performs the intraframe encoding process in the eighth to tenth embodiments, the VO header 91, the VOL header 92, the GOV header 93, etc. defined in MPEG-4 are added. If generated, the VO header 91, the VOL header 92, the GOV header 93, etc. are stored together with the intra-frame encoded data, and transmitted to the receiving client 3.

  The image streaming system of the thirteenth embodiment has the same configuration as the image streaming system of FIG. 1 described in the first embodiment and the image streaming system of FIG. 7 described in the eighth to tenth embodiments. Description is omitted.

  Next, a processing procedure for streaming control in the transmission server 2 will be described. Here, similarly to Embodiments 1 to 3 and 8 to 10, a processing procedure of streaming control in the transmission server 2 will be described using the flowchart of FIG. In the image streaming system according to the twelfth embodiment, the moving image streaming control to the receiving client 3 being connected is the same as the processing described in the first to third and eighth to tenth embodiments, and thus the description thereof is omitted. To do.

  In the thirteenth embodiment, when the moving image encoder 22 performs the intra-frame encoding process (S15), when the VOL header 92 and the GOV header 93 defined in MPEG-4 are generated, the moving image code The VO header 91, the VOL header 92, and the GOV header 93 are stored together with the intra-frame encoded data generated by the encoder 22 (S16).

  The processing when there is a new client that is a newly connected receiving client during streaming to an existing client that is a connected receiving client is the same as that described in Embodiments 8 to 10 except for the content described below. It is the same.

  First, in the thirteenth embodiment, when the moving image encoder 22 performs the intra-frame encoding process (S15), when the VOL header 92 and the GOV header 93 defined in MPEG-4 are generated, The VO header 91, the VOL header 92, and the GOV header 93 are stored together with the intra-frame encoded data generated by the image encoder 22 (S16).

  Furthermore, the difference from Embodiments 8 to 10 is that the new distribution identification is not canceled in S14, and if the new distribution identification is set when the intraframe encoded data is generated in S15, the new distribution is set. The identification is canceled, and the intra-frame encoded data is server-processed together with the VO header 91, the VOL header 92, and the GOV header 93 stored in S16 (S17). As a result, the intra-frame encoded data can be streamed to the new client together with the stored VO header 91, VOL header 92 and GOV header 93.

  In the thirteenth embodiment, the VO header 91, the VOL header 92, and the GOV header 93 defined in MPEG-4 are stored together with the intra-frame encoded data generated by the moving image encoder 22 and streamed to a new client. However, the headers stored together with the intra-frame encoded data and streamed to the new client are not limited to the VO header 91, the VOL header 92, and the GOV header 93 defined in MPEG-4. That is, together with the intra-frame encoded data generated by the moving image encoder 22, headers defined by other moving image encoding methods (a header equivalent to the VO header 91, a header equivalent to the VOL header 92, and a GOV header 93 May be saved and streamed to a new client.

  As described above, according to the thirteenth embodiment, in addition to the effects of the eighth to tenth embodiments, the new client can receive information contained in a plurality of headers (information contained in the VO header 91 or an equivalent header, VOL Header 92 or information included in the header equivalent thereto, information included in the GOV header 93 or header equivalent thereto, and the receiving client does not need to operate with a predetermined control parameter. It is possible to provide an image streaming system that is more flexible than the image streaming systems of modes 8-12.

(Embodiment 14)
Next, a fourteenth embodiment of the present invention will be described with reference to FIGS. In the fourteenth embodiment, when the moving image encoder 22 performs the intra-frame coding process in the eighth to tenth embodiments, the VOS header 90, the VO header 91, the VOL header 92, and the GOV header 93 defined in MPEG-4 are used. And the like, the VOS header 90, the VO header 91, the VOL header 92, the GOV header 93 and the like are stored together with the intra-frame encoded data, and transmitted to the receiving client 3.

  The image streaming system of the fourteenth embodiment has the same configuration as the image streaming system of FIG. 1 described in the first embodiment and the image streaming system of FIG. 7 described in the eighth to tenth embodiments. Description is omitted.

  Next, a processing procedure for streaming control in the transmission server 2 will be described. Here, similarly to Embodiments 1 to 3 and 8 to 10, a processing procedure of streaming control in the transmission server 2 will be described using the flowchart of FIG. In the image streaming system according to the twelfth embodiment, the moving image streaming control to the receiving client 3 being connected is the same as the processing described in the first to third and eighth to tenth embodiments, and thus the description thereof is omitted. To do.

  In the fourteenth embodiment, when the moving image encoder 22 performs the intraframe encoding process (S15), the VOS header 90, the VO header 91, the VOL header 92, and the GOV header 93 defined in MPEG-4 are generated. If so, the VOS header 90, the VO header 91, the VOL header 92, and the GOV header 93 are stored together with the intra-frame encoded data generated by the moving image encoder 22 (S16).

  The processing when there is a new client that is a newly connected receiving client during streaming to an existing client that is a connected receiving client is the same as that described in Embodiments 8 to 10 except for the content described below. It is the same.

  First, in the thirteenth embodiment, when the moving image encoder 22 performs the intraframe encoding process (S15), the VOS header 90, the VO header 91, the VOL header 92, and the GOV header 93 defined in MPEG-4 are used. Is generated, the VOS header 90, the VO header 91, the VOL header 92, and the GOV header 93 are stored together with the intra-frame encoded data generated by the moving image encoder 22 (S16).

  Furthermore, the difference from Embodiments 8 to 10 is that the new distribution identification is not canceled in S14, and if the new distribution identification is set when the intraframe encoded data is generated in S15, the new distribution is set. The identification is canceled, and the intra-frame encoded data is processed along with the VOS header 90, the VO header 91, the VOL header 92, and the GOV header 93 stored in S16 (S17). Thereby, the intra-frame encoded data can be streamed to the new client together with the stored VOS header 90, VO header 91, VOL header 92, and GOV header 93.

  In the fourteenth embodiment, the VOS header 90, the VO header 91, the VOL header 92, and the GOV header 93 defined in MPEG-4 are stored together with the intra-frame encoded data generated by the moving image encoder 22, and a new one is stored. Although streaming to the client is performed, the header to be stored together with the intra-frame encoded data and streamed to the new client is not limited to the VOS header 90, VO header 91, VOL header 92, and GOV header 93 defined in MPEG-4. That is, together with the intra-frame encoded data generated by the moving image encoder 22, a header defined by another moving image encoding method (a header equivalent to the VOS header 90, a header equivalent to the VO header 91, and a VOL header 92 May be stored and streamed to a new client.

  As described above, according to the fourteenth embodiment, in addition to the effects of the eighth to tenth embodiments, the new client can receive information contained in a plurality of headers (information contained in the VOS header 90 or an equivalent header, VO Information included in the header 91 or an equivalent header, information included in the VOL header 92 or an equivalent header, information included in the GOV header 93 or an equivalent header), and the receiving client It is not necessary to operate with the determined control parameters, and it is possible to provide an image streaming system having higher flexibility than the image streaming systems of the eighth to thirteenth embodiments.

(Embodiment 15)
Next, a fifteenth embodiment of the present invention will be described with reference to FIGS. In the fifteenth embodiment, a predetermined video encoder is selected from a plurality of video encoders, and streaming to a new client is performed.

  FIG. 10 is a block diagram showing a configuration of an image streaming system according to Embodiment 15 of the present invention. Components that achieve the same functions as those in the image streaming systems of Embodiments 1 and 8 shown in FIGS. 2 and 7 among the components in FIG. 15 are assigned the same numbers, and redundant descriptions are omitted.

  The transmission server 2 of the image streaming system according to the fifteenth embodiment includes an image input unit 21, a plurality of moving image encoders 22A to 22Z, a still image encoder 25, an application processing unit 23, and a server control unit 24. Yes. The image input unit 21 is connected to the moving image encoders 22A to 22Z and the still image encoder 25, and the moving image encoders 22A to 22Z and the still image encoder 25 are connected to the application processing unit 23. .

  The plurality of moving image encoders 22A to 22Z define image data from the image input unit 21 at various resolutions (moving image resolutions) set for each of the moving image encoders 22A to 22Z, such as MPEG-4. According to the above, moving image compression encoding is performed.

  The server control unit 24 here performs connection control and disconnection control for connection requests and disconnection requests from the receiving client 3, protocol control such as starting and stopping of streaming, and moving image encoders 22A to 22Z, The encoded data generated by the still image encoder 25 is streamed to the receiving client 3 via the network 1.

  The receiving client 3 of the image streaming system according to the fifteenth embodiment has the same configuration as the receiving client 3 of the image streaming system according to the eighth embodiment shown in FIG.

  Next, a processing procedure for streaming control in the transmission server 2 will be described. Here, the processing procedure of the streaming control in the transmission server 2 will be described using the flowchart of FIG. 3 as in the first to third and eighth embodiments. In the image streaming system according to the fifteenth embodiment, the moving image streaming control to the receiving client 3 being connected is the same as the processing described in the eighth embodiment, so that the description thereof is omitted. Here, as an operation different from the image streaming system of the eighth embodiment, an operation when a new receiving client is connected during moving image streaming control will be described.

  When a new client is connected to the transmission server 2, if there is a moving image encoder that performs encoding at the same resolution as the moving image requested by the new client, the moving image encoder is selected. The streaming to the new client is performed according to any of the procedures described in the first to fourteenth embodiments.

  There is no moving image encoder that performs encoding at the same resolution as the moving image resolution requested by the new client, and there is a free space that has not been encoded among a plurality of moving image encoders ( If there is a video encoder that is not in use, a free video encoder is selected and a new client is selected according to the procedure of the first embodiment (control similar to the video streaming control to the receiving client). Streaming. If the moving image encoder for performing streaming to the new client cannot be selected with the above-described content, the connection with the new receiving client is disconnected.

  In the fifteenth embodiment, the transmission server 2 of the image streaming system includes the still image encoder 25. However, the transmission server 2 may not include the still image encoder 25.

  In the fifteenth embodiment, the reception client 3 of the image streaming system includes the still image encoder 35. However, the reception client 3 may not include the still image encoder 35.

  As described above, according to the fifteenth embodiment, when there is a connection from a new client, even when all of the plurality of moving image encoders are being executed, the moving image encoder that is executing the encoding at the same resolution. Can be streamed to a new client according to any of the procedures described in Embodiments 1 to 14, so that the new client can display an image in a short time while maintaining stable streaming to the existing client. Is possible.

  As described above, the image streaming system according to the present invention is suitable for streaming image data.

The figure which shows schematic structure of the image streaming system which concerns on Embodiment 1 of this invention. 1 is a block diagram showing a configuration of an image streaming system according to Embodiment 1 of the present invention. Flowchart showing processing procedure of streaming control in transmission server The figure which shows the frame composition of moving picture streaming The figure for demonstrating the internal structure of the encoding data in a frame, and the prediction encoding data between frames. The figure which shows the frame composition of moving picture streaming Block diagram showing a configuration of an image streaming system according to an eighth embodiment of the present invention. The figure which shows the frame structure of the image streaming of Embodiment 8 of this invention. The figure which shows the frame structure of the image streaming of Embodiment 9 of this invention. Block diagram showing a configuration of an image streaming system according to Embodiment 15 of the present invention. The figure for demonstrating the 1st example of the conventional image streaming The flowchart which shows the process sequence of the 1st example of the conventional image streaming. The figure for demonstrating the 2nd example of the conventional image streaming The flowchart which shows the process sequence of the 2nd example of the conventional image streaming.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Network 2 Transmission server 3 Reception client 9 Video frame 21 Image input part 22, 22A-22Z Video encoder (encoded data production | generation part)
23 Application processing unit 24 Server control unit (streaming unit)
25 Still image encoder (still image data generator)
31 Image Output Unit 32 Video Decoder 33 Application Processing Unit 34 Client Control Unit 35 Still Image Encoder 90 VOS Header 91 VO Header 92 VOL Header 93 GOV Header 94 VOP Header 95 Encoded Data

Claims (14)

A transmission server that compresses and encodes moving image data and streams the compressed and encoded data through the network, and a compression server that receives and receives the compressed and encoded data streamed by the transmission server through the network. A plurality of reception clients that perform decoding and decoding of the encoded data to display an image, and the intra-frame encoded data generated by intra-frame encoding of the compressed encoded data from the transmission server to the reception client; and In an image streaming system for streaming inter-frame predictive encoded data generated by inter-frame predictive encoding of the compressed encoded data,
The sending server is
An encoded data generation unit for generating the intra-frame encoded data and the inter-frame predictive encoded data;
A streaming unit for streaming the intra-frame encoded data and the inter-frame prediction encoded data generated by the encoded data generation unit to the receiving client and storing the intra-frame encoded data to be streamed to the receiving client When,
With
The streaming unit
When the data to be streamed to the connected client that is a receiving client that is already connected to the local server is the timing of the compressed encoded data other than the intra-frame encoded data, a new receiving client different from the connected client When receiving a connection request from a new client, the image streaming system starts streaming to the new client using the stored intra-frame encoded data. After the streaming unit starts streaming to the new client, the encoded data generation unit generates the intra-frame encoded data and the inter-frame prediction encoded data to be streamed to the connected client. The image streaming system according to claim 1, wherein streaming is performed to the new client at the same timing as streaming to the client. The encoded data generation unit stores the frame that has been streamed to the connected client during the period from storing the intra-frame encoded data to streaming to the new client using the stored intra-frame encoded data. Inter-frame predictive encoded data having the same number of frames as inter-predicted encoded data, and generating inter-frame predictive encoded data in which a header corresponding to the number of frames is set,
When the streaming unit starts streaming to the new client, the intra-frame encoded data stored, and the same number of inter-frame predictive encodings as the number of frames set as headers in the encoded data generation unit The intra-frame encoded data and the inter-frame predictive codes in the order of the inter-frame predictive encoded data streamed to the connected client after streaming the stored intra-frame encoded data to the new client 2. The image streaming system according to claim 1, wherein the digitized data is streamed to the new client. The encoded data generation unit calculates an elapsed time after storing the intra-frame encoded data as inter-frame predictive encoded data to be streamed to a new client next to the stored intra-frame encoded data. Generate inter-frame predictive encoded data with the header shown
When the streaming unit starts streaming to the new client, the frames are stored in the order of the stored intra-frame encoded data and inter-frame predictive encoded data in which a header is set in the encoded data generation unit. The image streaming system according to claim 3, wherein inner coded data and the inter-frame predictive coded data are streamed to the new client. The encoded data generation unit generates the intra-frame encoded data by setting a control parameter for displaying an image in a header,
The streaming unit, when streaming or storing the intra-frame encoded data, streams or stores the intra-frame encoded data together with a header set by the encoded data generation unit. The image streaming system as described in any one of -4. A transmission server that compresses and encodes moving image data and streams the compressed and encoded data through the network, and a compression server that receives and receives the compressed and encoded data streamed by the transmission server through the network. A plurality of reception clients that perform decoding and decoding of the encoded data to display an image, and the intra-frame encoded data generated by intra-frame encoding of the compressed encoded data from the transmission server to the reception client; and In an image streaming system for streaming inter-frame predictive encoded data generated by inter-frame predictive encoding of the compressed encoded data,
The sending server is
An encoded data generation unit for generating the intra-frame encoded data and the inter-frame predictive encoded data;
When a connection request is received from a new client that is a new receiving client other than the connected client that is already connected to the local server, the still image data is generated by compressing and encoding the still image data. A still image data generating unit,
A streaming unit for streaming the intra-frame encoded data and the inter-frame prediction encoded data generated by the encoded data generation unit and the still image encoded data generated by the still image data generation unit to the reception client;
With
The streaming unit
Still image coding generated by the still image data generation unit upon receiving a connection request from the new client when the data to be streamed to the connected client is at the timing of compressed encoded data other than the intra-frame encoded data After sending data to the new client, the encoded data generation unit generates and streams the intra-frame encoded data and the inter-frame predictive encoded data to the connected client in the same manner as the streaming to the connected client. An image streaming system for streaming to the new client at a timing. The still image data generation unit generates a plurality of still image encoded data when receiving a connection request from the new client,
The image streaming system according to claim 6, wherein the streaming unit continuously transmits a plurality of still image encoded data generated by the still image data generation unit to the new client. The still image data generation unit
The encoded still image and data of a predetermined pattern sent to the new client at the timing for generating the encoded still image data until the first intra-frame encoded data is streamed to the new client that has received the connection request. Whether or not to generate the still image encoded data is determined based on the accumulated code amount, and when it is determined to generate the still image encoded data, the still image encoded data is generated and transmitted to the new client. The image streaming system according to claim 7, wherein the image streaming system is transmitted. The encoded data generation unit generates the intra-frame encoded data by setting a control parameter for displaying an image in a header,
The streaming unit, when streaming the intra-frame encoded data, streams the intra-frame encoded data together with a header set by the encoded data generation unit. The image streaming system as described in any one. The header in which the encoded data generation unit sets the control parameter is:
10. The image streaming according to claim 5 or 9, wherein the image streaming is a GOV header defined in MPEG-4 or a header corresponding to the GOV header defined in another moving image encoding system different from MPEG-4. system. The header in which the encoded data generation unit sets the control parameter is a header corresponding to the GOV header specified in MPEG-4 or another moving image encoding method different from MPEG-4, 11. A VOL header defined in MPEG-4 or a header corresponding to the VOL header defined in another moving image encoding system different from MPEG-4, The image streaming system according to one. The header in which the encoded data generation unit sets the control parameter is a header corresponding to the GOV header specified in MPEG-4 or another moving image encoding method different from MPEG-4, A VOL header defined in MPEG-4 or a header corresponding to the VOL header defined in another moving image encoding system different from MPEG-4, and a VO header defined in MPEG-4 or different from MPEG-4 The image streaming system according to any one of claims 5 and 9 to 11, wherein the image streaming system is a header corresponding to the VO header defined by another moving image encoding method. The header in which the encoded data generation unit sets the control parameter is a header corresponding to the GOV header specified in MPEG-4 or another moving image encoding method different from MPEG-4, VOL header defined in MPEG-4 or a header corresponding to the VOL header defined in another moving image coding system different from MPEG-4, VOL header defined in MPEG-4 or other different from MPEG-4 Corresponding to the header corresponding to the VO header defined by the moving picture encoding system of V. and the VOS header defined by MPEG-4 or another moving picture encoding system different from MPEG-4. The image streaming system according to claim 5, wherein the image streaming system is a header. The encoded data generation unit comprises a plurality of
When there is a connection request from the new client, the intra-frame encoded data and the inter-frame prediction encoded data are being generated in the connected client at the same resolution as the moving image resolution requested from the new client. 2. The encoded data generation unit according to claim 1, wherein the encoded data generation unit causes the new client that has requested connection to generate the intra-frame encoded data and the inter-frame prediction encoded data. The image streaming system as described in any one of -13.

Images