JP2007259128A - Distribution server device, client device and program used therefor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To distribute stream media data by making it into false streams. <P>SOLUTION: A distribution server device transmits voice data and still image data after conversion to a client device by every data transfer period upon converting the voice data and moving image data included in the media data stream distributed from a content server into voice data in a data form which can be reproduced by the client device and a plurality of pieces of still image data by unit of frame. In this case, the voice data and the still image data after conversion is dividedly transferred to the voice data corresponding to reproduction time length matched to the data transfer period, image frame data with reproduction time length matched to the data transfer period and an image archive packaged by approximately averaging the still image data in time. Thus, moving image reproduction by the false streams for reproducing the voice data and the plurality of pieces of still image data while downloading them is allowed in the client device. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a distribution server device, a client device, and a program used therefor that can distribute content via a communication network and reproduce the distributed content. In particular, the present invention relates to a technique for distributing stream media data to a delivery destination client device in a pseudo stream.

  Recently, with the development of communication networks such as the Internet, mobile communication terminals such as mobile phones and PDAs (Personal Data (or Digital) Assistants) with communication functions have been used as platforms, anytime and anywhere via communication networks. You can enjoy content such as acquired music and ringtones. In particular, in recent years, the so-called RSS (RDF (Resource Description Framework) Site Summary) feed, which is widely used as an information distribution service for personal computer terminals, is used to distribute update information in RSS format from each site even on mobile communication terminals. You can refer to it. For example, the page address URL (Uniform Resource Locator) is "http: //www.ecreal.co.jp/pc/service/rss/index.html". The technology described on the homepage (that is, the Web site) on the Internet indicated by “” is conventionally known.

  According to the prior art described in the above website, a dedicated application software program called a so-called “RSS reader” (or “feed reader”) or the like can be used to access a user from a previously registered website. Update information in RSS format for PC terminals can also be viewed on mobile phones. That is, by patroling each registered site and displaying a digest of one or more articles on each site together with a link to the article, the user is notified of the articles published on the site. Hey. Then, the user can refer to the HTML file (Web page related to the updated article) as a link destination by clicking the displayed link. However, in consideration of the display size of the mobile phone and the data communication speed, only text of up to about 10,000 characters can be displayed as a screen at a time. In addition, when an image or the like is attached to RSS, the size of the image is reduced so that it can be displayed on a mobile phone.

  As described above, in the conventional technology, by appropriately converting data such as reducing the size of an image, update information by an RSS feed for a personal computer terminal can be used as a mobile communication terminal even in a mobile communication terminal. You can refer to it. However, images, sounds, and videos attached to RSS cannot be played on mobile communication terminals or are difficult to play back (for example, MP3 encoded audio data, WMV and other mobile communication terminals are not supported) If the data size exceeds the transfer limit (for example, data exceeding 2 MB (megabytes)), the media data cannot be received or received by the mobile communication terminal. It is not possible to use it. That is, the conventional technology has a problem in that it is not convenient because it cannot sufficiently provide an information delivery service using an RSS feed to a user who uses a mobile communication terminal as a platform.

  The present invention has been made in view of the above points, and a distribution server device and a client device that can provide an information distribution service by an RSS feed for a personal computer terminal to a mobile communication terminal (client device) without any inconvenience. It aims at providing the program used for them.

  A distribution server device according to claim 1 of the present invention is a distribution server device that is interconnected to a client device and a content server via a communication network, and mediates data distribution between the devices, and includes a stream from the content server. Data receiving means for receiving media data for video playback to be distributed; audio data and video data included in the received media data, respectively, audio data in a predetermined format and a plurality of still images in a predetermined format in units of frames Conversion means for converting into image data and storing; generation means for generating reproduction instruction data defining at least a predetermined data transfer period, a reproduction bit rate of audio data and / or a reproduction frame rate of image data; The converted audio data for each playback time length that matches the data transfer cycle Dividing means for dividing, and image frame data having a reproduction time length that matches the data transfer cycle, and the plurality of still image data converted for each reproduction time that matches the data transfer cycle are substantially averaged in terms of time. The archive generation means for generating a plurality of packaged image archives by averaging the still image data in terms of time so as to be transmitted in response to a request from the client device for each data transfer period. Transmitting means for transmitting the data to the client device in the order of the generated reproduction instruction data, the divided audio data, and the plurality of generated image archives.

  According to the present invention, the distribution server device has a function of mediating data transfer between a content server that provides an information distribution service and a client device that receives the service, and data of media data stream-distributed from the content server After the conversion, the converted audio data and still image data are transmitted to the client device at predetermined data transfer cycles. In other words, the distribution server device converts each of the audio data and the moving image data included in the media data stream-distributed from the content server into audio data in a data format that can be reproduced by the client device and still image data in a plurality of frames. To do. The audio data corresponding to the playback time length that matches the data transfer cycle and the image frame data of the playback time length that matches the data transfer cycle, which are substantially averaged in time and packaged The converted audio data and still image data are transferred to the image archive in a divided manner. In this way, the client device can perform moving image reproduction in a so-called pseudo stream in which audio data and a plurality of still image data are reproduced while being downloaded every reproduction time that matches the data transfer cycle. Therefore, an information distribution service can be sufficiently provided to a user who uses a client device such as a mobile communication terminal as a platform without any inconvenience.

  A client device according to claim 3 of the present invention is a client device interconnected to a distribution server device via a communication network, wherein a predetermined data transfer cycle, a reproduction bit rate of audio data, and / or image data One or a plurality of images comprising reproduction instruction data defining at least a reproduction frame rate, audio data corresponding to a reproduction time length matching the data transfer cycle, and image frame data having a reproduction time length matching the data transfer cycle Means for requesting the distribution server device to repeatedly transfer data in the order of archiving; receiving means for receiving the data from the distribution server device in response to the transfer request; and the received reproduction instruction Setting means for setting a data transfer cycle for repeatedly performing the transfer request according to data; A reproduction means for reproducing the received audio data for each data transfer period, and the one or more received image archives are sequentially expanded for each set data transfer period, and are included in the expanded image archive Display means for sequentially displaying a frame image at a time position corresponding to each frame image, wherein the display by the frame image included in the image archive that could not be received in the data transfer period is arranged at the time position immediately before The display of the frame image included in the received image archive is maintained and displayed instead. According to this, the client device can perform moving image reproduction in a so-called pseudo stream that is reproduced while downloading audio data and a plurality of still image data for each data transfer cycle.

  The present invention can be constructed and implemented not only as a device invention but also as a method invention. Further, the present invention can be implemented in the form of a program of a processor such as a computer or a DSP, or can be implemented in the form of a storage medium storing such a program.

  According to the present invention, the distribution server device converts the media data stream-distributed from the content server into a data format that can be reproduced by the client device, and transfers the converted audio data and still image data in small portions. Therefore, there is an effect that any client device can play and enjoy media data stream-distributed from the content server.

  Embodiments of the present invention will be described below in detail with reference to the accompanying drawings.

  FIG. 1 is a system block diagram showing the overall configuration of an embodiment of a content distribution system to which a client device and a distribution server device according to the present invention are applied. The content distribution system shown in this embodiment is a content server CS that provides an information distribution service using an RSS feed, and an end user who receives the information distribution service and related content using an RSS feed, and an RSS article. And transmission / reception of data between the user terminal MT (client device according to the present invention) that reproduces voices, images, and moving pictures attached to articles and the content server CS and the user terminal MT. This is a system in which a distribution server HS (distribution server apparatus according to the present invention) functioning as a so-called proxy is appropriately connected so as to be capable of bidirectional communication via an existing communication network such as the Internet. The user terminal MT is a mobile phone, for example, and the distribution server HS and the content server CS are general computer devices. The user terminal MT and various server devices are interconnected by the Internet via a mobile phone communication line network and a predetermined gateway server (not shown). On the other hand, the mutual connection between the servers may be either wired or wireless, and is not limited to a public telephone network or the Internet, and may include a LAN (Local Area Network) or the like. Note that a content distribution system capable of providing an information distribution service using an RSS feed may have hardware other than the above, but here, a case where the minimum necessary resources are used will be described.

  First, an example of the hardware configuration of either the client device (user terminal MT) or the server device (content server CS, distribution server HS) illustrated in FIG. 1 will be briefly described with reference to FIG. FIG. 2 is a block diagram of a hardware configuration showing an embodiment of the overall configuration of any one of the above devices. However, since each of the above devices can be described as using a similar hardware configuration, only one block diagram of the distribution server HS will be described here as a representative.

  The distribution server HS shown in the present embodiment is controlled by a microcomputer (control means) comprising a microprocessor unit (CPU) 1, a read only memory (ROM) 2, and a random access memory (RAM) 3. . The CPU 1 controls the operation of the entire apparatus. A ROM 2, a RAM 3, an input device 4, an output device 5, a communication interface 6, and an external storage device 7 are connected to the CPU 1 via a data and address bus 1D. The ROM 2 stores various programs executed by the CPU 1 and various data. The RAM 3 is used as a working memory that temporarily stores various data generated when the CPU 1 executes a predetermined program, or as a memory that stores a currently executed program and related data. A predetermined address area of the RAM 3 is assigned to each function and used as a register, flag, table, memory, or the like.

  The input device 4 is an input device such as various switches and buttons, a keyboard for inputting character data, a numeric keypad for inputting numerical data, or a mouse for operating a pointer displayed on a display. The output device 5 displays various screens based on still image data, moving image data, and the like, for example, a display device (display) such as a liquid crystal display panel (LCD) or a CRT, and a playback that emits sounds such as sounds and musical sounds based on audio data. Equipment. The communication interface 6 is a communication interface that is connected to a communication network and transmits / receives various information and various data between devices (user terminal MT, content server CS, etc.) connected via the communication network. The communication interface 6 is not limited to a wired one but may be a wireless one. Moreover, you may have both.

  The external storage device 7 stores various control programs (for example, well-known network software and various application software) executed by the CPU 1. When the control program is not stored in the ROM 2, the control program is stored in the external storage device 7 (for example, a hard disk) and read into the RAM 3 to store the control program in the ROM 2. A similar operation can be performed by the CPU 1. In this way, control programs can be easily added and upgraded. The external storage device 7 is not limited to a hard disk (HD), but can be attached in various forms such as a floppy disk (FD), compact disk (CD), magneto-optical disk (MO), DVD (abbreviation of Digital Versatile Disk), etc. It may be a storage device using the external storage medium. Alternatively, a semiconductor memory such as a flash memory may be used. The distribution server HS is not limited to one having the input device 4 and the output device 5 incorporated in one apparatus main body, and each is configured separately and configured to connect the respective apparatuses using a predetermined interface. Needless to say, this can be applied to the same.

  Returning to the description of FIG. 1, the content server CS provides an automatic update type information distribution service using an RSS feed known as a so-called pod cast for a general computer device (for example, a personal computer terminal) not shown. It is a server computer to provide. Specifically, the content server CS stores a large amount of stream media data (simply referred to as media data) such as audio, images, and videos in various formats such as MP3, WMA, WMV, and MOV as content, The server administrator (or content holder) updates, adds, and deletes the media data at any time from the usual time. The public information of the media data is described in digests using RDF (Resource Description Framework) or XML (eXtensible Markup Language) in an arbitrary directory unit, and is provided to the user terminal MT as an RSS feed. Yes.

  In a general computer device (personal computer terminal) targeted by the above-mentioned content server CS for an RSS feed information distribution service, an application software program called “RSS reader” is used to connect the content server CS corresponding to the URL described in the RSS. When accessed, the content is acquired from the accessed content server CS. That is, not only the articles described in RSS but also media data such as images, sounds, and videos attached to the articles are received from the content server CS by stream distribution, and the display and playback of the contents are performed on the computer device side. It can be done while receiving data. On the other hand, if the content server CS is originally a user terminal MT (for example, a mobile phone) that is not an object of an information delivery service using an RSS feed, it is described in RSS using the current hardware and software incorporated in the terminal. Articles and various media data attached to articles cannot be displayed or played while receiving the data. In particular, various media data composed of formats such as MP3, WMA, WMV, and MOV does not have a playback engine corresponding to those formats, and is streamed from the content server CS to general computer devices. Since the media data has a large data size and exceeds the transfer upper limit of the mobile phone, it cannot be delivered to the mobile phone. Therefore, the mobile phone is basically unable to display / reproduce the media data.

The user terminal MT is a mobile communication terminal such as a mobile phone or a PDA, and includes a dedicated application software program (hereinafter simply referred to as an application) for exchanging data with the distribution server HS. The application can hold page addresses, that is, URLs, regarding a plurality of RSSs arbitrarily registered by the user. Further, the application requests RSS (RDF / XML) transfer to the distribution server HS in accordance with a user instruction (specifically, URL designation) or periodically. For example, in the example shown in FIG. 1, when receiving the feed of RSS “podcast1.rdf” in the root directory of the content server CS (page address “http: // contserv”), the page address “http” of the distribution server HS A request describing “ http: // distserv /? url = http: //contserv/podcast1.rdf& ...” in the URL indicating “: // distserv” is transmitted to the distribution server HS. In the RSS feed from the content server CS, description information (URL) of attached media (compressed audio file, compressed moving image data, etc.) is described in addition to the description of the article title, article summary, and link. In the program (RSS reader) according to the present invention, an article is displayed on the output device 5 in accordance with information described in the RSS feed, and a transfer request for the attached media data is made to the distribution server HS. (To be described later). The user terminal MT can reproduce the audio data or display the still image data received from the distribution server HS in response to the media data transfer request by the output device 5.

  The distribution server HS functions as an intermediary server (so-called proxy) between the user terminal MT and the content server CS. The URL access unit U makes a URL request (request in the figure) requested from the user terminal MT to another server (here, the content server CS). The data conversion unit D performs media conversion if the response from the content server CS is media data (Media in the figure) such as voice, image, or video to be converted that is defined in advance, and converts the converted data into The data is accumulated in the media cache MC and sent to the transmission control unit H to cause the user terminal MT to sequentially transfer data. When the converted data corresponding to the URL requested from the user terminal MT has already been stored in the media cache MC, the cache checking unit C does not receive the media data stream from the content server CS again. The converted data in the media cache MC is read and transferred to the user terminal MT. Here, when the URL requested from the user terminal MT corresponds to the RSS feed, the response from the content server CS is directly transferred to the user terminal MT without performing the media conversion and storing the converted data. On the other hand, when the requested URL is media data such as audio data or moving image data (moving image data having an audio track), the media conversion and the storage of the converted data are performed.

  In the distribution server HS, the media data conversion method differs depending on the type of data. That is, when the media data is compressed audio data (MP3 or the like), it is expanded into ADPCM (Adaptive Differential Pulse Code Modulation) data. When the media data is still image data (JPEG, PNG, etc.), it is converted into data of a prescribed format (for example, JPEG). When the media data is moving image data (WMV, MOV, etc.), it is converted into a plurality of still image data (for example, JPEG) and ADPCM data in a specified drawing frame unit. In the media cache MC, the data converted in the above manner is held as a package (for example, with a directory and path name corresponding to a URL) for a predetermined period of several days to several weeks. In the example shown in FIG. 1, the ADPCM data after the compressed audio data conversion is held with the package name “audio.mp3” in the “contserv / media” directory. The data after the still image data conversion is stored with the package name “image.jpg” in the “contserv / media” directory. A plurality of still image data and ADPCM data after conversion of the moving image data are individually stored with the package name “video.wmv” in the “contserv / media” directory. Of course, it goes without saying that the size of each converted data (ADPCM, JPEG) is converted into a data format that can be reproduced by the user terminal MT.

  Here, the data conversion procedure of the distribution server HS will be described with reference to FIG. FIG. 3 is a functional block diagram of the data conversion unit D for explaining the data conversion procedure. FIG. 3A illustrates the conversion procedure for audio data, and FIG. 3B illustrates the conversion procedure for moving image data.

  As shown in FIG. 3A, the voice data is input to the provider (distributor) PV. The provider PV includes a plurality of decoders DC corresponding only to the data format of the audio data that is the conversion source data. The provider PV determines the data format of the input audio data and sends the data to the corresponding decoder DC. On the other hand, the audio data is output. The decoder DC decodes the input audio data, and stores (buffers) the decoded audio sample data in the buffer BF. The buffer BF stores audio sample data of at least a specified number of seconds (for example, 8 seconds). The resampler RS outputs the audio sampling data in the buffer BF as ADPCM data by re-sampling the audio sampling data in the buffer BF at a specified sampling rate (for example, 44 kHz) every specified number of seconds. In this way, the compressed audio data is converted into ADPCM data.

  On the other hand, as shown in FIG. 3B, the moving image data is converted into a plurality of still image data (JPEG) and ADPCM data in accordance with a conversion procedure substantially similar to that of the audio data described above. That is, the moving image data input to the provider PV is decoded by the corresponding decoder DC according to the data format determination, and the decoded data is stored in the buffer BF. However, in the case of moving image data, the buffer BF accumulates frame image data for at least a specified number of seconds (for example, 8 seconds). Then, the resampler RS resamples the frame image data in the buffer BF every specified number of seconds at a specified frame rate (for example, 30 fps) and outputs it as JPEG data for each frame image. Although not shown, audio data included in the moving image data is converted into ADPCM data by the above procedure after being separated from the frame image data. In this way, the moving image data is converted into a plurality of pieces of JPEG data and ADPCM data.

  Returning to the description of FIG. 1, when the media data transferred to the user terminal MT is data to be converted, the distribution server HS appropriately transfers the data quality, transfer rate, etc. when transferring the media data to the user terminal MT. The data transfer mode is adjusted. That is, the user terminal MT notifies the distribution server HS of information (quality request) regarding the requested transfer quality at an arbitrary timing according to the status of the data file that can actually be received from the distribution server HS. The distribution server HS adjusts the data transfer mode according to the notified quality request. For example, if the media data is audio data to be converted, change the sampling rate, if it is still image data, change the image size and / or change the encoding quality (if the format is JPEG, etc.) or video data For example, the frame rate is changed and / or the image size is changed. In addition, when the distribution server HS transfers the audio data to be converted to the user terminal MT, the converted ADPCM data is divided into an incoming melody file (MLD file), and when the moving image data is transferred. The converted JPEG data is archived and transferred in units of a predetermined frame group. Detailed description of such data transmission control to the user terminal MT will be described later. According to this, the transfer of data from the distribution server HS to the user terminal MT is based on the download method instead of the streaming method (referred to as pseudo stream distribution). The user terminal MT actually measures the communication state between the two devices by sending dummy data and Ping to the distribution server MS before starting the data transfer, and notifies the distribution server MS of a quality request corresponding to this. It may be.

  Next, data transfer control to the user terminal MT in the distribution server HS will be described with reference to FIG. FIG. 4 is a functional block diagram of the transmission control unit H for explaining the data transfer control. Data transfer control by the transmission controller H is periodically performed at regular time slot intervals (for example, 8 seconds). That is, the time slot interval is the data transfer cycle. The data transfer procedures 1 to 8 are shown below.

Transfer procedure ADPCM data for one time slot and a plurality of pieces of JPEG data are acquired from the media cache MC in time series, and each is stored in the corresponding buffer BF. That is, the data stored in the buffer BF is updated every time slot.
Transfer procedure 2. The resampler RS reconverts the ADPCM data and JPEG data (data once converted) stored in the buffer BF so as to conform to the current communication state between the user terminal MT and the distribution server HS. . That is, the resampling quality of ADPCM data and JPEG data is changed at an arbitrary timing according to a quality value change request (indicated by req in the figure) for each time slot notified from the user terminal MT. If it is ADPCM data, the sample rate (and / or bit rate) is changed, and if it is JPEG data, the image size (and / or frame rate) is appropriately changed and resampled. As an example, the sample rate is changed to any one of 11, 22, and 44 kHz, and the image size is changed to any one of 240 × 180 pixels, 176 × 144 pixels, and 128 × 96 pixels. Normally, the bit rate is fixed at, for example, 16 kbps, and the frame rate is fixed at, for example, 24 fps.

2. Transfer procedure The ADPCM data is ready for transfer after resampling by the resampler RS.
Transfer procedure 4. The JPEG data is archived in units of groups for each predetermined frame image by the archiver AV after resampling by the resampler RS. For the sake of easy understanding, for example, when the time slot interval is 8 seconds and the frame rate is 8 fps,
Archive 1: 1, 9, 17, 25, 33, 41, 49, 57
Archive 2: 5, 13, 21, 29, 37, 45, 53, 61
Archive 3: 3, 11, 19, 27, 35, 43, 51, 59
Archive 4: 7, 15, 23, 31, 39, 47, 55, 63
Archive 5: 2, 10, 18, 26, 34, 42, 50, 58
Archive 6: 6, 14, 22, 30, 38, 46, 54, 62
Archive 7: 4, 12, 20, 28, 36, 44, 52, 60
Archive 8: 8, 16, 24, 32, 40, 48, 56, 64
As described above, an image archive containing a predetermined number of frame images (JPEG data) is created so that the frame images to be displayed during one time slot are transmitted on average as much as possible. A number from 1 to 64 in each image archive is a unique number (frame number) given to each frame image, and a frame image in which the frame image with frame number 1 is displayed at the beginning of this time slot period. In this case, display is sequentially performed in the order of frame numbers according to the frame rate.

4. Transfer procedure A reproduction instruction is generated according to the state of each data prepared for transfer. The contents described in the reproduction instruction will be described later.
Transfer procedure 6. The data transfer order is in the order of reproduction instruction, ADPCM data, archive 1, archive 2,..., Archive N for each time slot. Explaining the transfer of ADPCM data, the distribution server HS stores in advance a plurality of types of ringing melody file templates (substantially empty) corresponding to the model of the user terminal MT, and the model of the user terminal MT. The ADPCM data is embedded in the template of the format corresponding to, and the packet is transmitted as an incoming melody file. On the other hand, for image archiving, packets are transmitted in units of archives.
Transfer Procedure 7.1 An image archive that could not be transferred within a time slot is interrupted at that point. In the archiving example described above, smooth video playback without frame skipping (corresponding to a frame rate of 8 fps) is possible if it can be transmitted up to archive 8 in one time slot, and video playback of one frame (frame rate is possible if it can be transmitted up to archive 4). 4 frames per movie) (up to archive 2), 4 frames of video playback (equivalent to a frame rate of 2 fps), and 8 frames of video playback (equivalent to a frame rate of 1 fps) if only archive 1 is used.
Transfer procedure 8. When the end of the data to be reproduced is reached, a null data reproduction instruction (referred to as a null instruction) is transmitted.

It should be noted that how to group a plurality of frame images corresponding to one time slot period into an archive is defined in advance, and the distribution server HS stores a table or the like that defines the frame image included in each archive. Is stored in the storage means. Further, in the above-described embodiment, an example in which 8 frames are simply stored in 8 archives has been shown for easy understanding of the description. The method of creating an image archive in the distribution server HS at the time of data transfer is that “the frame images stored in one archive are approximately averaged in time, and the frame rate increases in a pseudo manner every time the image archive is received”. If it satisfies, it will not be specifically limited. For example, as a first mode when archiving a 24 fps moving picture every 8 seconds, the number of frame images per archive may be set to 8, and a total of 24 archives may be created. That is,
Archive 1: 1, 25, 49, 73, 97, 121, 145, 169
Archive 2: 13, 37, 61, 85, 109, 133, 157, 181
Archive 3: 7, 31, 55, 79, 103, 127, 151, 175
Archive 4: 19, 43, 67, 91, 115, 139, 163, 187
Archive 5: 4, 28, 52, 76, 100, 124, 148, 172
Archive 6: 16, 40, 64, 88, 112, 136, 160, 184
Archive 7: 10, 34, 58, 82, 106, 130, 154, 178
Archive 8: 22, 46, 70, 94, 118, 142, 166, 190
Archive 9: 2, 26, 50, 74, 98, 122, 146, 170
Archive 10: 14, 38, 62, 86, 110, 134, 158, 182
Archive 11: 8, 32, 56, 80, 104, 128, 152, 176
Archive 12: 20, 44, 68, 92, 116, 140, 164, 188
Archive 13: 5, 29, 53, 77, 101, 125, 149, 173
Archive 14: 17, 41, 65, 89, 113, 137, 161, 185
Archive 15: 11, 35, 59, 83, 107, 131, 155, 179
Archive 16: 23, 47, 71, 95, 119, 143, 167, 191
Archive 17: 3, 27, 51, 75, 99, 123, 147, 171
Archive 18: 15, 39, 63, 87, 111, 135, 159, 183
Archive 19: 9, 33, 57, 81, 105, 129, 153, 177
Archive 20: 21, 45, 69, 93, 117, 141, 165, 189
Archive 21: 6, 30, 54, 78, 102, 126, 150, 174
Archive 22: 18, 42, 66, 90, 114, 138, 162, 186
Archive 23: 12, 36, 60, 84, 108, 132, 156, 180
Archive 24: 24, 48, 72, 96, 120, 144, 168, 192
In this case, video playback is performed every time an image archive is received 1 *, 2 *, 3, 4 *, 5, 6, 7, 8 *, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24 * fps (however, except for * is an approximate number), the frame rate is increased in a pseudo manner.

As a second mode when archiving a moving image of 24 fps every 8 seconds, the number of frame images per archive may be 24, and all 8 archives may be created. Archive 1: 1, 9, 17, 25, 33, 41, 49, 57, 65, 73, 81, 89, 97, 105, 113,
121, 129, 137, 145, 153, 161, 169, 177, 185 Archive 2: 5, 13, 21, 29, 37, 45, 53, 61, 69, 77, 85, 93, 101, 109, 117,
125, 133, 141, 149, 157, 165, 173, 181, 189
Archive 3: 3, 11, 19, 27, 35, 43, 51, 59, 67, 75, 83, 91, 99, 107, 115,
123, 131, 139, 147, 155, 163, 171, 179, 187
Archive 4: 7, 15, 23, 31, 39, 47, 55, 63, 71, 79, 87, 95, 103, 111, 119,
127, 135, 143, 151, 159, 167, 175, 183, 191
Archive 5: 2, 10, 18, 26, 34, 42, 50, 58, 66, 74, 82, 90, 98, 106,114,
122, 130, 138, 146, 154, 162, 170, 178, 186
Archive 6: 6, 14, 22, 30, 38, 46, 54, 62, 70, 78, 86, 94, 102,110, 118,
126, 134, 142, 150, 158, 166, 174, 182, 190
Archive 7: 4, 12, 20, 28, 36, 44, 52, 60, 68, 76, 84, 92, 100, 108, 116,
124, 132, 140, 148, 156, 164, 172, 180, 188
Archive 8: 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120,
128, 136, 144, 152, 160, 168, 176, 184, 192
In this case, the video is played back in a pseudo frame every 3 *, 6 *, 9, 12 *, 15, 18, 21, 24 * fps (however, except for *) every time an archive is received. The rate goes up.
As described above, various aspects can be considered as a method of creating an image archive in the distribution server HS.

  The time slot interval is not limited to 8 seconds. For example, assuming that the effective communication speed between the user terminal MT and the distribution server HS is 192 kbps, and the upper limit of the data size that can be downloaded per transfer is 100 kB, nine pieces of data with a data size of 100 kB are downloaded. Takes approximately 4.7 seconds. Furthermore, considering the loss when performing packet communication and the margin when the amount of data increases, it is appropriate to set the time slot interval to about 8 seconds as shown in the above embodiment. That is, the time slot interval is arbitrarily set according to the mounting environment.

  At the start of data (converted data) transfer from the distribution server HS, the user terminal MT sequentially reproduces the transferred data and erases the reproduced data from the memory (pseudo stream reproduction). Therefore, a transfer data reproduction procedure in the user terminal MT will be described with reference to FIG. FIG. 5 is a functional block diagram showing a transfer data reproduction procedure. The data reproduction procedures a to g are shown below.

Reproduction procedure a. From the distribution server HS, packet data is sequentially transmitted in order of reproduction instruction, ADPCM data (incoming melody file), archive 1,..., Archive N for each time slot, and these are stored in the reception buffer RB.
Reproduction procedure b. The control means (CPU) reads the data stored in the reception buffer RB on a first-come-first-served basis.

Reproduction procedure c. During one time slot, the contents of the register RG, the audio buffer OB, the image buffer GB, and the video buffer VB are updated in the order of (1) to (5) shown below.
(1) The content of the reproduction instruction is held in the register RG. Also, in response to the reception of the reproduction instruction, the timer TM for counting time is initialized (cleared to 0). In the reproduction instruction, the following information is described in a text format, for example. Time slot interval (usually fixed: 8000 msec, for example). Image data frame rate fps (normally fixed: 24 fps, but 0 fps for still images only). Number N of image archives to be transmitted (received) in one time slot (default value: 8, may be changed according to the communication state). ADPCM data sampling cycle fs (can be omitted: it can also be included in the incoming melody file). Bit rate of ADPCM data bps (However, it can be omitted: it can also be included in the incoming melody file).
(2) A read cycle for controlling data reading from the audio buffer OB / image buffer GB and an interrupt cycle for controlling data fetch from the reception buffer RB are set according to the contents of the register RG.
(3) The received ADPCM data is taken into the audio buffer OB.
(4) The received image archive is stored in the image buffer GB while being sequentially expanded by the data expansion unit UP. The image buffer GB is divided into storage areas corresponding to the frame rate (fps) × the number of time slots, and the developed JPEG data is discretely stored in the area corresponding to the frame position. At this time, the counter CN counts the number of developed (received) image archives.
(5) According to the set read cycle, ADPCM data is read from the audio buffer OB, JPEG data is read from the image buffer GB, ADPCM data is read to the D / A converter DA, and JPEG data is displayed on the display device (for example, LCD) DP video. Transfer each to the buffer VB (drawing RAM). Here, the JPEG data stored in the image buffer GB cannot be completely received at the time of data reception by the user terminal MT due to the state of communication between the user terminal MT and the distribution server HS. In some cases, the tooth is stored in a missing tooth state. In that case, it is controlled to skip reading of unstored JPEG data and keep the previous valid JPEG data in the video buffer VB.

Reproduction procedure d. When the end of one time slot is reached, the contents of the audio buffer OB, the image buffer GB, and the counter CN are each cleared to zero.
Reproduction procedure e. When starting playback of the first time slot, data reading from each buffer is delayed until sufficient data is accumulated in the audio buffer OB and the image buffer GB.
Reproduction procedure f. When each time slot reaches the end, the quality changing unit QH has developed the number of image archives (counted by the counter CN) and the number of image archives to be transmitted (received) described in the playback instruction ( N) and a quality change request according to the comparison result is notified to the distribution server HS. For example, if (number of developed image archives / number of image archives to be transmitted (received)) falls below 0.8, a request is made to reduce the quality of one stage.
Reproduction procedure g. The end of reproduction of the transfer data is identified by whether or not the reproduction instruction is a “null instruction”.

  Next, a procedure for changing the quality during data transfer according to the communication state between the user terminal MT and the distribution server HS will be described with reference to FIG. FIG. 6 is a schematic diagram for explaining the quality change procedure.

  As shown in FIG. 6, the distribution server HS has a session management table as internal information. The session management table holds information on each session that is transferring data to a client interconnected with the distribution server HS (that is, a record is added to the session management table at the start of transfer, and the corresponding record is stored at the end of transfer). Deleted from the session management table). The session management table includes a user terminal MT address (cliant address), a URL requested by the user terminal MT (request URL), a status (STATUS: model information of the user terminal, a corresponding incoming melody file format, time slot setting, A set of transfer quality of audio and image data to be transferred is stored as one record. When the distribution server HS receives a URL request from the user terminal MT, the distribution server HS first refers to the session management table and searches for a record that matches the address of the user terminal MT that has issued the URL request. If there is no record that matches the address of the same user terminal MT, a record consisting of “address of user terminal MT, requested URL, default quality” is added to the session management table.

  On the other hand, if there is a record that matches the address of the same user terminal MT, it is further checked whether or not the request URLs match. If the request URLs do not match, the previous session record is deleted, the transfer process is terminated, and a new session is started. If the request URLs match, it is regarded as a transfer quality change request, and processing related to transfer quality change is executed. In this case, the transfer quality requested from the user terminal MT to the distribution server HS is added as a URL parameter at the end of the requested URL (Note that if there is no URL parameter or the parameter value is unknown, distribution is performed. The server HS does not need to perform processing relating to the transfer quality change by ignoring the parameters). When receiving the URL request from the user terminal MT, the distribution server HS extracts the URL parameter to update the status of the session management table and changes the transfer quality in the transfer process. Note that the session management table is not in the table format as shown in FIG. 6, and information may be held in a predetermined temporary storage area of the distribution server HS as one session and one file.

  FIG. 7 shows an example of transfer quality that can be requested to be changed from the user terminal MT. As can be understood from FIG. 7, the quality value includes H (high quality), M (medium quality), L (low quality), and 0 (silence or still image) for both audio quality and image quality. For example, specific quality values for audio quality are 44khz / 16 kbps (high quality), 22khz / 16 kbps (medium quality), and 11khz / 16 kbps (low quality). These are 240 × 180 pixels / 24 fps (high quality), 176 × 144 pixels / 24 fps (medium quality), 128 × 96 pixels / 24 fps (low quality), and the like. One of the quality values shown in FIG. 7 is notified from the user terminal MT together with the request URL (quality request). The change in the quality of the sound and the image may be performed in any one of the first to third patterns as shown below, for example. The default value is “H” (high quality) for both the transfer quality of ADPCM (audio) data and the transfer quality of JPEG (still image) data. As the first pattern, initial: audio H image H, 1: audio H image M, 2: audio M image M, 3: audio M image L, 4: audio L image L, 5: audio L image 0 Select one from. The first pattern is intended to transmit ADPCM data and JPEG data with substantially the same quality. As the second pattern, initial: audio H image H, 1: audio H image M, 2: audio H image L, 3: audio H image 0, 4: audio M image 0, 5: audio L image 0 Select one from. This second pattern is intended to preferentially transmit ADPCM data with high quality. As the third pattern, initial: audio H image H, 1: audio M image H, 2: audio L image H, 3: audio 0 image H, 4: audio 0 image M, 5: audio 0 image L Select one from. This third pattern is intended to preferentially transmit JPEG data with high quality. At the time of data transfer, data corresponding to these qualities is prepared (by re-sampling the data stored in the media cache MC as described above).

  In the content distribution system shown in FIG. 1, from a user access start to content acquisition is performed according to a known communication procedure such as TCP / IP or HTTP (HTTPS). A series of communication procedures from the start of user access to content acquisition for the attached media attached to the RSS will be described with reference to FIG. FIG. 8 is a sequence diagram illustrating an outline of a communication procedure among the user terminal MT, the distribution server HS, and the content server CS. However, here, regarding the processing after the transfer request (URL transfer request) of the attached media after the RSS feed, the video data (a in the WMV format) which is media data stream-distributed from the content server CS to a general personal computer terminal (a .wmv) will be described as an example for distribution to a mobile phone (user terminal MT). Also, here, the time slot interval is 8 seconds, the frame rate is 24 fps, the number of frame images per archive shown in the second aspect is 24, and a total of 8 image archives are created. The case of transferring is shown. In FIG. 8, only the parameters included in the request transmitted from the user terminal MT are shown.

  First, a “URL transfer request” is transmitted from the user terminal MT to the distribution server HS. For example, the user terminal MT notifies the distribution server HS of the source URL (contserv / a.wmv) of the user-desired video data requesting transfer and the model information (xxx) of the user terminal MT (url = http: //contserv/a.wmv&typ=xxx). Upon receiving the notification, the distribution server HS generates a sound file (incoming melody file) suitable for the user terminal MT. However, if the converted video data has already been stored in the media cache MC, the process jumps to the instruction creation process. If the converted data of the corresponding moving image data is not already stored in the media cache MC, the distribution server HS sends the notified moving image data request to the content server CS (forward). Then, since the stream distribution of the corresponding moving image data is started from the content server CS (accept and stream start), the distribution server HS receives the moving image data as a stream and performs data conversion of the received moving image data in parallel. The converted data is buffered in the media cache MC (buffering, convert, catching). Further, the distribution server HC sends initial data (properties) to the user terminal MT. The initial data (properties) includes copyright information (right holder name, title, etc.), playback time information (original content playback time; can be omitted), other display information (content introduction text, optional hyperlinks, etc.), etc. The user terminal MT performs initial setting based on the initial data. In the initial setting here, settings related to screen display are mainly performed. On the other hand, the setting relating to data reproduction (reproduction setting) is performed based on a reproduction instruction received from the distribution server HS for each time slot as shown in FIG.

  At the first time slot (time = 1), the user terminal MT has a reproduction instruction (manifest), ADPCM data (a.wmv.1.mld), and an image archive (a.wmv.1.1.zip to a. Request transfer in the order of wmv.1.8.zip). Here, the parameters included in the transfer request will be briefly described. “Time” represents the number of timeslots (1 to M: where M is equal to or greater than (the playback time of the entire source / time slot interval)) Smallest integer). “Dat” represents the type of data to be transferred (MANIF: reproduction instruction, MLD: ADPCM data (ring melody file), IMG: image archive). “Seq” represents a sequence number in the time slot of the image archive to be transferred (1 to N: N = 8 in this example). “Qt” represents the transfer quality (for example, when qt = HM is described, the voice quality is high quality and the image quality is medium quality). In this way, a transfer request for the corresponding data is realized by sending a value specifying the number of timeslots and the data type instead of the actual file name.

  The distribution server HS generates and transmits a reproduction instruction to the user terminal MT in response to the reproduction instruction transfer request (url =... & Time = 1 & dat = MANIF & qt = HH) received from the user terminal MT. The user terminal MT performs settings related to data reproduction (reproduction setting) based on the received reproduction instruction. Next, when a transfer request of ADPCM data is made from the user terminal MT (url = ... & time = 1 & dat = MLD & qt = HH), the distribution server HS determines that the performance time is the time slot interval from the ADPCM data stored in the media cache MC. Only the time length is cut out, the cut out ADPCM data is embedded in the incoming melody file, and this is transmitted to the user terminal MT. In response to the end of reception of the extracted ADPCM data, the user terminal MT makes a transfer request for image data. The distribution server HS reads out the corresponding image data from the media cache MC, archives it, and transmits the generated image archive to the user terminal MT. In the first time slot shown here, an example is shown in which all eight image archives generated within that time are sent sequentially (a.wmv.1.1.zip to a.wmv.1.8.zip). ). When a predetermined offset time (for example, a time corresponding to the time slot interval) elapses from the time when all data (files) are received or when the reproduction instruction is transferred, the user terminal MT receives the received data. Playback starts based on As a result, content composed of 8 seconds of audio and 192 (8 archives × 24 frames) frames of still images is reproduced.

  Next, a case where the transfer time has passed the time slot interval before the completion of image archive reception will be described. In FIG. 8 shown here, only the fifth image archive (a.wmv.2.5.zip) could be acquired in the second time slot (time = 2) (transfer request of the sixth image archive ( url = ... & time = 2 & dat = IMG & seq = 6 & qt = HH), but the time slot interval elapses before reception of the sixth image archive (a.wmv.2.6.zip) is completed. In this case, a content composed of audio based on newly extracted ADPCM data (not shown) for 8 seconds and a still image of 120 (5 archives × 24 frames) frames is reproduced. Compared with the content playback at the first time slot, the playback audio is played back in the same way, but the playback image is played back by 72 frames. If the time slot interval elapses during the reception of such data, the transfer quality is changed from the user terminal MT to the distribution server HS when starting the data transfer of the next time slot (third time). Request to do. For example, the image quality is lowered by one level (url = ... & time = 3 & dat = MANIF & qt = HM).

  In the third time slot (time = 3), the distribution server HS receives ADPCM data and image data with quality corresponding to the transfer quality (qt = HM) at the time of the first or second time slot. Is transferred to the user terminal MT according to the same procedure as described above, and the user terminal MT reproduces the content based on the data. When the data to be transferred has reached the end (the 25th time slot: time = 25), the distribution server HS has a reproduction instruction in which all (or part of) the information described in the reproduction instruction is NULL. A certificate (null instruction) is transmitted to the user terminal MT. When the user terminal MT receives a playback instruction (null instruction) containing a null value, the user terminal MT determines that it is the end of the playback data and ends the playback of the content. The above processing is repeatedly executed in parallel with the media data stream reception from the content server CS.

  Here, an example of quality change rules (rules) in the user terminal MT is shown below. For example, when a slight image archive of about 10 to 20% cannot be sent, it is not necessary to change the quality. When the number of image archives that can be actually transmitted is less than 80% of the number of image archives that should be sent, the size of the frame image is reduced. If transmission is not possible even after reducing the image size to the minimum size, the frame rate is lowered (for example, lowered from 24 fps to 8 fps, etc.). If it still fails, the last received frame image is maintained as a still image, or the message “Image cannot be displayed” is displayed and image transfer is stopped. If even audio data file reception fails, playback is paused and the quality of the audio data file that could not be received is reduced and re-requested (in this case, image archive transfer is not requested (qt = * 0)) ). If even the lowest quality audio data file cannot be received, the message “Cannot be received” is displayed and the process is terminated. In short, “send at least audio”. The quality change rules as described above are stored in a programmed manner, for example, and the control for the quality change is performed according to this rule.

The distribution server HS may include the above-mentioned table for designating the frame position corresponding to each image archive in the reproduction instruction, and may transmit the frame image in the image archive corresponding to the frame position. You may give it a name and archive it. In the latter case, for example, in the case of the first aspect described above, in the third archive with time slot = 2, the eight file names included in the image archive “xxx.2.3.zip” are each “xxx.2.7.jpg”. , "Xxx.2.31.jpg", "xxx.2.55.jpg", "xxx.2.79.jpg", "xxx.2.103.jpg", "xxx.2.127.jpg", "xxx.2.151.jpg", " xxx.2.175.jpg ".
The archiving method is not particularly limited. In addition, the image archive may be compressed as a matter of course. The algorithm in that case does not matter as long as it is a reversible compression algorithm (for example, ZIP). Further, each frame image may be archived as an uncompressed bitmap data using an arbitrary reversible compression algorithm.

  According to the communication procedure as described above, the distribution server HS mediates data transfer, and content distribution between the user terminal MT and the content server CS is performed. Therefore, specific processing for distributing contents in accordance with the communication procedure will be described with reference to FIGS. However, in reality, the distribution server HS and the user terminal MT execute each process in parallel while transmitting and receiving predetermined information in accordance with the communication procedure. Here, for easy understanding of the description, the distribution server HS and the user terminal MT The processes will be described separately for the server HS and the user terminal MT.

  FIG. 9 is a flowchart illustrating an example of “content distribution processing” executed by the distribution server HS. The distribution server HS is always in operation, and the control means (CPU) of the distribution server HS operates the distribution server HS as a so-called WWW server by a server process (not shown) according to a control program stored in advance on a hard disk or the like. . The server process communicates with another server or the user terminal MT through a communication interface (see FIG. 2) such as a network card provided in the distribution server HS, and receives a request (HTTP request) from the user terminal MT. One or more programs and / or various data for executing “content distribution processing” (see FIG. 9) or “data transfer processing” (see FIG. 10) described below are read out from a hard disk or the like and predetermined processing is performed. Execute. Various data (requests received from the user terminal MT, session data, cache, etc.) used in the “content distribution process” described below has a buffer BF or storage area in the RAM or hard disk of the distribution server HS. It is provided and stored, held, and updated as appropriate.

  Step S1 extracts the address (client address) of the user terminal MT and other parameters included in the request URL transmitted from the user terminal MT. Other parameters are “Source URL (url)”, “Model information (typ)”, “Time slot (time)”, “Data type (dat)”, “Image archive sequence number (seq)”, “ "Transfer quality (qt)". In step S2, it is determined with reference to the session management table shown in FIG. 6 whether there is a session. A case where the client address of the user terminal MT has already been registered in the session management table is determined as “session present”. If it is determined that “there is a session” (yes in step S2), it is determined whether or not the extracted “source URL” is the same as the “request URL” held in the session management table ( Step S6). If it is determined that the URLs are not the same (No in step S6), the current session data of the client address held in the session management table is discarded (step S7), and the process jumps to step S3. On the other hand, if it is determined that the URLs are the same (Yes in step S6), model information, time slot, quality setting, transmission data, and image archive number are set according to each extracted parameter (step S8). , “Data transfer processing” is executed (step S9). The data transfer process will be described later (see FIG. 10).

  On the other hand, if it is determined in step S2 that “no session”, that is, the client address of the user terminal MT is not registered in the session management table (no in step S2), or the current session data is discarded. In the case (step S7), session data relating to the user terminal MT is newly generated in the session management table (step S3). In step S4, it is determined whether or not the source URL is a URL indicating the media data to be cached. If the URL is not a cache target, that is, if it is a URL indicating data other than media data (No in step S4), a request and response transfer process to the target URL is executed (step S5). If the URL is a cache target, that is, a URL indicating media data (step S4: yes), it is determined whether or not the converted data is already in the media cache MC (step S10). When the converted data is already in the media cache MC (Yes in step S10), reading of data from the media cache MC is started (step S11). The read data is sequentially stored in the buffer BF. If the converted data is not in the media cache MC (no in step S10), the request for the target URL to the content server CS, the conversion start of the response (that is, media data) distributed from the content server CS, and the media cache MC A series of processes such as starting the cache of the converted data and starting reading the converted data cached from the media cache MC are executed (step S12). After executing the process of step S11 or step S12, the process proceeds to step S8, and the requested data is transferred.

  Here, the operation of the “data transfer process” (see step S9 in FIG. 9) executed in the “content distribution process” will be described with reference to FIG. FIG. 10 is a flowchart illustrating an example of “data transfer processing”.

  Step S21 determines whether the converted data being transferred to the user terminal MT has reached the end. When it is determined that the data has reached the end, that is, when the transfer is completed (Yes in step S21), a reproduction instruction (null instruction) whose content is null is generated and transmitted to the user terminal MT ( Step S22). And the session record regarding the said user terminal MT is deleted from a session management table (step S23). On the other hand, if it is determined that the data has not reached the end (no in step S21), the request content based on the request URL transmitted from the user terminal MT is determined, and processing according to the determined request content is executed. (Step S24). If the request content is “model information (typ = xxx)”, the format of the incoming melody file corresponding to the model information is specified (step S25), and initial data (property) is transmitted to the user terminal MT (step S25). S26). When the requested content is “reproduction instruction (dat = MANIF)”, a reproduction instruction is generated based on the session management table and transmitted to the user terminal MT (step S28).

  If the requested content is “ADPCM data (dat = MLD)”, the voice data corresponding to the time length of the time slot interval is cut out (step S29), embedded in the specified ringing melody file, The incoming melody file is transmitted to the user terminal MT (step S30). That is, the voice data is transmitted in a form embedded in the incoming melody file. Since the format of the incoming melody file varies depending on the model of the user terminal MT, the model information of the user terminal MT is acquired before data transmission. When the requested content is “JPEG data (dat = IMG)”, the frame image corresponding to the designated image archive number is read out of the frame images in the time slot (step S31), and the read frame image is read out. After resampling as necessary, a plurality of frame images are archived and transmitted (step S32).

  FIG. 11 is a flowchart illustrating an example of a main process of “RSS reader” which is an application executed on the user terminal MT side. FIG. 12 is a flowchart illustrating an example of sub-processing of the “RSS reader”. FIG. 12A shows a sub process related to reproduction of audio data, and FIG. 12B shows a sub process related to reproduction of image data. These sub-processes are interrupt processes that operate by appropriately interrupting during execution of the main process. As described above, the user terminal MT is a mobile communication terminal, and the control means (CPU) of the user terminal MT follows a program stored in advance in a ROM or a flash memory, and in addition to a call function that is an original function of the terminal, a network by packet communication. Functions such as access to the above content are executed, and communication with other servers, base stations, or other user terminals MT is performed wirelessly or by wire via a communication interface such as a network card.

  The user terminal MT stores one or more control programs and / or various data constituting an application for executing “main processing” (see FIG. 11) and “sub-processing” (see FIG. 12) described below. The control means (CPU) of the user terminal MT, which is held in the flash memory or the like, stores an application generated based on the operation of the user terminal MT by the user using operation means such as various button switches (not shown) and voice input. In response to the execution instruction, the control program is read and execution of “main processing” described below is started.

  As shown in FIG. 11, in the main process, first, the registered RSS list is displayed on the screen by a display device such as an LCD (step S41). That is, the application (RSS reader) holds an RSS list (specifically, a URL indicating RSS) as saved data corresponding to the application, and presents the RSS list to the user as an initial display. In step S42, it is determined what operation event has occurred in response to the operation of the user terminal MT by the user using the input device (operation means). When the operation event is “URL input” generated in response to a direct URL input by the user, the input URL is registered in the RSS list (step S43), and the RSS list display screen being displayed is displayed. Is updated (step S44). If the operation event is “URL selection” that occurs in response to selecting an arbitrary URL from the RSS list, the processing from step S46 is executed. If the operation event is “other” that occurs in response to an operation other than the above, “other processing” corresponding to the operation is executed (step S45). After the process of step S44 or step S45 is completed, the process returns to step S41. However, if the end instruction processing is performed in step S45, the application is ended.

  If the operation event is “URL selection”, the distribution server HS is requested to send a URL transfer request based on the selected URL (step S46). Then, the display screen is switched to the playback / display screen (step S47), initial data is received from the distribution server HS, and initial settings such as display of the initial screen are performed based on this (step S48). In step S49, a request including a URL and a reproduction instruction distribution request is transmitted to the distribution server HS. A step S50 decides whether or not the reproduction instruction received from the distribution server HS is a “null instruction”. When the reproduction instruction is “null instruction” (Yes in step S50), the data reproduction is finished and the display screen is returned from the reproduction screen to the initial screen or the like (step S51), and the process of step S41 is performed. Return.

  On the other hand, when the reproduction instruction is not “null instruction” (no in step S50), the following processes of steps S52 to S62 are executed. In step S52, reproduction setting is performed based on the reproduction instruction received from the distribution server HS. This reproduction setting includes setting of a time slot, a reproduction rate of sound / image, and the like. In step S53, the timer TM is initialized (dT = 0). The timer TM is a timer that counts up time independently of the main process. The timing for managing data reproduction is first for each time slot interval, second for each audio reproduction interval, and third for each display image update interval. However, for the latter two, after the start of reproduction, data reproduction management may be left to a sound source board (chip) or a video board (chip) (not shown). Step S54 requests the distribution server HS to distribute audio data. A step S55 decides whether or not the data reception from the distribution server HS is completed within the time slot. If it is determined that the data reception is not completed within the time slot (no in step S55), the entire reproduction is paused (step S56). On the other hand, when it is determined that the data reception is completed within the time slot (step S55: yes), the received data is expanded (step S57). That is, it is temporarily stored in the audio buffer as reproduction preparation data. The received data itself is discarded after expansion.

  Step S58 determines whether or not the current transfer time dT counted by the timer TM is larger than the time slot interval T. If it is determined that the current transfer time dT is not greater than the time slot interval T (no in step S58), the distribution server HS is requested for the first image data (step S59) and received from the distribution server HS. The developed image data (image archive) is expanded (step S60). That is, it is temporarily stored in the image buffer as reproduction preparation data. The received data itself is discarded after expansion. The processes in step S59 and step S60 are repeated N times corresponding to the number of image archives or for a time slot interval of T seconds, and the remaining image archives are sequentially requested / acquired (request sequence value: update of seq). . The T seconds are measured by the timer TM initialized in step S53. Step S61 starts reproduction of data received from the distribution server HS. However, execution of this process is started by generating a “sub-process” thread that is a timer interrupt process described later only when the first playback or playback is paused. That is, the sub-process to be described later continues to move by itself until a playback end instruction is generated from the main process (until the object is discarded), and thereby data playback is continued. A step S62 updates the request parameter. The parameters to be updated here are time slot: time (add 1), request data type: dat, and quality value: qt. However, it is determined whether or not the quality value is changed for each time slot, and the parameter is updated according to the determination.

  In the sub-process related to the reproduction of the audio data shown in FIG. 12A, the audio data received at the t-th time slot is sequentially reproduced (step S71). That is, the audio data received from the distribution server HS is continuously written in the audio buffer, and continuous audio sampling data is recorded (unless reception of audio data fails due to deterioration of the communication state or the like). In this audio buffer, a storage area having a size capable of storing audio data of a reproduction time corresponding to at least two time slot intervals (16 seconds in this case) is secured, and the audio buffer is recursively read out. By updating the address, the audio data can be reproduced continuously. That is, the time slot interval corresponds to the performance time of one received audio file, and the next received audio data is reproduced when this time has elapsed. At this time, in the main processing, 0 or the address position in the middle of the audio buffer is alternately designated as the writing start address for each time slot. In this way, the reproduction of the audio data in response to the reading from the audio buffer and the writing of the next audio data following the audio data being reproduced to the audio buffer can be performed in parallel. It will be possible to play continuously.

  In the sub-process related to the reproduction of the image data shown in FIG. 12B, the contents of the display buffer are sequentially used for each of the plurality of image data received at the t-th time slot in accordance with the drawing display rate of a display device such as an LCD. (Step S81). That is, the image buffer has a storage area of (number of frame images in one image archive × 2 × time slot interval T), and each frame image in the image archive is arranged in a corresponding storage area. In accordance with the drawing update rate of the display device, a frame image at a corresponding position in the image buffer (a decimal point may appear, in which case it is converted to an integer or interpolation) is displayed. That is, the time slot interval corresponds to a time for sequentially displaying all the plurality of frame images included in one received image archive, and the plurality of frames included in the next received image archive when this time has elapsed. Start displaying images in order. At this time, in the main processing, 0 or the address position in the middle of the image buffer is alternately designated as the write start address for each time slot. In this way, it is possible to simultaneously display the frame image included in the image archive in response to reading from the image buffer and write to the image buffer of the next image archive subsequent to the currently displayed image archive. Therefore, the frame images included in the image archive can be continuously displayed.

  As described above, the distribution server HS that mediates data transfer is provided between the content server CS that feeds RSS (for example, podcasts) and the user terminal MT (mobile communication terminal) that receives the RSS feed, The distribution server HS converts the moving image data provided by the stream distribution from the content server CS into audio data (ADPCM) and still image data (JPEG) in a format reproducible on the user terminal MT, and subdivides them. To the user terminal MT. The user terminal MT can reproduce the moving image in the pseudo stream by reproducing the divided audio data and the still image data while downloading. This makes it possible to receive a sufficient information distribution service using RSS feeds, including video playback, even on mobile communication terminals.

In addition, although the example which converts stream media data into ADPCM data and several JPEG data was shown about data conversion, it is not restricted to this. For example, the stream media data may be converted into subdivided MPEG4 or other format data other than those described above. However, the quality of each data after conversion stored in the media cache MC is assumed to be the highest (for example, audio 44khz, 16kbps / image 240x180pixel, 24fps) that can be transferred by the distribution server HS. Regarding data transfer, audio quality and image quality at the time of transfer are not limited to those illustrated. For example, the number of frame images may be 30 fps. In addition, a plurality of types of quality data may be prepared in the data conversion process for one piece of data corresponding to the transfer controlled quality. Furthermore, when storing the converted data in the media cache MC, multiple quality data may be generated in parallel, or the data once stored in the media cache MC with the default quality may be stored in multiple types of quality. The data may be reconverted and stored.
Note that the frame image is not limited to archiving and data transfer is performed, but audio data (ADPCM) may be archived and data transferred.

  When the client application (RSS reader) is operating, the reception of all data files that should be received during the time slot may be completed within one time slot. ) In such a case, it may be possible to wait for the transmission request for data to be received at the next time slot until the start timing of the next time slot arrives, or to receive regardless of the start timing of the time slot. Upon completion, a transfer request for data to be received at the next time slot may be made. In the latter case, it is necessary to separately manage and control at which time slot the received data was to be received and the timing at which each received data starts to be developed in the audio buffer and the image buffer. In the former case, since a series of processing is repeated for each time slot, the received data may be developed sequentially (at a predetermined writing position). When the quality designation parameter is “qt = 0 *” or “qt = * 0”, the transfer request itself for the corresponding audio data or image data may be skipped.

  If the quality is changed during data transfer / data reproduction, the audio sampling values to be reproduced may be discontinuous. Therefore, on the user terminal MT side, when reproducing the data in the audio buffer, the sample value smoothly transitions between the data by crossfading or interpolating in the vicinity of the data corresponding to the file connection section. You may make it control to. Similarly, if reception of a small file does not arrive in time, the current playback is paused and force dumped, and when the next small file arrives, the playback pointer is returned to the force dumped address, and from there Playback may be resumed (and the volume may be increased smoothly). The quality change at the time of data transfer / data reproduction is not limited to low quality, but may be high quality. For example, when reception is completed faster than the time slot interval elapses, the user terminal MT requests the distribution server HS to change the transfer quality to high quality when starting the next time slot. May be. This is advantageous in that the quality of the data can be restored so that the high-quality data is transferred again in accordance with the return of the communication state even if the quality is changed to a low quality as the communication state deteriorates.

  In the above-described embodiment, “moving image data with an audio track” has been described as an example. However, an image transmission request does not have to be performed when only audio is transmitted, and an audio transmission request does not have to be performed when only moving images are generated. If the data is only a still image, the parameter “qt = * 0” may be requested in the first time slot. Note that the combination of data to be received (to be requested) may be determined by the extension of the URL (wmv, mp3, swf, etc.) on the user terminal MT side, or the distribution server HS may When converting and caching data, determine the media data combination, what kind of data combination is in the initial data (profile) or playback instruction (manifest), and what request should be sent You may make it describe the information for identifying.

The distribution server HS has been described as mediating content stored in another server (content server CS). However, the distribution server HS itself may be a content server CS that stores content. In this case, data corresponding to the converted data may be stored in advance in the media cache MC from the beginning, or data conversion / cache may be performed before distribution. The parameter “typ” indicating the model of the user terminal MT has been described as being sent only at the beginning. However, every time a request is made, “typ” is designated, and the distribution server HS responds to each request with a corresponding template file. (Incoming melody file) may be selected to embed voice data. The distribution server HS has been described as embedding voice data in the template file of the incoming melody file corresponding to the model of the user terminal MT. However, when the distribution server HS returns the voice data to the user terminal MT as a response, the media type The ringtone melody format corresponding to the model of the user terminal MT may be designated, or voice data may be added to the standard ringtone melody file and then converted to the ringtone melody file corresponding to the model. .

  Each process described above is performed by a control unit (not shown) in each apparatus executing a predetermined program (software). Of course, the present invention can be implemented not only in the form of computer software but also in the form of a microprogram processed by a DSP (digital signal processor), and is not limited to the form of this type of program. You may implement in the form of a dedicated hardware device comprised including a discrete circuit, an integrated circuit, a large-scale integrated circuit, or a gate array.

1 is a system block diagram showing an overall configuration of an embodiment of a content distribution system to which a client device and a distribution server device according to the present invention are applied. It is a hardware block diagram which shows one Example of the whole structure of either the user terminal which comprises a content delivery system, or a server apparatus. It is a functional block diagram in the data conversion part for demonstrating the conversion procedure of data, Fig.3 (a) shows about audio | voice data, FIG.3 (b) shows about moving image data, respectively. It is a functional block diagram of the transmission control part for demonstrating data transfer control. It is a functional block diagram which shows the reproduction | regeneration procedure of transfer data. It is a schematic diagram for demonstrating the outline | summary of a quality change. An example of the transfer quality which can request | require a change from a user terminal is shown. It is a sequence diagram which shows the communication procedure between the user terminal which comprises the content delivery system shown in FIG. 1, a delivery server, and a content server. It is a flowchart which shows one Example of a content delivery process. It is a flowchart which shows one Example of a data transfer process. It is a flowchart which shows one Example of the main process of an RSS reader. FIGS. 12A and 12B are flowcharts showing an embodiment of the RSS reader sub-process, in which FIG. 12A shows audio data playback processing, and FIG. 12B shows image data playback processing.

Explanation of symbols

MT: user terminal (client device) HS: distribution server, CS: content server, C: cache inspection unit, U: URL access unit, H: transmission control unit, D: data conversion unit, MC: media cache, PV: provider , DC ... decoder, BF ... buffer, RS ... resampler, AV ... archiver, SL ... selector, RB ... receive buffer, RG ... register, TM ... timer, OB ... audio buffer, GB ... image buffer, DA ... D / A Conversion unit, SP ... Amplifier / speaker, VB ... Video buffer, DP ... LCD (display), CN ... Counter, QH ... Quality changing unit, 1 ... CPU, 2 ... ROM, 3 ... RAM, 4 ... Input device, 5 ... Output device, 6 ... communication interface, 7 ... external storage device

Claims (8)

A distribution server device interconnected to a client device and a content server via a communication network and mediating data distribution between the devices,
Data receiving means for receiving media data for video playback, streamed from the content server;
Conversion means for converting and storing audio data and moving image data included in the received media data into audio data in a predetermined format and a plurality of still image data in a predetermined format in units of frames;
Generating means for generating reproduction instruction data defining at least a predetermined data transfer period and a reproduction bit rate of audio data and / or a reproduction frame rate of image data;
Dividing means for dividing the converted audio data for each reproduction time length that matches the data transfer period;
Image frame data having a reproduction time length that matches the data transfer cycle, and the plurality of converted still image data are transmitted after being substantially averaged over time for each reproduction time that matches the data transfer cycle. , Archive generation means for generating a plurality of image archives obtained by averaging the still image data in terms of time, and packaging,
For each data transfer cycle, in response to a request from the client device, the generated reproduction instruction data, the divided audio data, and the generated plurality of image archives in the order of the generated image archives. A distribution server device comprising transmission means for transmitting data. A determination unit for determining whether audio data and still image data corresponding to the media data requested from the client device are already stored in a predetermined storage unit;
In accordance with the determination, the conversion means is stream-distributed from a content server connected via a communication network when audio data and still image data corresponding to the media data are not stored in the storage means. Receiving media data for reproducing a moving image, converting the received media data into audio data of a predetermined format and a plurality of still image data of a predetermined format, and storing the data in the storage means. Item 2. The distribution server device according to Item 1. A client device interconnected to a distribution server device via a communication network,
Playback instruction data defining at least a predetermined data transfer cycle and a playback bit rate of audio data and / or a playback frame rate of image data, audio data corresponding to a playback time length matching the data transfer cycle, and the data Means for requesting transfer to the distribution server device to repeatedly transfer data in order of one or a plurality of image archives composed of image frame data having a playback time length that matches a transfer cycle;
In response to the transfer request, receiving means for receiving each data from the distribution server device, setting means for setting a data transfer cycle for repeatedly performing the transfer request according to the received reproduction instruction data,
Reproducing means for reproducing the received audio data for each set data transfer cycle;
Display that sequentially expands the received one or a plurality of image archives for each set data transfer cycle, and sequentially displays frame images included in the expanded image archives at time positions corresponding to the respective frame images. The display by the frame image included in the image archive that could not be received in the data transfer period is replaced by maintaining the display of the frame image included in the received image archive arranged at the previous time position. A client device comprising what is displayed. A storage unit that stores a plurality of URLs associated with RSS fed from an appropriate content server; and a request unit that requests an RSS feed based on the URL from the distribution server device.
4. The client device according to claim 3, wherein a response from the distribution server device in response to the request is the reproduction instruction data, audio data, or an image archive. On the computer,
A procedure for receiving media data for video playback, which is streamed from a content server connected via a predetermined communication interface;
A procedure of converting audio data and moving image data included in the received media data into audio data in a predetermined format and a plurality of still image data in a predetermined format in units of frames and storing them in a predetermined storage unit;
A procedure for generating reproduction instruction data defining at least a predetermined data transfer period and a reproduction bit rate of audio data and / or a reproduction frame rate of image data;
Dividing the converted audio data for each reproduction time length that matches the data transfer period;
Image frame data having a reproduction time length that matches the data transfer cycle, and the plurality of converted still image data are transmitted after being substantially averaged over time for each reproduction time that matches the data transfer cycle. A procedure for generating a plurality of packaged image archives by substantially averaging still image data stored in the storage means in terms of time;
For each data transfer cycle, in response to a request from the client device, the generated reproduction instruction data, the divided audio data, and the generated plurality of image archives in the order of the generated image archives. A program for executing a procedure for transmitting data via a predetermined communication interface. On the computer,
A procedure for determining whether audio data and still image data corresponding to media data requested from a client device connected via a predetermined communication interface are already stored in a predetermined storage unit;
According to the determination, when audio data and still image data corresponding to the media data are not stored in the storage unit, the video data is streamed from a content server connected via a communication network. Media data is received, audio data and video data included in the received media data are converted into audio data in a predetermined format and a plurality of still image data in a predetermined format in units of frames, and stored in the storage means And the steps to
A procedure for generating reproduction instruction data defining at least a predetermined data transfer period and a reproduction bit rate of audio data and / or a reproduction frame rate of image data;
Dividing the converted audio data for each reproduction time length that matches the data transfer period;
Image frame data having a reproduction time length that matches the data transfer cycle, and the plurality of converted still image data are transmitted after being substantially averaged over time for each reproduction time that matches the data transfer cycle. A procedure for generating a plurality of packaged image archives by substantially averaging still image data stored in the storage means in terms of time;
For each data transfer cycle, in response to a request from the client device, the generated reproduction instruction data, the divided audio data, and the generated plurality of image archives in the order of the generated image archives. A program for executing a procedure for transmitting data via a predetermined communication interface. On the computer,
Playback instruction data defining at least a predetermined data transfer cycle and a playback bit rate of audio data and / or a playback frame rate of image data, audio data corresponding to a playback time length matching the data transfer cycle, and the data A procedure for making a transfer request to a distribution server device connected via a predetermined communication interface so as to repeatedly transfer data in the order of one or a plurality of image archives composed of image frame data having a playback time length that matches a transfer cycle; ,
In response to the transfer request, a procedure for receiving the data from the distribution server device via the communication interface;
In accordance with the received reproduction instruction data, a procedure for setting a data transfer cycle for repeatedly performing the transfer request;
Replaying the received audio data for each of the set data transfer cycles;
For each set data transfer period, the received one or more image archives are sequentially expanded, and frame images included in the expanded image archives are arranged at time positions corresponding to the respective frame images, and sequentially specified. When displaying on the display means, the display of the frame image included in the received image archive arranged at the previous time position is maintained as the display by the frame image included in the image archive that could not be received in the data transfer period. Program to execute the procedure to display instead. On the computer,
A procedure of storing a plurality of URLs associated with RSS fed from an appropriate content server connected via a predetermined communication interface in a predetermined storage unit;
A procedure for requesting an RSS feed based on the URL to a distribution server device connected via the communication interface;
In response to the request, the playback instruction data that at least defines a predetermined data transfer cycle and a playback bit rate of audio data and / or a playback frame rate of image data, and a playback time length that matches the data transfer cycle To a distribution server device connected via the communication interface so as to repeatedly transfer data in the order of one or a plurality of image archives composed of audio data to be played and image frame data having a playback time length matching the data transfer cycle. A procedure to request transfer,
In response to the transfer request, a procedure for receiving the data from the distribution server device via the communication interface;
In accordance with the received reproduction instruction data, a procedure for setting a data transfer cycle for repeatedly performing the transfer request;
Replaying the received audio data for each of the set data transfer cycles;
For each set data transfer cycle, the received one or more image archives are sequentially developed, and frame images included in the developed image archives are arranged at time positions corresponding to the respective frame images, and predetermined display means When displaying in sequence, the display by the frame image included in the image archive that could not be received in the data transfer period is maintained as the display of the frame image included in the received image archive arranged at the previous time position. Program for executing the procedure to display instead of.

Images