JP2019114941A - Reproducer, remote reproduction system, reproduction method, and computer program - Google Patents

Abstract

To provide a reproducer, a remote reproduction system, a reproduction method and a computer program, capable of restraining deviation between video and voice, while making low traffic and high picture quality compatible.SOLUTION: A reproducer includes a communication unit, a first video reproduction section, a second video reproduction section, and a sound reproduction section. The communication unit receives voice data added with audio time information, finite difference data between two frames that go back and forth in time, and moving image data added with moving image time entry. The first video reproduction section reproduces the video by the later frame generated based on the finite difference data, and the former frame out of the two frames. The second video reproduction section reproduces the video by multiple frames generated based on the moving image data and continuous in time. The sound reproduction section reproduces sound on the basis of the voice data, and synchronizes the sound with the video on the basis of moving image time entry and audio time information, when the second video reproduction section is reproducing the video.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a playback apparatus for playing back video and audio distributed by streaming, a remote playback system, a playback method, and a computer program for causing a computer to function as a playback apparatus.

  For example, in a school class, a remote reproduction system is known that remotely reproduces (mirrors) the same video and audio output to a terminal device used by a teacher on a large display device installed in a classroom. (E.g., Patent Document 1). In such a remote reproduction system, the terminal apparatus takes in the video and audio output from the terminal apparatus and generates video data and audio data, and transmits these data by wireless communication, for example. A reproduction device such as an access point of a wireless LAN (Local Area Network) receives and reproduces these data, and video and audio are output to a display device connected to the reproduction device.

JP, 2016-213541, A

  By the way, in order to balance low traffic and high image quality, video reproduction processing with low image quality but small amount of data processed per unit time, and high quality image with large amount of data processed per unit time It is conceivable to switch between playback processing. However, in such switching of video reproduction processing, the latter is more expensive than the former, and the delay time from the reception of data to the reproduction of video is large. On the other hand, there is a problem that when switching from the former to the latter, the video is delayed relative to the audio because the load of reproducing the audio is low and the delay time for reproduction is short.

  The present invention has been made in view of such circumstances, and its main object is to provide a reproduction apparatus, a remote reproduction system, a reproduction method, and a computer program which can solve the above-mentioned problems.

  In order to solve the problems described above, the reproduction device according to one aspect of the present invention is provided with audio data to which audio time information is added, differential data between two frames which are temporally adjacent to each other, and moving picture time information. A communication unit for receiving moving image data, a first video reproduction unit for reproducing an image by a frame generated based on the difference data, and a previous frame of the two frames, and the moving image data A second video reproduction unit that reproduces the video by a plurality of temporally consecutive frames generated based on the second video reproduction unit, and reproduces the audio based on the audio data, and the second video reproduction unit reproduces the video In this case, the apparatus further includes an audio reproduction unit that synchronizes the audio with the video based on the moving image time information and the audio time information.

  In addition, the remote reproduction system according to another aspect of the present invention transmits audio data to which audio time information is added, differential data between two frames before and after in time, and moving image data to which moving image time information is added. A content transmission device and the reproduction device of the above aspect are provided.

  The reproduction method according to another aspect of the present invention receives the audio data to which audio time information is added and the differential data indicating the differential between two temporally consecutive frames and receives the differential data. The video is reproduced by the frame after being generated based on the previous frame of the two frames, and the audio is reproduced simultaneously with the video based on the received audio data, and the difference data When switching from video to video by moving image data for generating a plurality of temporally consecutive frames, audio data to which audio time information is added and moving image data to which moving image time information is added is received, The reproduction of the image by the difference data is stopped, and the image is reproduced by a plurality of temporally consecutive frames generated based on the received moving image data, and received. It reproduces the sound based on the voice data, synchronizing the sound based on the video time information and the audio time information on the video.

  Further, a computer program according to another aspect of the present invention is a computer program for causing a computer to function as a content transmitting device for transmitting content including video and audio, and is a difference between two frames which are temporally back and forth in time. Video transmission means for transmitting differential data indicating video, second video transmission means for transmitting video data to which video time information is added, audio transmission means for transmitting audio data to which audio time information is added, and the difference When switching video reproduction from one of the data and the moving image data to the other, the computer is functioned as switching means for switching an execution target from one of the first video transmission means and the second video transmission means to the other.

  According to the present invention, it is possible to suppress the deviation between video and audio while achieving both low communication amount and high image quality.

BRIEF DESCRIPTION OF THE DRAWINGS The schematic diagram which shows the structure of the remote reproduction | regeneration system which concerns on embodiment. FIG. 1 is a block diagram showing a configuration of a content transmission apparatus according to an embodiment. FIG. 2 is a functional block diagram showing functions of the content transmission apparatus according to the embodiment. FIG. 2 is a block diagram showing the configuration of a reproduction device according to the embodiment. FIG. 1 is a block diagram showing a configuration of a display device according to an embodiment. FIG. 2 is a functional block diagram showing the function of the playback device in the embodiment. The flowchart which shows the procedure of the remote reproduction | regeneration of the content by the remote reproduction system which concerns on embodiment (the 1). The flowchart which shows the procedure of the remote reproduction | regeneration of the content by the remote reproduction system which concerns on embodiment (the 2). The flowchart which shows the procedure of the remote reproduction | regeneration of the content by the remote reproduction system which concerns on embodiment (the 3). The flowchart which shows the procedure of the remote reproduction | regeneration of the content by the remote reproduction system which concerns on embodiment (the 4). The figure for demonstrating the production | generation process of difference data. 5 is a graph showing changes in the first transmittance α1 and the second transmittance α2 when switching from the difference mode to the moving image mode. The graph which shows change of sound output level SL in, when changing from difference mode to animation mode. The graph which shows change of the 1st transmissivity alpha 1 and the 2nd transmissivity alpha 2 in the case of changing from animation mode to difference mode.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings. Each embodiment shown below is an example of a method and apparatus for embodying the technical idea of the present invention, and the technical idea of the present invention is not limited to the following. Absent. The technical concept of the present invention can be variously modified within the technical scope described in the claims.

  In this embodiment, a remote reproduction system will be described which remotely reproduces (mirrors) the same video and audio as the video and audio (content) output from the content transmission apparatus which is a terminal apparatus used by a teacher.

[Configuration of remote playback system]
FIG. 1 is a schematic view showing the configuration of the remote reproduction system according to the present embodiment. The remote reproduction system according to the present embodiment is installed in, for example, a school, a company building, a hospital, a public facility such as a public hall, and the like. In the present embodiment, an example in which a remote reproduction system is installed in a school will be described. As shown in FIG. 1, the remote reproduction system 100 includes a content transmission device 200, a reproduction device 300, and a display device 400. The playback device 300 is installed in a classroom. The content transmission device 200 is a tablet and is used by a teacher. The content transmission apparatus 200 can perform wireless communication according to the wireless LAN standard IEEE 802.11a / b / g / n / ac, and the reproduction apparatus 300 is an access point capable of wireless communication according to the wireless LAN standard. A wireless LAN 450 is configured by the reproduction device 300 in each classroom and the principal room. Also, the playback apparatus 300 is connected to a wired LAN 460 and the Internet (not shown) laid across each classroom and other rooms (such as a principal's room and a staff room), and is connected to the wired LAN 460 and the Internet. Communication is possible with the device.

  The content transmission device 200 may be a laptop personal computer, a smartphone, a mobile phone, or the like as long as wireless communication by wireless LAN is possible.

  The configuration of the content transmission device 200 will be described. FIG. 2 is a block diagram showing the configuration of the content transmission apparatus 200. As shown in FIG. The content transmission apparatus 200 includes a wireless communication unit 201, a display unit 202, a speaker 203, an input unit 204, a storage unit 205, and a control unit 206.

  The wireless communication unit 201 enables wireless communication according to IEEE 802.11a / b / g / n / ac, and includes an antenna 207 for wireless communication. The display unit 202 is configured of an LCD (Liquid Crystal Display) or an organic EL (Electro Luminescence). In addition, the input unit 204 is a position input device such as a pressure-sensitive or capacitive-type transparent touch pad. The input unit 204 is attached to the surface of the display unit 202, and the display unit 202 and the input unit 204 are configured as an integrated touch panel. The storage unit 205 is a non-volatile storage device such as a flash memory or a hard disk, and stores a content transmission program 210 which is a computer program for content transmission, and an educational program 230 such as video reproduction software and presentation software. . The control unit 206 includes a CPU 261 and a memory 262, and controls the above-described units.

  When the CPU 261 executes the content transmission program 210, the tablet functions as the content transmission device 200, and causes the content transmission device 200 to realize an operation as described later.

  FIG. 3 is a functional block diagram showing the functions of content transmitting apparatus 200 according to the present embodiment. As shown in FIG. 3, the content transmitting apparatus 200 includes a communication unit 211, a display unit 212, a speaker 213, a content reproduction unit 214, a video capture unit 215, an audio capture unit 216, and a change amount detection. It has the functions of the unit 217, the input detection unit 218, the switching instruction receiving unit 219, the transmission target switching unit 220, and the switching request unit 221.

  The communication unit 211 is the above-described wireless communication unit 201, and transmits data of content to the reproduction device 300. Specifically, the communication unit 211 selectively transmits difference data indicating a difference between two temporally successive frames and moving image data for generating a plurality of temporally consecutive frames. Further, the communication unit 211 transmits audio data at the same time as the difference data and the moving image data described above. Further, to each of the moving image data and the audio data, each of moving image time information and audio time information which are time stamps is added.

  The display unit 212 is the display unit 202 described above, and the speaker 213 is the speaker 203 described above. The content reproduction unit 214 reproduces content such as a moving image, presentation material, and audio, and the display unit 212 outputs video and the speaker 213 outputs audio.

  The image capture unit 215 captures an image displayed on the display unit 212 and generates image data. The video capture unit 215 has a function of generating difference data of two frames as video data, and a function of generating moving image data obtained by encoding a plurality of temporally consecutive frames by a predetermined moving image compression method. The audio capture unit 216 captures the audio output from the speaker 213 and generates audio data in the linear PCM format.

  The difference data or the moving image data generated by the video capture unit 215 and the audio data generated by the audio capture unit are transmitted by the communication unit 211. When the reproduction device 300 reproduces these contents, the display device 400 displays an image and outputs an audio. Further, in the display device 400, as described later, input by a stylus or a finger is possible, and input information given by the user is transmitted from the reproduction device 300. The communication unit 211 transmits the difference data or the moving image data and the audio data as described above, and receives the input information.

  The change amount detection unit 217 detects a temporal change amount of an image between frames, that is, the number of pixels changed from the previous frame. The input detection unit 218 monitors the input information received by the communication unit 211, and detects an input provided by the user with respect to the displayed image. For example, when there is an input to the display device 400 with a stylus or a user's finger, the input detection unit 218 detects this input. When the user performs a specific operation of instructing the display device 400 to switch the video reproduction between the difference data and the moving image data, the switching instruction receiving unit 219 receives the input information received by the communication unit 211. Accept the instructions. In the present embodiment, detection of a predetermined change amount by change amount detection unit 217, detection of a predetermined input by input detection unit 218, and reception of a video switching instruction by switching instruction reception unit 219 are respectively difference data and It is assumed that the video data switching condition. That is, when one of the above switching conditions is satisfied, the transmission target is switched from one of the difference data and the moving image data to the other.

  The switching condition includes a first switching condition which is a condition for switching a transmission target from difference data to moving image data, and a second switching condition which is a condition for switching a transmission target from moving image data to difference data. The first switching condition and the second switching condition will be described later.

  The transmission target switching unit 220 switches the transmission target to the reproduction device 300 from one of the difference data and the moving image data to the other when the switching condition is satisfied. That is, when the content transmission apparatus 200 is transmitting difference data, when the first switching condition is satisfied, the video data generated by the video capture unit 215 is switched from the difference data to the moving image data, and the transmission target is Switches from difference data to moving image data. On the other hand, when the content transmission apparatus 200 is transmitting moving image data, when the second switching condition is satisfied, the video data generated by the image capturing unit 215 is switched from moving image data to difference data, and the transmission target is Switches from moving image data to difference data.

  When the switching condition is satisfied, the switching request unit 220 requests the reproducing device 300 to switch the reception target from one of the difference data and the moving image data to the other. That is, in the case where the content transmission apparatus 200 is transmitting difference data, when the first switching condition is satisfied, the switching request unit 220 causes the communication unit 211 to transmit a switching request for receiving moving image data. On the other hand, in the case where the content transmission apparatus 200 is transmitting moving image data, when the second switching condition is satisfied, the switching request unit 220 causes the communication unit 211 to transmit a switching request for receiving moving image data.

  Here, in the reproduction device 300, in the switching of the image reproduction, the transition reproduction processing is performed in which the image generated by one of the difference data and the moving image data is changed stepwise to the image generated by the other. In this process, both differential data and video data are simultaneously used. Therefore, the communication unit 211 simultaneously transmits difference data and moving image data (each including the same frame) after a switching condition is satisfied, and is used for video reproduction after switching after the switching period has elapsed. Send only the data that Details of the transition reproduction process will be described later.

  Next, the configuration of the playback device 300 will be described. FIG. 3 is a block diagram showing the configuration of the playback apparatus 300. As shown in FIG. As shown in FIG. 4, the playback apparatus 300 includes a wireless communication unit 301, a wired communication unit 302, a control unit 303, an external output unit 304, and an external connection terminal 305.

  The wireless communication unit 301 enables wireless communication according to IEEE 802.11a / b / g / n / ac, and includes an antenna 306 for wireless communication. The wired communication unit 302 is an Ethernet communication interface, and is connected to the wired LAN 460. The control unit 303 includes a CPU 331 and a memory 332, and controls communication by the wireless communication unit 301 and the wired communication unit 302. Further, the memory 332 includes two frame memories of a first frame memory 341 and a second frame memory 342.

  The external output unit 304 is an HDMI (registered trademark, High-Definition Multimedia Interface) terminal, is connected to the display device 400, and can output a video signal and an audio signal to the display device 400 to output video and audio. . The external connection terminal 405 is a USB (Universal Serial Bus) terminal. The external connection terminal 405 is also connected to the display device 400.

  FIG. 5 is a block diagram showing the configuration of the display device 400. As shown in FIG. The display device 400 includes a display unit 401, a speaker 402, an input unit 403, an external input unit 404, an external connection terminal 405, and a control unit 406. The display unit 401 is a large LCD or organic EL panel, and the display unit 401 outputs an image and the speaker 402 outputs an audio so that all members of the class can view the content. The input unit 403 is a position input device such as a pressure-sensitive or capacitive-type transparent touch pad. The input unit 403 is attached to the surface of the display unit 401, and the display unit 401 and the input unit 204 are configured as an integrated touch panel.

  The external input unit 404 is an HDMI terminal, and the external connection terminal 405 is a USB terminal. The external input unit 404 and the external connection terminal 405 are connected to the external output unit 304 and the external connection terminal 305 of the playback apparatus 300, respectively. The external input unit 404 receives a video signal and an audio signal output from the playback apparatus 300, and the external connection terminal 405 transmits an output signal from the input unit 403 to the playback apparatus 300.

  The control unit 406 includes a CPU and a semiconductor circuit or a semiconductor circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA), and has a function of controlling the above-described units. The control unit 406 causes the display unit 401 to display an image based on the video signal and the audio signal input to the external input unit 404, and causes the speaker 402 to output an audio.

  FIG. 6 is a functional block diagram showing the functions of the playback apparatus according to the present embodiment. As shown in FIG. 6, the playback apparatus 300 has functions of a communication unit 311, a first video playback unit 312, a second video playback unit 313, an audio playback unit 314, and a transition playback unit 315.

  The communication unit 311 is the above-described wireless communication unit 301, and receives data of content from the content transmission device 200. Specifically, the communication unit 311 selectively receives difference data indicating a difference between two temporally consecutive frames and moving image data for generating a plurality of temporally consecutive frames. The communication unit 311 also receives audio data at the same time as the above difference data and moving image data. In addition, a time stamp, which is time information, is added to each of the moving image data and the audio data.

  The first video reproduction unit 312 reproduces a video by generating and outputting a subsequent frame based on difference data of two frames and a previous frame of the two frames. The second video reproduction unit 313 generates a plurality of frames based on the moving image data, and reproduces a video by outputting these frames continuously in time. The audio reproduction unit 314 reproduces audio based on the audio data. Here, “reproduction” means processing differential data and moving image data and audio data to generate a video signal and an audio signal, and outputting the generated video signal and audio signal. In the following description, the operation mode for performing video reproduction by the first video reproduction unit 312, that is, video reproduction with difference data is referred to as "difference mode", and video reproduction by the second video reproduction unit 313, that is, video with moving image data. The operation mode for performing reproduction is called "moving image mode".

  When reproducing an image in the differential mode, the audio reproduction unit 314 processes audio data and reproduces audio each time the audio data is received without using time information. On the other hand, when the video is played back in the video mode, the audio playback unit 314 plays back the audio in synchronization with the video based on the time stamp added to the video data and the time stamp added to the audio data. Do. The audio reproduction unit 314 reproduces the audio in each of the difference mode and the moving image mode using the common audio data.

  As described above, when switching from video reproduction by one of the difference data and the moving image data to video reproduction by the other, the content reproduction apparatus 300 transmits a switching request for the reception target. When the communication unit 311 receives the switching request, the communication unit 311 switches the reception target from one of the difference data and the moving image data to the other.

  The content transmission apparatus 200 simultaneously transmits difference data and moving image data for a predetermined switching period after the switching condition is satisfied, and the communication unit 311 receives these simultaneously. The transition reproduction unit 315 executes a transition reproduction process of changing the image generated by one of the simultaneously received differential data and moving image data stepwise to the image generated by the other and reproducing the image. Hereinafter, the operation mode for executing this transition reproduction process is referred to as "transition mode". Specifically, in the transition mode, the transition reproduction unit 315 performs transmission processing based on the first transmittance α1 on a frame generated from difference data to generate a first transmission image, and a frame generated from moving image data Is subjected to transmission processing based on the second transmittance α2 (that is, α2 = 1−α1) which is the complement of the first transmittance α1 to generate a second transmission image, and the first transmission image and the second transmission image are Combine to generate a composite image, that is, execute α blending. By changing the first transmittance α1 from 1 to 0 or 0 to 1 (that is, the second transmittance α2 from 0 to 1 or 1 to 0) from the beginning to the end of the switching period The image generated by one of the data and the moving image data gradually changes to the image generated by the other.

  The transition reproduction unit 315 performs the above-described transition reproduction processing between the first period in which one of the first image reproduction unit 312 and the second image reproduction unit 313 reproduces the image and the second period in which the other reproduces the image. It performs in the switching period provided in. That is, when switching from the difference mode to the moving image mode, at the end of the first period, that is, at the start of the switching period, the first transmittance α1 is set to 1 and the second transmittance α2 is set to 0 to start α blending, gradually Blending is sequentially performed while bringing the first transmittance α1 close to 0 and the second transmittance α2 close to 1, and the first transmittance α1 is set to 0 at the end of the switching period, that is, at the start of the second period. The α blending is performed with the transmittance α 2 set to 1. On the other hand, when switching from the moving image mode to the difference mode, at the end of the first period, that is, at the start of the switching period, the first transmittance α1 is set to 0 and the second transmittance α2 is set to 1 to start α blending, gradually Blending is sequentially performed while bringing the first transmittance α1 close to 1 and the second transmittance α2 close to 0, and the first transmittance α1 is set to 1 at the end of the switching period, that is, at the start of the second period. The α blending is performed with the transmittance α2 set to 0.

  The audio reproduction unit 314 gradually reduces the level of the audio reproduced along with the video reproduced by one of the first video reproduction unit and the second video reproduction unit during the above switching period, and the reproduction is performed by the other. The level of the audio to be reproduced along with the video being played is gradually increased. That is, the audio reproduced with the video reproduced in the first period is faded out in the switching period, and the audio reproduced with the video reproduced in the second period is faded in in the switching period.

[Operation of remote playback system]
Next, the operation of the remote reproduction system 100 according to the present embodiment will be described. In remote reproduction system 100 according to the present embodiment, content transmission apparatus 200 and reproduction apparatus 300 use TCP (Transmission Control Protocol), which is a connection type protocol, and UDP (User Datagram Protocol), which is a connectionless type protocol. It sends and receives content via wireless LAN. UDP is used for sending and receiving audio data, and TCP is used for sending and receiving difference data and moving image data. The playback device 300 plays back the received content and causes the display device 400 to output video and audio.

  7A to 7D are flowcharts showing the procedure of the remote reproduction of content by the remote reproduction system according to the present embodiment. As an example, consider a case where a school teacher (hereinafter referred to as a "user") conducts a lesson. The user activates the content transmission program 210 of the content transmission apparatus 200, and also activates the educational program 230. When the educational program 230 is executed, a moving image is played back, a slide of a presentation is played back, and a video is displayed on the display unit 202 of the playback device 200. Further, audio attached to the moving image or slide is output from the speaker 203.

  When the content transmission program 210 is executed, first, the CPU 261 operates in the difference mode. The CPU 261 takes in the video displayed on the display unit 202, and generates video data of the first frame or several frames (step S101). Next, the CPU 261 generates a TCP packet based on the generated video data. At this time, a TCP header is added to uncompressed video data to generate a TCP packet. Further, the CPU 261 adds an IP header to a TCP packet to generate an IP packet, and further adds a MAC header, a physical header, etc. to generate a wireless LAN frame (hereinafter referred to as "initial video packet") including video data. (Step S102). The destination of this initial video packet is the IP address of the playback apparatus 300. The CPU 261 causes the wireless communication unit 201 to transmit the generated initial video packet by TCP (step S103). The transmitted initial video data is received by the playback device 300 (step S104).

  Next, the CPU 261 takes in the video displayed on the display unit 202 and the audio output from the speaker 203, and generates video data and audio data (step S105). The CPU 261 generates difference data of the previous frame (that is, the previously transmitted video data; hereinafter referred to as “previous frame”) and the frame fetched this time (hereinafter referred to as “this frame”) (step S106). ). In this process, the difference between the luminance values (RGB values) of the corresponding pixels of the previous frame and the current frame is calculated, and difference data is generated based on the calculation result. FIG. 8 is a diagram for explaining the generation process of difference data. The luminance value of each pixel is the same between the previous frame 501 and the current frame 502 except for the difference portion 503. Therefore, when the difference in luminance value of each pixel between the previous frame 501 and the current frame 502 is taken, the value becomes 0 except for the difference portion 503. The CPU 261 removes the information on the pixel of which the difference in luminance is 0 and generates difference data. Therefore, only the information of the difference unit 503 is included in the difference data, and the amount of information is suppressed.

  The CPU 261 generates a TCP packet based on the generated difference data. At this time, a TCP header is added to the non-compressed differential data to generate a TCP packet. Furthermore, the CPU 261 adds an IP header to a TCP packet to generate an IP packet, and further adds a MAC header, a physical header, and the like to generate a wireless LAN frame (hereinafter referred to as “difference packet”) including difference data Step S107). The destination of the difference packet is also taken as the IP address of the playback device 300. The CPU 261 causes the wireless communication unit 201 to transmit the generated difference packet by TCP (step S108).

  Further, the CPU 261 adds a UDP header to voice data of linear PCM (Pulse Code Modulation) format, and generates a UDP packet. The linear PCM format is an uncompressed audio data format. The CPU 261 adds an IP header to the UDP packet to generate an IP packet, and further adds a MAC header, a physical header, and the like to generate a wireless LAN frame (hereinafter referred to as “voice packet”) including voice data ( Step S109). The voice packet includes a time stamp which is time information. Also, the destination of the audio packet is the IP address of the playback device 300. The CPU 261 causes the wireless communication unit 201 to transmit the generated voice packet by UDP (step S110).

  The reproduction device 300 receives the differential packet and the audio packet transmitted from the content transmission device 200 (steps S111 and S112). The CPU 331 of the playback device 300 extracts difference data from the received difference packet, replaces the location (pixel) where the difference data is present from the previous frame with the difference data, and generates video data of the current frame (step S113) Is reproduced (step S114). Specifically, the CPU 261 writes the generated video data in the first frame memory 341, and outputs the video data in the first frame memory 341 as a video signal from the external output unit 304 to the display device 400. Thereby, the video reproduction in the difference mode is performed.

  Also, the CPU 331 extracts voice data from the received voice packet, and reproduces voice (step S115). That is, the CPU 261 generates an audio signal from the audio data, and outputs the audio signal from the external output unit 304 to the display device 400. As described above, the video is displayed on the display unit 401 of the display device 400, and the audio is output from the speaker 402.

  In the display device 400, the user may make an input on the screen with a stylus or a finger. The input unit 403 receives the input given by the user in this manner, and transmits the input information to the playback apparatus 300 through the external connection terminal 405. The CPU 331 of the reproduction device 300 determines whether or not an input is given from the user (step S116), and when the input is given (YES in step S116), transmits the input information to the content transmission device 200 (step S116) Step S117). On the other hand, when the user does not receive an input (NO in step S116), the CPU 331 shifts the processing to step S120.

  When the content transmission device 200 receives the input information, the content transmission device 200 reflects the input by the user on the image and displays it. For example, when the user performs handwriting input on the screen on the display device 400, an image including the input character is displayed on the display unit 202 of the reproduction device 200.

  The CPU 261 of the content transmission apparatus 200 determines whether the first switching condition, which is the switching condition of the transmission object from difference data to moving image data, is satisfied (step S118).

Here, the first switching condition will be described. The first switching condition includes the following two conditions (1) to (2), and it is determined that the first switching condition is satisfied when either of these conditions is satisfied.
(1) The image change amount exceeds a predetermined reference value.
(2) The user gives a switching instruction to the moving image mode.

  The condition (1) will be described. The CPU 261 of the content transmission apparatus 200 detects a pixel having a difference in luminance value (RGB value) between the previous frame and the current frame, counts the number of pixels, and sets it as an image change amount. In the difference mode, although the amount of data communication is small, it is difficult to display a moving image smoothly. On the other hand, in the moving image mode, the time required for the data compression process increases. For this reason, a moving image mode is suitable for a video that changes significantly over time, and a differential mode is suitable for a video that changes less over time. Therefore, the content transmitting apparatus 200 detects the video change amount, and switches the transmission target from difference data to moving image data when the video change amount exceeds a predetermined reference value.

  The condition (2) will be described. In the present embodiment, a user operation for instructing switching from the difference mode to the moving image mode is defined. For example, the user operation may be an operation of tracing a screen (that is, the input unit 403) in a specific direction, or an operation of selecting a selection unit for selecting switching from the difference mode to the moving image mode included in the display screen Is realized by When this operation input is given to the input unit 403, an instruction to switch from the difference mode to the moving image mode is given to the CPU 261 of the content transmission apparatus 200.

  If the above first switching condition is not satisfied (NO in step S118), the CPU 261 returns the process to step S105. Thereby, the processes of steps S105 to S110 and step S118 are repeated, and transmission of difference data and sound data is continuously performed. On the other hand, when the first switching condition is satisfied (YES in step S118), the CPU 261 requests the reproduction device 300 to make a request to switch the reception target from differential data to moving image data (hereinafter referred to as “reception target switching request”). It transmits (step S119).

  The CPU 331 of the reproducing device 300 determines whether or not the reception target switching request has been received (step S120). If the reproducing device 300 has not received the reception target switching request (NO in step S120), the CPU 331 returns the process to step S111. Thereby, the processes of steps S111 to S117 and step S120 are repeated, and the video reproduction by the difference data and the audio reproduction by the audio data are continuously executed.

  After transmitting the reception target switching request, the CPU 261 of the content transmission device 200 transmits both the difference data and the moving image data to the reproduction device 300 in a predetermined switching period as follows. First, the CPU 261 starts a switching period (step S121). Next, the CPU 261 takes in the video displayed on the display unit 202 and the audio output from the speaker 203, and generates video data and audio data (step S122).

  The CPU 261 generates difference data (step S123), generates a difference packet (step S124), and transmits the difference packet to the wireless communication unit 201 by TCP (step S125), as in steps S106 to S108.

  In addition, the CPU 261 causes a plurality of temporally consecutive frames to be a moving image compression standard H.264. The image data is encoded by the moving image compression method defined in H.264 to generate moving image data (step S126). In the present embodiment, H. Although the configuration for generating H.264 format moving image data is described, the present invention is not limited to this, and other moving image data such as Divx, Xvid, VP9, or MPEG-4 may be generated. .

  The CPU 261 generates a TCP packet from the generated moving image data. Furthermore, the CPU 261 adds an IP header to a TCP packet to generate an IP packet, and further adds a MAC header, a physical header, and the like to generate a wireless LAN frame (hereinafter referred to as "moving image packet") including moving image data Step S127). The moving image packet includes a time stamp which is moving image time information. Also, the destination of the moving image packet is also taken as the IP address of the playback device 300. The CPU 261 causes the wireless communication unit 201 to transmit the generated moving image packet by TCP (step S128).

  The CPU 261 also generates an audio packet (step S129) as in steps S109 to 110, and causes the wireless communication unit 201 to transmit the audio packet by UDP (step S130).

  The CPU 261 determines whether or not the switching period has ended (step S131), and if it has not ended (NO in step S131), returns the process to step S122. Thus, both the difference data and the moving image data are continuously transmitted from the content transmission apparatus 200 until the switching period ends.

  On the other hand, when the reception target switching request is received in the reproducing device 300 (YES in step S120), the CPU 331 starts the switching period (step S132). Furthermore, the reproduction device 300 receives each of the differential packet, the moving image packet, and the audio packet transmitted from the content transmission device 200 (steps S133 to S135). The CPU 331 sets the first transmittance α1 and the second transmittance α2, and the audio output level SL (step S136). When switching from the difference mode to the moving image mode, the initial value of the first transmittance α1 is “1”, and the initial value of the second transmittance α2 is “0”. Further, the initial value of the audio output level SL is the same value as the audio output level in the difference mode.

  Next, the CPU 331 extracts difference data from the received difference packet, adds the luminance value of the previous frame and the difference data for each pixel, and generates video data of the current frame (step S137). The generated video data is written to the first frame memory 341. Further, the CPU 331 multiplies the luminance value of each pixel of the video data stored in the first frame memory 341 by the first transmittance α1 to generate a first transmission image (step S138).

  The CPU 331 also extracts moving image data from the received moving image packet, decodes the moving image data, and generates video data of a frame (step S139). The generated video data is written to the second frame memory 342. Further, the CPU 331 multiplies the luminance value of each pixel of the video data stored in the second frame memory 342 by the second transmittance α2 to generate a second transmission image (step S140).

  The CPU 331 combines the first transmission image and the second transmission image, and outputs a video signal of the combined image from the external output unit 304 to the display device 400 (step S141). Thereby, the video reproduction in the transition mode is performed.

  Further, the CPU 331 extracts audio data from the received audio packet, and reproduces audio at the set audio output level SL (step S142). That is, the CPU 261 generates an audio signal from the audio data, and outputs the audio signal from the external output unit 304 to the display device 400.

  Next, the CPU 331 determines whether or not the switching period has ended (step S143). If the switching period has not ended (NO in step S143), the process returns to step S133. Thereby, the processing of steps S133 to S143 is repeated, and the alpha blending of the image is continuously executed. In this switching period, the first transmittance α1 and the second transmittance α2 and the audio output level SL are gradually changed.

  FIG. 9 is a graph showing changes in the first transmittance α1 and the second transmittance α2 when switching from the difference mode to the moving image mode. In FIG. 9, the vertical axis indicates the transmittance, and the horizontal axis indicates the time. As shown in FIG. 9, the first transmittance α1 is 1 and the second transmittance α2 is 0 at the beginning of the switching period. As the switching period progresses, the first transmittance α1 gradually decreases and the second transmittance α2 gradually increases. At the end of the switching period, the first transmittance α1 becomes 0 and the second transmittance α2 becomes 1. As described above, by changing the first transmittance α1 and the second transmittance α2, the video by the difference data is gradually switched to the video by the moving image data in the switching period.

  FIG. 10 is a graph showing changes in the audio output level SL when switching from the difference mode to the moving image mode. In FIG. 10, the vertical axis represents the audio output level SL, and the horizontal axis represents time. As shown in FIG. 10, the audio output level SL is maximum at the beginning of the switching period, the audio output level SL gradually decreases as the switching period progresses, and the audio output level SL becomes 0 in the middle of the switching period. . This causes the audio to fade out. When the time further advances from the middle of the switching period, the audio output level SL gradually increases, and the audio output level SL becomes maximum at the end of the switching period. This causes the audio to fade in. Further, the CPU 331 synchronizes the audio with the video of the moving image data by the time stamp added to the audio data and the time stamp added to the moving image data.

  Reference is made to FIG. 7B. When the switching period is over (YES in step S131), the content transmitting apparatus 200 transmits the moving image packet and the audio packet to the reproducing apparatus 300 as follows. First, the CPU 261 takes in the video displayed on the display unit 202 and the audio output from the speaker 203, and generates video data and audio data (step S144). Further, the CPU 261 generates moving image data (step S145), generates a moving image packet (step S146), and transmits the moving image packet to the wireless communication unit 201 by TCP (step S147) as in steps S126 to S128.

  Further, the CPU 261 generates an audio packet in the same manner as in steps S109 to 110 (step S148), and causes the wireless communication unit 201 to transmit the audio packet by UDP (step S149).

  On the other hand, when the switching period has ended in the reproducing apparatus 300 (YES in step S143), the reproducing apparatus 300 receives the moving image packet and the audio packet transmitted from the content transmitting apparatus 200 (steps S150 and S151). The CPU 331 of the reproduction device 300 extracts moving image data from the received moving image packet, decodes the moving image data to generate video data of a frame (step S152), and reproduces a video (step S153). Specifically, the CPU 261 writes the generated video data into the second frame memory 342, and outputs the video data of the second frame memory 342 from the external output unit 304 to the display device 400 as a video signal. Thereby, the video reproduction in the moving image mode is performed.

  Also, the CPU 331 extracts voice data from the received voice packet and reproduces voice (step S154). That is, the CPU 261 generates an audio signal from the audio data, and outputs the audio signal from the external output unit 304 to the display device 400. At this time, the CPU 331 synchronizes the audio with the video using the time stamp added to the moving image data and the time stamp added to the audio data. As described above, the video is displayed on the display unit 401 of the display device 400, and the audio is output from the speaker 402.

  The CPU 331 determines whether or not an input is given from the user to the input unit 403 (step S155), and when the input is given (YES in step S155), transmits the input information to the content transmission apparatus 200 (Step S156). On the other hand, when the user does not receive an input (NO in step S155), the CPU 331 shifts the processing to step S159.

  The CPU 261 of the content transmission apparatus 200 determines whether the second switching condition, which is the switching condition of the transmission object from moving image data to difference data, is satisfied (step S157).

Here, the second switching condition will be described. The second switching condition includes the following three conditions (3) to (5), and it is determined that the second switching condition is satisfied when either of these conditions is satisfied.
(3) The predetermined first determination period is exceeded while the image change amount does not exceed the predetermined reference value.
(4) The user's input to the screen is detected continuously for a predetermined second determination period or more.
(5) An instruction to switch to the difference mode is given from the user.

  The condition (3) will be described. As described above, the moving image mode is suitable for a video that changes greatly in time, and the differential mode is suitable for a video that changes less in time. Therefore, the content transmitting apparatus 200 detects the video change amount, and switches the transmission target from moving image data to difference data when the video change amount does not continuously exceed the reference value for the first determination period or more.

  The condition (4) will be described. The CPU 261 of the content transmission device 200 monitors the operation on the input unit 403 according to the input information supplied from the reproduction device 300. The CPU 331 detects a predetermined input to the video when the operation on the input unit 403 is continuously applied for the second determination period or more. For example, the user may use the display device 400 as an electronic blackboard and draw characters on the screen by the stylus, that is, perform handwriting input. It is important that such handwriting input reflect the user's input on the display without delay as much as possible. For this reason, video reproduction using differential data with less delay is more suitable than video reproduction using moving image data that causes a delay in video reproduction. Therefore, the content transmission apparatus 200 switches the transmission target from moving image data to difference data when detecting a user input to the video continuously for the second determination period or more.

  The condition (5) will be described. In the present embodiment, a user operation for instructing switching from the moving image mode to the difference mode is defined. For example, the user operation may be an operation of tracing a screen (that is, the input unit 403) in a specific direction, or an operation of selecting a selection unit for selecting switching from the moving image mode to the difference mode included in the display screen. Is realized by When this operation input is given to the input unit 403, an instruction to switch from the moving image mode to the difference mode is given to the CPU 261 of the content transmission apparatus 200.

  If the second switching condition is not satisfied (NO in step S157), the CPU 261 returns the process to step S144. Thereby, the processes of steps S144 to S149 and step S157 are repeated, and transmission of moving image data and audio data is continuously performed. On the other hand, when the second switching condition is satisfied (YES in step S157), the CPU 261 transmits, to the playback apparatus 300, a request for switching the reception target from moving image data to difference data (reception target switching request) (step S158). ).

  The CPU 331 of the reproducing device 300 determines whether or not the reception target switching request has been received (step S159). If the reproducing device 300 has not received the reception target switching request (NO in step S159), the CPU 331 returns the process to step S150. Thereby, the processes of steps S150 to S156 and step S159 are repeated, and the video reproduction by the moving image data and the audio reproduction by the audio data are continuously executed.

  After transmitting the reception target switching request, the CPU 261 of the content transmission apparatus 200 transmits difference data, moving image data, and audio data to the reproduction apparatus 300 in a predetermined switching period, as in steps S121 to S131 (step S160). ~ S170).

  On the other hand, when the reception target switching request is received in the reproducing device 300 (YES in step S120), the CPU 331 starts the switching period (step S171). Furthermore, the reproduction device 300 receives each of the differential packet, the moving image packet, and the audio packet transmitted from the content transmission device 200 (steps S172 to S174). The CPU 331 sets the first transmittance α1 and the second transmittance α2, and the audio output level SL (step S175). When the moving image mode is switched to the differential mode, the initial value of the first transmittance α1 is “0”, and the initial value of the second transmittance α2 is “1”. The initial value of the audio output level SL is the same value as the audio output level in the moving image mode.

  Next, the CPU 331 extracts difference data from the received difference packet, adds the luminance value of the previous frame and the difference data for each pixel, and generates video data of the current frame (step S176). The generated video data is written to the first frame memory 341. Further, the CPU 331 multiplies the luminance value of each pixel of the video data stored in the first frame memory 341 by the first transmittance α1 to generate a first transmission image (step S177).

  Further, the CPU 331 extracts moving image data from the received moving image packet, decodes the moving image data, and generates frame video data (step S178). The generated video data is written to the second frame memory 342. Further, the CPU 331 multiplies the luminance value of each pixel of the video data stored in the second frame memory 342 by the second transmittance α2 to generate a second transmission image (step S179).

  The CPU 331 combines the first transmission image and the second transmission image, and outputs a video signal of the combined image from the external output unit 304 to the display device 400 (step S180). Thereby, the video reproduction in the transition mode is performed.

  Further, the CPU 331 extracts voice data from the received voice packet, and reproduces voice at the set voice output level SL (step S181). That is, the CPU 331 generates an audio signal from the audio data, and outputs the audio signal from the external output unit 304 to the display device 400.

  Next, the CPU 331 determines whether or not the switching period has ended (step S182). If the switching period has not ended (NO in step S182), the process returns to step S172. Thereby, the processes of steps S172 to S182 are repeated, and the alpha blending of the image is continuously executed. In this switching period, the first transmittance α1 and the second transmittance α2 and the audio output level SL are gradually changed.

  FIG. 11 is a graph showing changes in the first transmittance α1 and the second transmittance α2 when switching from the moving image mode to the difference mode. In FIG. 11, the vertical axis indicates the transmittance, and the horizontal axis indicates the time. As shown in FIG. 11, the first transmittance α1 is 0 and the second transmittance α2 is 1 at the start of the switching period. As the switching period progresses, the first transmittance α1 gradually increases and the second transmittance α2 gradually decreases. At the end of the switching period, the first transmittance α1 is 1, and the second transmittance α2 is 0. By changing the first transmittance α1 and the second transmittance α2 as described above, the video of the moving image data is gradually switched to the video of the difference data in the switching period.

  In the switching period from the moving image mode to the difference mode, the audio output level SL changes similarly to the switching period from the difference mode to the moving image mode. That is, the audio output level SL is maximum at the beginning of the switching period, and the audio output level SL gradually decreases as the switching period progresses, and the audio output level SL becomes 0 in the middle of the switching period. This causes the audio to fade out. When the time further advances from the middle of the switching period, the audio output level SL gradually increases, and the audio output level SL becomes maximum at the end of the switching period. This causes the audio to fade in. Further, the CPU 331 synchronizes the audio with the video of the moving image data by the time stamp added to the audio data and the time stamp added to the moving image data.

  Refer to FIG. 7D. If the switching period has ended (YES in step S170), the CPU 261 of the content transmission device 200 returns the process to step S105. As a result, the transmission target is switched from the moving image packet to the differential packet, and the differential packet and the audio packet are transmitted to the reproduction device 300 (steps S105 to S110).

  On the other hand, when the switching period has ended in the reproducing device 300 (YES in step S182), the CPU 331 of the reproducing device 300 returns the process to step S111. As a result, the mode is shifted to the difference mode, and the video based on the difference data is reproduced.

  By configuring as described above, it is possible to switch between the differential mode in which the image quality is low but the amount of data processed per unit time is small, and the moving image mode in which the image quality is high but the amount of data processable per unit time is large It is possible to achieve both communication volume and high image quality. Further, in the moving image mode in which a shift between the video and the audio is likely to occur, since the video and the audio are synchronized, the shift between the video and the audio can be suppressed. In the difference mode, since the amount of data is small and the reproduction impossibility of the video reproduction process is small, the difference between the video and the audio is small without synchronization.

  Further, the content transmission apparatus 200 generates an audio packet separately from the differential packet and the moving image packet, and transmits the audio packet when transmitting each of the differential packet and the moving image packet. That is, audio data is shared in the differential mode and the moving image mode, and audio data dedicated to the differential mode and audio data dedicated to the moving image mode are not generated. For this reason, there is no need for the audio data for reproducing the same audio to be duplicated, and it is necessary to reduce the amount of communication, and at the same time to generate audio data of different types for the difference mode and the moving image mode. Instead, the configuration of the content transmission apparatus 200 is simplified. Similarly, the playback apparatus 300 also does not have to process different types of audio data, and the configuration is simplified.

(Other embodiments)
In the embodiment described above, a switching period for switching the operation mode between the difference mode and the moving image mode is provided, and in this switching period, video reproduction by one of the difference data and the moving image data is gradually performed to video reproduction by the other. However, the present invention is not limited to this. It is also possible to adopt a configuration in which the difference mode is directly switched to the moving image mode or the moving image mode is directly switched to the difference mode without providing the switching period.

  Further, in the above-described embodiment, the configuration in which the content transmission apparatus 200 transmits both the difference data and the moving image data to the reproduction apparatus 300 in the switching period has been described. However, the present invention is not limited to this. For example, when the transmission target is switched, the transmission target data before switching may not be transmitted, and the transmission target data after switching may be transmitted. In this case, the video of the last frame reproduced by the data before switching and the video generated by the new transmission target data received after switching are gradually switched by α blending in the switching period. Can.

  In the above-described embodiment, the first switching condition includes the conditions (1) and (2), and the second switching condition includes the conditions (3) to (5). It is not limited to The first switching condition may include only one of the conditions (1) and (2), and the second switching condition may include only one or two of the conditions (3) to (5). It may be The first switching condition and the second switching condition may include conditions other than the conditions (1) and (2) and the conditions (3) to (5).

  In the embodiment described above, during the switching period, the audio reproduced along with the video reproduced by one of the difference data and the moving image data is faded out, and the audio reproduced along with the video reproduced by the other Has been described, but is not limited thereto. The audio may be switched at one time point within the switching period without increasing or decreasing the output level SL of the audio reproduced with the video reproduced by each of the differential data and the moving image data. However, a configuration in which the audio is faded out and faded in is preferable because the sense of discomfort given to the viewer can be suppressed.

  Further, in the above-described embodiment, the configuration in which the content transmitting apparatus 200 is provided with the respective functions of the change amount detecting unit 217, the input detecting unit 218, the switching instruction receiving unit 219, the transmission target switching unit 220, and the switching request unit 221 Although said, it is not limited to this. The playback apparatus 300 is provided with each function of a change amount detection unit, an input detection unit, a switching instruction reception unit, and a switching request unit, detection of the change amount of the video reproduced by the playback apparatus 300, and detection of input from the user When the request for switching the image reproduction from the user is received, the switch request for the transmission object is transmitted to the content transmitter 200, and when the content transmitter 300 receives the switch request, the difference for the transmission object is Alternatively, one of the data and the moving image data may be switched to the other.

  Further, in the above-described embodiment, the configuration has been described in which the voice data is transmitted by UDP and the difference data and the moving image data are transmitted by TCP, but the present invention is not limited to this. Audio data may be transmitted by TCP, and one or both of differential data and moving image data may be transmitted by UDP. However, by transmitting the difference data and the moving image data by the TCP that performs the response confirmation and retransmission processing, it is possible to suppress the interruption or disturbance of the image and reproduce the high-quality image. It is preferable to use TCP for transmission.

  Further, in the above-described embodiment, the configuration has been described in which non-compressed differential data and audio data are transmitted and compressed video data is transmitted, but the present invention is not limited to this. Either or both of the difference data and the audio data may be in a compressed format, or the moving image data may be in an uncompressed format. However, by making differential data and audio data in a non-compressed format, it is not necessary to decompress the reproduction apparatus and the reproduction load is small, and therefore delay of video and audio between the content transmission apparatus 200 and the display apparatus 400 is suppressed. be able to. In addition, by setting moving image data in a compression format, the bit rate of video can be improved, and high-quality video can be reproduced while suppressing the amount of communication.

  Moreover, in the embodiment described above, although the configuration in which the playback device 300 is a wireless access point has been described, the present invention is not limited to this. For example, a reproduction device dedicated to reproduction of content may be configured to be able to execute the above-described operation. In addition, the playback device and the display device 400 may be separately provided and not connected to each other, but may be a playback device including a display unit. The content transmission apparatus 200 and the reproduction apparatus 300 may not be configured to communicate by wireless LAN, and may be configured to perform wired communication.

  The reproduction apparatus, remote reproduction system, reproduction method, and computer program according to the present invention are a reproduction apparatus for reproducing video and audio distributed by streaming, a remote reproduction system, a reproduction method, and a computer program for causing a computer to function as a reproduction apparatus. It is useful.

100: remote reproduction system 200: content transmission apparatus 201: wireless communication unit 205: storage unit 206: control unit 210: content transmission program 217: change amount detection unit 218: input detection unit 219: switching instruction reception unit 220: transmission target switching unit 261 CPU
262 Memory 300 Reproduction device 301 Radio communication unit 303 Control unit 311 Communication unit 312 First video reproduction unit 313 Second video reproduction unit 314 Audio reproduction unit 315 Transition reproduction unit 331 CPU
341 first frame memory 342 second frame memory 400 display device

Claims (18)

A communication unit for receiving audio data to which audio time information is added, differential data between two temporally preceding and succeeding frames, and moving image data to which moving image time information is added;
A first video reproduction unit that reproduces a video by a frame after being generated based on the difference data and a previous frame of the two frames;
A second video reproduction unit that reproduces a video by a plurality of temporally consecutive frames generated based on the moving image data;
The audio is reproduced based on the audio data, and when the second video reproduction unit reproduces the video, the audio is synchronized with the video based on the moving image time information and the audio time information. And an audio playback unit,
Playback device. The audio reproduction unit reproduces audio by common audio data in each of the case where the first video reproduction unit is reproducing video and the case where the second video reproduction unit is reproducing video. ,
The playback device according to claim 1. The communication unit receives the voice data according to a connectionless protocol.
The playback device according to claim 1. The communication unit receives uncompressed audio data.
The playback device according to claim 3. The communication unit receives the difference data according to a connection-oriented protocol.
The reproduction apparatus according to any one of claims 1 to 4. The communication unit receives uncompressed differential data.
The playback device according to claim 5. The communication unit receives the moving image data by a connection type protocol.
The reproduction apparatus according to any one of claims 1 to 6. The communication unit receives the compressed moving image data.
The playback device according to claim 7. The communication unit receives, from a transmission source of the difference data and the moving image data, a request for switching a target to be received from one of the difference data and the moving image data to the other.
The first video playback unit and the second video playback unit play back video from one of the first video playback unit and the second video playback unit when the switching request is received by the communication unit. Switch,
The reproduction apparatus according to any one of claims 1 to 8. The communication unit simultaneously receives both the difference data and the moving image data when the reproduction of the video is switched.
It is simultaneously received in a predetermined switching period provided between a first period in which one of the first video reproduction unit and the second video reproduction unit reproduces a video and a second period in which the other reproduces a video. And a transition reproduction unit for changing the image generated by one of the difference data and the moving image data stepwise to an image generated by the other and reproducing the image.
The reproduction apparatus according to any one of claims 1 to 9. A first frame memory and a second frame memory,
The transition reproduction unit stores, in the first frame memory, a first transmission image obtained by performing transmission processing based on a predetermined first transmittance on an image generated by the difference data, and the moving image data A second transmission image obtained by performing transmission processing based on a second transmission that is a complement of the first transmission to the image generated by the second frame memory is stored in the second frame memory; Combining the second transmission image with the second transmission image,
The playback device according to claim 10. The audio reproduction unit gradually reduces the level of audio reproduced along with the video reproduced by one of the first video reproduction unit and the second video reproduction unit when the reproduction of the video is switched. , Gradually increase the level of the audio reproduced with the video reproduced by the other,
A reproduction apparatus according to any one of claims 1 to 11. A content transmitting apparatus for transmitting audio data to which audio time information is added, differential data between two frames which are temporally adjacent to each other, and moving image data to which moving image time information is added;
A reproduction apparatus according to any one of claims 1 to 12.
Remote playback system. The content transmission device is
A change amount detection unit that detects a temporal change amount of a video to be reproduced;
A transmission target switching unit that switches a transmission target from one of the difference data and the moving image data to the other based on the temporal change amount of the video detected by the change amount detection unit;
And a switching request unit for requesting the playback apparatus to switch the reception target from one of the difference data and the moving image data to the other.
The reproduction device is the reproduction device according to claim 9.
The remote playback system according to claim 13. The content transmission device is
An input detection unit that detects an input to the displayed image;
A transmission target switching unit that switches a transmission target from one of the difference data and the moving image data to the other based on the detection of the input by the input detection unit;
And a switching request unit for requesting the playback apparatus to switch the reception target from one of the difference data and the moving image data to the other.
The reproduction device is the reproduction device according to claim 9.
The remote reproduction system according to claim 13 or 14. The content transmission device is
A switching instruction receiving unit that receives a switching instruction between video reproduction based on the difference data and video reproduction based on the moving image data;
A transmission target switching unit that switches a transmission target from one of the difference data and the moving image data to the other based on the switching instruction received by the switching instruction reception unit;
And a switching request unit for requesting the playback apparatus to switch the reception target from one of the difference data and the moving image data to the other.
The reproduction device is the reproduction device according to claim 9.
The remote reproduction system according to any one of claims 13 to 15. Receiving voice data to which voice time information is added and difference data indicating a difference between two frames preceding and following in time;
The video is reproduced by the frame after being generated based on the received differential data and the previous frame of the two frames,
Based on the received audio data, reproduce the audio simultaneously with the video,
Audio data to which audio time information is added and moving image data to which moving image time information is added when switching from the image based on the difference data to an image based on moving image data for generating a plurality of temporally consecutive frames Receive
Stop playing the video based on the difference data;
Reproduces an image by a plurality of temporally consecutive frames generated based on the received moving image data;
The audio is reproduced based on the received audio data, and the audio is synchronized with the video based on the moving image time information and the audio time information.
How to play. A computer program for causing a computer to function as a content transmission device that transmits content including video and audio,
A first video transmission unit that transmits difference data indicating a difference between two temporally consecutive frames;
A second video transmission unit for transmitting moving image data to which moving image time information is added;
Audio transmitting means for transmitting audio data to which audio time information is added; and switching video reproduction from one of the difference data and the moving image data to the other, the first video transmitting means and the second video transmitting means Causing the computer to function as switching means for switching an execution target from one to the other,
Computer program.

Images