JP7200313B2 - Playback device, remote playback system, playback method, and computer program - Google Patents

Description

The present invention relates to a playback device, a remote playback system, a playback method, and a computer program for causing a computer to function as a playback device for playing back video and audio distributed by streaming.

For example, there is known a remote playback system for remotely playing back (mirroring) the same video and audio as the video and audio output to a terminal device used by a teacher in a class on a large display device installed in the classroom. (for example, Patent Document 1). In such a remote playback system, a terminal device captures video and audio output by its own device, generates video data and audio data, and transmits these data by, for example, wireless communication. A reproducing device such as an access point of a wireless LAN (Local Area Network) receives and reproduces these data, and video and audio are output to a display device connected to the reproducing device.

JP 2016-213541 A

By the way, in order to achieve both low communication traffic and high image quality, there is a video playback process with low image quality but a small amount of data processed per unit time, and a video playback process with high image quality but a large amount of data processed per unit time. It is conceivable to switch to playback processing. However, in such switching of video reproduction processing, the reproduction load of the latter is higher than that of the former, and the delay time from reception of data to reproduction of video is long. On the other hand, since the reproduction load of audio is low and the delay time of reproduction is short, there is a problem that when the former is switched to the latter, the video is delayed with respect to the audio.

SUMMARY OF THE INVENTION The present invention has been made in view of such circumstances, and its main object is to provide a playback device, a remote playback system, a playback method, and a computer program that can solve the above problems.

In order to solve the above-described problems, a reproducing apparatus according to one aspect of the present invention provides audio data to which audio time information is added, difference data between two temporally preceding and following frames, and video time information to which video time information is added. a communication unit that receives the moving image data obtained from the video data; a first video reproducing unit that reproduces video using a subsequent frame generated based on the difference data and the previous frame of the two frames; and the moving image data. a second video reproduction unit that reproduces video from a plurality of temporally continuous frames generated based on the above; and reproduces audio based on the audio data, and the second video reproduction unit reproduces the video. and an audio reproducing unit for synchronizing the audio with the video based on the moving image time information and the audio time information.

A remote playback system according to another aspect of the present invention transmits audio data to which audio time information is added, differential data between two temporally preceding and following frames, and moving image data to which moving image time information is added. A content transmission device and a playback device according to the aspect described above are provided.

Further, a reproduction method according to another aspect of the present invention receives audio data to which audio time information is added and difference data indicating a difference between two temporally preceding and succeeding frames, and reproduces the received difference data. and a frame generated based on the previous frame of the two frames, reproduces the audio simultaneously with the video based on the received audio data, and reproduces the audio according to the difference data. receiving audio data to which audio time information is added and video data to which video time information is added when switching from video to video data for generating a plurality of temporally continuous frames; stopping reproduction of video based on the differential data, reproducing video with a plurality of temporally continuous frames generated based on the received moving image data, and reproducing audio based on the received audio data; and synchronizing the audio with the video based on the moving image time information and the audio time information.

A computer program according to another aspect of the present invention is a computer program for causing a computer to function as a content transmission device that transmits content including video and audio, the computer program comprising: a first video transmission means for transmitting differential data indicating the difference, a second video transmission means for transmitting video data to which video time information is added, an audio transmission means for transmitting audio data to which audio time information is added, and the difference When video reproduction is switched from one of the data and the video data to the other, the computer functions as switching means for switching the execution target from one of the first video transmission means and the second video transmission means to the other.

ADVANTAGE OF THE INVENTION According to this invention, the gap of an image|video and an audio|voice can be suppressed, making both low traffic and high image quality compatible.

1 is a schematic diagram showing the configuration of a remote playback system according to an embodiment; FIG. 1 is a block diagram showing the configuration of a content transmission device according to an embodiment; FIG. FIG. 2 is a functional block diagram showing functions of the content transmission device according to the embodiment; 1 is a block diagram showing the configuration of a playback device according to an embodiment; FIG. 1 is a block diagram showing the configuration of a display device according to an embodiment; FIG. FIG. 3 is a functional block diagram showing functions of the playback device according to the embodiment; 4 is a flowchart (Part 1) showing a procedure for remote playback of content by the remote playback system according to the embodiment; 4 is a flowchart (part 2) showing a procedure for remote playback of content by the remote playback system according to the embodiment; 3 is a flowchart (part 3) showing a procedure for remote playback of content by the remote playback system according to the embodiment; 4 is a flowchart (part 4) showing a procedure for remote playback of content by the remote playback system according to the embodiment; FIG. 4 is a diagram for explaining a difference data generation process; 5 is a graph showing changes in the first transmittance α1 and the second transmittance α2 when switching from the difference mode to the moving image mode; 4 is a graph showing changes in the audio output level SL when switching from the difference mode to the moving image mode; 7 is a graph showing changes in the first transmittance α1 and the second transmittance α2 when switching from the moving image mode to the differential mode;

Preferred embodiments of the present invention will be described below with reference to the drawings. It should be noted that each of the embodiments shown below exemplifies a method and apparatus for embodying the technical idea of the present invention, and the technical idea of the present invention is not limited to the following. Absent. Various modifications can be made to the technical idea of the present invention within the technical scope described in the claims.

In this embodiment, a remote playback system for remotely playing (mirroring) the same video and audio as the video and audio (content) output from a content transmission device, which is a terminal device used by a teacher, will be described.

[Configuration of remote playback system]
FIG. 1 is a schematic diagram showing the configuration of a remote playback system according to this embodiment. The remote playback system according to this embodiment is installed, for example, in public facilities such as schools, company buildings, hospitals, and public halls. In this embodiment, an example in which a remote playback system is installed in a school will be described. As shown in FIG. 1, the remote playback system 100 includes a content transmission device 200, a playback device 300, and a display device 400. FIG. A playback device 300 is installed in a classroom. The content transmission device 200 is a tablet used by the teacher. The content transmission device 200 is capable of wireless communication according to the wireless LAN standard IEEE802.11a/b/g/n/ac, and the playback device 300 is an access point capable of wireless communication according to the wireless LAN standard. A wireless LAN 450 is configured by the reproducing device 300 in each classroom and the principal's office. In addition, the playback device 300 is connected to a wired LAN 460 laid over each classroom and other rooms (principal's office, staff room, etc.) and the Internet (not shown), and communication connected to the wired LAN 460 and the Internet is performed. Communication with the device is possible.

Note that the content transmission device 200 may be a laptop personal computer, a smart phone, a mobile phone, or the like, as long as wireless communication is possible using a wireless LAN.

A configuration of the content transmission device 200 will be described. FIG. 2 is a block diagram showing the configuration of the content transmission device 200. As shown in FIG. The content transmission device 200 includes a wireless communication section 201 , a display section 202 , a speaker 203 , an input section 204 , a storage section 205 and a control section 206 .

The wireless communication unit 201 enables wireless communication according to IEEE802.11a/b/g/n/ac, and has an antenna 207 for wireless communication. The display unit 202 is configured by an LCD (liquid crystal display) or an organic EL (Electro Luminescence). The input unit 204 is a positional input device such as a pressure-sensitive or capacitive transparent touch pad. The input unit 204 is attached to the surface of the display unit 202, and the display unit 202 and the input unit 204 are configured as an integrated touch panel. The storage unit 205 is a nonvolatile storage device such as a flash memory or a hard disk, and stores a content transmission program 210, which is a computer program for content transmission, and an educational program 230 such as video playback software and presentation software. . The control unit 206 includes a CPU 261 and a memory 262, and controls each unit described above.

By executing the content transmission program 210 by the CPU 261, the tablet functions as the content transmission device 200 and causes the content transmission device 200 to realize operations described later.

FIG. 3 is a functional block diagram showing functions of content transmission device 200 according to the present embodiment. As shown in FIG. 3, the content transmission device 200 includes a communication unit 211, a display unit 212, a speaker 213, a content reproduction unit 214, a video capture unit 215, an audio capture unit 216, and a variation detection unit. It has the functions of a section 217 , an input detection section 218 , a switching instruction reception section 219 , a transmission target switching section 220 and a switching request section 221 .

The communication unit 211 is the wireless communication unit 201 described above, and transmits content data to the reproduction device 300 . Specifically, the communication unit 211 selectively transmits difference data indicating the difference between two temporally consecutive frames and moving image data for generating a plurality of temporally consecutive frames. Further, the communication unit 211 transmits audio data at the same time as the difference data and moving image data. Further, moving image time information and audio time information, which are time stamps, are added to each of the moving image data and the audio data.

The display unit 212 is the display unit 202 described above, and the speaker 213 is the speaker 203 described above. The content reproduction unit 214 reproduces content such as moving images, presentation materials, and audio, the display unit 212 outputs video, and the speaker 213 outputs audio.

The video capture unit 215 captures the video displayed on the display unit 212 and generates video data. The video capture unit 215 has a function of generating differential data between two frames as video data, and a function of generating video data by encoding a plurality of temporally continuous frames by a predetermined video compression method. The voice capture unit 216 captures the voice output from the speaker 213 and generates voice data in linear PCM format.

The communication unit 211 transmits the differential data or moving image data generated by the video capturing unit 215 and the audio data generated by the audio capturing unit. As the reproduction device 300 reproduces these contents, the display device 400 displays video and outputs audio. Further, the display device 400 allows input with a stylus, finger, or the like, as will be described later, and input information given by the user is transmitted from the playback device 300 . The communication unit 211 transmits difference data or moving image data and audio data as described above, and receives such input information.

The change amount detection unit 217 detects the temporal change amount of the video between frames, that is, the number of pixels changed from the previous frame. The input detection unit 218 monitors input information received by the communication unit 211 and detects input given by the user to the displayed image. For example, when an input is made to the display device 400 with a stylus or a user's finger, the input detection section 218 detects this input. When the user performs a specific operation to instruct the display device 400 to switch video reproduction between difference data and moving image data, the switching instruction receiving unit 219 receives the switching instruction through the input information received by the communication unit 211 . accept instructions. In the present embodiment, the detection of a predetermined amount of change by the change amount detection unit 217, the detection of a predetermined input by the input detection unit 218, and the reception of the video switching instruction by the switching instruction reception unit 219 are each performed using difference data and This is used as a switching condition for video data. That is, when any one of the above switching conditions is satisfied, the transmission target is switched from one of the difference data and the moving image data to the other.

The switching conditions include a first switching condition for switching the transmission target from the difference data to the moving image data, and a second switching condition for switching the transmission target from the moving image data to the difference data. The first switching condition and the second switching condition will be described later.

The transmission target switching unit 220 switches the transmission target to the reproduction device 300 from one of the difference data and the moving image data to the other when the switching condition is satisfied. That is, when the content transmission device 200 is transmitting difference data, when the first switching condition is satisfied, the video data generated by the video capturing unit 215 is switched from the difference data to the moving image data, and the transmission target data is changed from the difference data to the moving image data. is switched from differential data to video data. On the other hand, when the content transmission device 200 is transmitting moving image data, when the second switching condition is satisfied, the image data generated by the image capturing unit 215 is switched from moving image data to difference data, and the image data to be transmitted is changed from moving image data to difference data. is switched from video data to differential data.

The switching request unit 220 requests the playback device 300 to switch the reception target from one of the difference data and the moving image data to the other when the switching condition is satisfied. That is, when the content transmission device 200 is transmitting difference data and the first switching condition is satisfied, the switching request unit 220 causes the communication unit 211 to transmit a request to switch the reception target to video data. On the other hand, when the content transmission device 200 is transmitting moving image data and the second switching condition is satisfied, the switching request unit 220 causes the communication unit 211 to transmit a request to switch the reception target to moving image data.

Here, in the playback device 300, when switching video playback, a transition playback process is performed in which the video generated by one of the difference data and the moving image data is changed step by step to the video generated by the other. Both differential data and video data are used simultaneously in this process. Therefore, the communication unit 211 simultaneously transmits the difference data and the video data (including the same frame) for a certain switching period after the switching condition is satisfied, and after the switching period has passed, the data is used for video playback after switching. Send only data that is Details of the transition reproduction process will be described later.

Next, the configuration of the playback device 300 will be described. FIG. 3 is a block diagram showing the configuration of the playback device 300. As shown in FIG. As shown in FIG. 4 , playback device 300 includes wireless communication section 301 , wired communication section 302 , control section 303 , external output section 304 , and external connection terminal 305 .

The wireless communication unit 301 enables wireless communication according to IEEE802.11a/b/g/n/ac, and has an antenna 306 for wireless communication. The wired communication unit 302 is an Ethernet communication interface and is connected to the wired LAN 460 . The control unit 303 includes a CPU 331 and a memory 332 and controls communication by the wireless communication unit 301 and the wired communication unit 302 . The memory 332 also includes two frame memories, a first frame memory 341 and a second frame memory 342 .

The external output unit 304 is an HDMI (registered trademark, High-Definition Multimedia Interface) terminal, is connected to the display device 400, and can output a video signal and an audio signal to the display device 400 to output video and audio. . The external connection terminal 405 is a USB (Universal Serial Bus) terminal. Such an external connection terminal 405 is also connected to the display device 400 .

FIG. 5 is a block diagram showing the configuration of the display device 400. As shown in FIG. The display device 400 includes a display section 401 , a speaker 402 , an input section 403 , an external input section 404 , an external connection terminal 405 and a control section 406 . The display unit 401 is a large LCD or an organic EL panel, and by outputting video from the display unit 401 and audio from the speaker 402, everyone in the class can view the content. The input unit 403 is a position input device such as a pressure-sensitive or capacitive transparent touch pad. The input unit 403 is attached to the surface of the display unit 401, and the display unit 401 and the input unit 204 are configured as an integrated touch panel.

The external input unit 404 is an HDMI terminal, and the external connection terminal 405 is a USB terminal. The external input section 404 and the external connection terminal 405 are connected to the external output section 304 and the external connection terminal 305 of the playback device 300, respectively. The external input unit 404 receives the video signal and audio signal output from the reproducing device 300 , and the external connection terminal 405 transmits the output signal from the input unit 403 to the reproducing device 300 .

The control unit 406 has a CPU and a memory or a semiconductor circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array), and has a function of controlling each of the above units. The control unit 406 causes the display unit 401 to display video and the speaker 402 to output audio based on the video signal and audio signal input to the external input unit 404 .

FIG. 6 is a functional block diagram showing the functions of the playback device according to this embodiment. As shown in FIG. 6 , the playback device 300 has functions of a communication section 311 , a first video playback section 312 , a second video playback section 313 , an audio playback section 314 , and a transition playback section 315 .

The communication unit 311 is the wireless communication unit 301 described above, and receives content data from the content transmission device 200 . Specifically, the communication unit 311 selectively receives difference data indicating the difference between two temporally consecutive frames and moving image data for generating a plurality of temporally consecutive frames. Also, the communication unit 311 receives the audio data at the same time as the difference data and the moving image data. A time stamp, which is time information, is added to each of the moving image data and the audio data.

The first video reproduction unit 312 reproduces the video by generating and outputting the subsequent frame based on the difference data of the two frames and the preceding frame of the two frames. The second video reproducing unit 313 generates a plurality of frames based on the moving image data, and reproduces the video by outputting these frames continuously in terms of time. The audio reproduction unit 314 reproduces audio based on the audio data. It should be noted that the term "reproduction" as used herein means processing differential data, moving image data, and audio data to generate a video signal and an audio signal, and outputting the generated video signal and audio signal. In the following description, the operation mode in which video reproduction by the first video reproduction unit 312, that is, video reproduction based on difference data is performed will be referred to as a “difference mode”. An operation mode in which reproduction is performed is called a "moving image mode".

When the video is reproduced in the differential mode, the audio reproducing unit 314 processes the audio data and reproduces the audio each time the audio data is received without using the time information. On the other hand, when the video is being played back in the video mode, the audio playback unit 314 reproduces the audio in synchronization with the video based on the time stamp added to the video data and the time stamp added to the audio data. do. The audio reproduction unit 314 uses common audio data to reproduce audio in each of the difference mode and the moving image mode.

As described above, when video playback is switched from one of difference data and moving image data to video playback of the other, the content playback device 300 transmits a reception target switching request. Upon receiving this switching request, the communication unit 311 switches the reception target from one of the difference data and the moving image data to the other.

The content transmission device 200 simultaneously transmits the difference data and the moving image data for a certain switching period after the switching condition is satisfied, and the communication section 311 receives them simultaneously. The transition reproduction unit 315 performs a transition reproduction process of gradually changing the image generated by one of the difference data and the moving image data received at the same time to the image generated by the other. Hereinafter, an operation mode for executing this transition reproduction process is referred to as a "transition mode". Specifically, in the transition mode, the transition reproducing unit 315 performs transparency processing based on the first transmittance α1 on the frames generated from the difference data to generate the first transparent images, and the frames generated from the moving image data. is subjected to transparency processing based on a second transmittance α2 (that is, α2=1−α1) that is the complement of the first transmittance α1 to generate a second transmitted image, and the first transmitted image and the second transmitted image are generated. Synthesize to generate a synthetic image, that is, perform alpha blending. By changing the first transmittance α1 from 1 to 0 or from 0 to 1 (that is, the second transmittance α2 from 0 to 1 or from 1 to 0) from the beginning to the end of the switching period, the difference The image generated by one of the data and moving image data changes stepwise to the image generated by the other.

The transition reproduction unit 315 performs the transition reproduction process between the first period during which one of the first video reproduction unit 312 and the second video reproduction unit 313 reproduces the video and the second period during which the other reproduces the video. is executed during the switching period provided in That is, when switching from the difference mode to the moving image mode, at the end of the first period, that is, at the beginning of the switching period, the first transmittance α1 is set to 1, the second transmittance α2 is set to 0, and α-blending is started. The first transmittance α1 is brought close to 0 and the second transmittance α2 is brought close to 1, and α-blending is performed sequentially. The transmittance α2 is set to 1 and α-blending is performed. On the other hand, when switching from the moving image mode to the difference mode, at the end of the first period, that is, at the beginning of the switching period, the first transmittance α1 is set to 0, the second transmittance α2 is set to 1, and α-blending is started. The first transmittance α1 is brought close to 1 and the second transmittance α2 is brought close to 0, and the α-blending is sequentially executed. α-blending is performed with the transmittance α2 set to 0.

During the switching period, the audio reproducing unit 314 gradually reduces the level of the audio reproduced along with the video reproduced by one of the first video reproducing unit and the second video reproducing unit, and reproduces the audio by the other. The level of the sound reproduced along with the video being played is increased step by step. That is, the sound reproduced with the video reproduced in the first period is faded out during the switching period, and the sound reproduced with the video reproduced during the second period is faded in during the switching period.

[Operation of remote playback system]
Next, the operation of remote playback system 100 according to the present embodiment will be described. In the remote playback system 100 according to the present embodiment, the content transmission device 200 and the playback device 300 use TCP (Transmission Control Protocol), which is a connection-oriented protocol, and UDP (User Datagram Protocol), which is a connectionless protocol. Contents are transmitted and received via a wireless LAN. UDP is used for sending and receiving audio data, and TCP is used for sending and receiving differential data and video data. The reproduction device 300 reproduces the received content and causes the display device 400 to output video and audio.

7A to 7D are flowcharts showing procedures for remote playback of content by the remote playback system according to the present embodiment. As an example, let us consider a case where a school teacher (hereinafter referred to as a “user”) teaches a class. The user activates the content transmission program 210 of the content transmission device 200 and also activates the educational program 230 . When the educational program 230 is executed, a moving image or a presentation slide is reproduced, and images are displayed on the display unit 202 of the reproduction device 200 . Also, audio attached to the moving image or slide is output from the speaker 203 .

When the content transmission program 210 is executed, the CPU 261 first operates in difference mode. The CPU 261 captures the image displayed on the display unit 202 and generates image data for the first one frame or several frames (step S101). Next, the CPU 261 generates TCP packets from the generated video data. At this time, a TCP packet is generated by adding a TCP header to the uncompressed video data. Further, the CPU 261 adds an IP header to the TCP packet to generate an IP packet, and further adds a MAC header, a physical header, etc. to generate a wireless LAN frame containing video data (hereinafter referred to as "initial video packet"). (Step S102). Note that the destination of this initial video packet is the IP address of the playback device 300 . The CPU 261 causes the wireless communication unit 201 to transmit the generated initial video packet by TCP (step S103). The transmitted initial video data is received by the playback device 300 (step S104).

Next, the CPU 261 captures the video displayed on the display unit 202 and the audio output from the speaker 203, and generates video data and audio data (step S105). The CPU 261 generates differential data between the previous frame (that is, video data transmitted last time, hereinafter referred to as the "previous frame") and the frame captured this time (hereinafter referred to as the "current frame") (step S106). ). In this process, differences in luminance values (RGB values) between corresponding pixels in the previous frame and the current frame are calculated, and difference data is generated based on the calculation results. FIG. 8 is a diagram for explaining the process of generating difference data. The previous frame 501 and the current frame 502 have the same luminance value of each pixel except for the difference portion 503 . Therefore, when the difference in the luminance value of each pixel between the previous frame 501 and the current frame 502 is taken, the value becomes 0 except for the difference portion 503 . The CPU 261 generates difference data by removing information on pixels with a luminance difference of 0. FIG. Therefore, the difference data contains only the information of the difference part 503, and the amount of information is suppressed.

The CPU 261 generates TCP packets from the generated difference data. At this time, a TCP header is added to the uncompressed differential data to generate a TCP packet. Furthermore, the CPU 261 adds an IP header to the TCP packet to generate an IP packet, and further adds a MAC header, a physical header, etc. to generate a wireless LAN frame containing differential data (hereinafter referred to as a "differential packet"). step S107). The destination of the differential packet is also the IP address of the playback device 300 . The CPU 261 causes the wireless communication unit 201 to transmit the generated difference packet by TCP (step S108).

The CPU 261 also adds a UDP header to the audio data in linear PCM (Pulse Code Modulation) format to generate a UDP packet. The linear PCM format is an uncompressed audio data format. The CPU 261 adds an IP header to the UDP packet to generate an IP packet, and further adds a MAC header, a physical header, etc. to generate a wireless LAN frame containing audio data (hereinafter referred to as "audio packet") ( step S109). Such voice packets contain time stamps, which are time information. Also, the destination of the voice packet is the IP address of the playback device 300 . The CPU 261 causes the wireless communication unit 201 to transmit the generated voice packet by UDP (step S110).

The reproduction device 300 receives the difference packet and the audio packet transmitted from the content transmission device 200 (steps S111 and S112). The CPU 331 of the playback device 300 extracts the difference data from the received difference packet, replaces the locations (pixels) with the difference data from the previous frame with the difference data, and generates video data of the current frame (step S113). is reproduced (step S114). Specifically, CPU 261 writes the generated video data to first frame memory 341 and outputs the video data in first frame memory 341 to display device 400 from external output unit 304 as a video signal. As a result, video reproduction is performed in the difference mode.

Also, the CPU 331 extracts audio data from the received audio packet and reproduces the audio (step S115). That is, the CPU 261 generates an audio signal from the audio data and outputs it from the external output unit 304 to the display device 400 . As described above, video is displayed on the display unit 401 of the display device 400 and audio is output from the speaker 402 .

In display device 400, a user may provide input to the screen with a stylus or finger. The input unit 403 accepts the input given by the user in this way, and transmits the input information to the playback device 300 through the external connection terminal 405 . CPU 331 of reproduction device 300 determines whether or not an input has been given by the user (step S116), and if an input has been given (YES in step S116), the input information is transmitted to content transmission device 200 ( step S117). On the other hand, if the user has not given an input (NO in step S116), CPU 331 moves the process to step S120.

Upon receiving the input information, the content transmission device 200 reflects the input by the user in the image and displays it. For example, when the user performs handwriting input on the screen of the display device 400 , an image including the input characters is displayed on the display unit 202 of the playback device 200 .

The CPU 261 of the content transmission device 200 determines whether or not a first switching condition, which is a switching condition for transmission from difference data to moving image data, is satisfied (step S118).

Here, the first switching condition will be explained. The first switching condition includes the following two conditions (1) to (2), and the first switching condition is considered to be satisfied when one of these conditions is satisfied.
(1) The image change amount exceeds a predetermined reference value.
(2) The user gives an instruction to switch to the moving image mode.

Condition (1) will be explained. The CPU 261 of the content transmission device 200 detects pixels having different luminance values (RGB values) between the previous frame and the current frame, counts the number of pixels, and obtains the amount of video variation. In difference mode, the amount of data communication is small, but it is difficult to display moving images smoothly. On the other hand, in moving picture mode, the time required for data compression processing increases. For this reason, the moving image mode is suitable for images that change greatly over time, and the differential mode is suitable for images that change less over time. Therefore, the content transmission device 200 detects the amount of change in the image, and if the amount of change in the image exceeds a predetermined reference value, switches the object of transmission from difference data to moving image data.

Condition (2) will be explained. In this embodiment, a user operation for instructing switching from difference mode to moving image mode is defined. For example, the user operation is an operation of tracing the screen (that is, the input unit 403) in a specific direction, an operation of selecting a selection unit for selecting switching from the difference mode to the video mode included in the display screen, or the like. is realized by When this operation input is given to the input unit 403, the CPU 261 of the content transmission device 200 is given an instruction to switch from the differential mode to the video mode.

If the above first switching condition is not satisfied (NO in step S118), CPU 261 returns the process to step S105. As a result, the processes of steps S105 to S110 and step S118 are repeated, and the difference data and the voice data are continuously transmitted. On the other hand, if the first switching condition is satisfied (YES in step S118), CPU 261 sends a request for switching the reception target from differential data to video data (hereinafter referred to as "reception target switching request") to playback device 300. It transmits (step S119).

The CPU 331 of the playback device 300 determines whether or not a reception target switching request has been received (step S120). If reproduction device 300 has not received a reception target switching request (NO in step S120), CPU 331 returns the process to step S111. As a result, the processes of steps S111 to S117 and step S120 are repeated, and video reproduction based on the difference data and audio reproduction based on the audio data are continuously performed.

After transmitting the receiving target switching request, the CPU 261 of the content transmitting device 200 transmits both the difference data and the moving image data to the reproducing device 300 in a predetermined switching period as follows. First, the CPU 261 starts a switching period (step S121). Next, the CPU 261 captures the video displayed on the display unit 202 and the audio output from the speaker 203, and generates video data and audio data (step S122).

Similar to steps S106 to S108, CPU 261 generates difference data (step S123), generates a difference packet (step S124), and causes the difference packet to be transmitted to wireless communication section 201 by TCP (step S125).

Also, the CPU 261 converts a plurality of temporally continuous frames into H.264, which is a video compression standard. The video data is encoded by the video compression method defined in H.264 to generate video data (step S126). Note that in this embodiment, the H.264 standard is used. 264 format video data is described, but the present invention is not limited to this, and may be configured to generate video data in other video compression formats, such as Divx, Xvid, VP9, MPEG-4, etc. .

The CPU 261 generates TCP packets from the generated video data. Furthermore, the CPU 261 adds an IP header to the TCP packet to generate an IP packet, and further adds a MAC header, a physical header, etc. to generate a wireless LAN frame containing moving image data (hereinafter referred to as a "moving image packet"). step S127). The moving picture packet includes a time stamp, which is moving picture time information. Also, the destination of the moving picture packet is the IP address of the playback device 300 . The CPU 261 causes the wireless communication unit 201 to transmit the generated video packet by TCP (step S128).

CPU 261 also generates voice packets (step S129) in the same manner as in steps S109 to 110, and causes wireless communication section 201 to transmit the voice packets by UDP (step S130).

CPU 261 determines whether or not the switching period has ended (step S131), and if not (NO in step S131), returns the process to step S122. As a result, both the difference data and the video data are continuously transmitted from the content transmission device 200 until the switching period ends.

On the other hand, if the reception target switching request has been received by playback device 300 (YES in step S120), CPU 331 starts the switching period (step S132). Further, the reproducing device 300 receives each of the difference packet, moving image packet, and audio packet transmitted from the content transmitting device 200 (steps S133 to S135). The CPU 331 sets the first transmittance α1, the second transmittance α2, and the audio output level SL (step S136). The initial value of the first transmittance α1 is "1" and the initial value of the second transmittance α2 is "0" when switching from the difference mode to the moving image mode. Also, the initial value of the audio output level SL is the same value as the audio output level in the difference mode.

Next, the CPU 331 extracts difference data from the received difference packet, adds the luminance values of the previous frame and the difference data for each pixel, and generates video data of the current frame (step S137). The generated video data is written to the first frame memory 341 . Furthermore, the CPU 331 multiplies the luminance value of each pixel of the video data stored in the first frame memory 341 by the first transmittance α1 to generate a first transparent image (step S138).

Further, the CPU 331 extracts moving image data from the received moving image packet, decodes the moving image data, and generates frame image data (step S139). The generated video data is written to the second frame memory 342 . Furthermore, the CPU 331 multiplies the luminance value of each pixel of the video data stored in the second frame memory 342 by the second transmittance α2 to generate a second transparent image (step S140).

CPU 331 synthesizes the first transparent image and the second transparent image, and outputs a video signal of the synthesized image from external output unit 304 to display device 400 (step S141). As a result, video playback is performed in the transition mode.

The CPU 331 also extracts audio data from the received audio packet and reproduces the audio at the set audio output level SL (step S142). That is, the CPU 261 generates an audio signal from the audio data and outputs it from the external output unit 304 to the display device 400 .

Next, CPU 331 determines whether or not the switching period has ended (step S143), and if the switching period has not ended (NO in step S143), the process returns to step S133. As a result, the processing of steps S133 to S143 is repeated, and the α-blending of the video is continuously executed. During this switching period, the first transmittance α1, the second transmittance α2, and the audio output level SL are gradually changed.

FIG. 9 is a graph showing changes in the first transmittance α1 and the second transmittance α2 when switching from the differential mode to the moving image mode. In FIG. 9, the vertical axis indicates transmittance and the horizontal axis indicates time. As shown in FIG. 9, the first transmittance α1 is 1 and the second transmittance α2 is 0 at the beginning of the switching period. As the switching period progresses, the first transmittance α1 gradually decreases and the second transmittance α2 gradually increases. At the end of the switching period, the first transmittance α1 becomes 0 and the second transmittance α2 becomes 1. By changing the first transmittance α1 and the second transmittance α2 in this manner, the image based on the differential data is gradually switched to the image based on the moving image data during the switching period.

FIG. 10 is a graph showing changes in the audio output level SL when switching from the differential mode to the moving image mode. In FIG. 10, the vertical axis indicates the audio output level SL, and the horizontal axis indicates time. As shown in FIG. 10, the audio output level SL is maximum at the beginning of the switching period, gradually decreases as the switching period progresses, and becomes 0 in the middle of the switching period. . This causes the audio to fade out. Further, as the time progresses further from the middle of the switching period, the audio output level SL gradually increases and reaches the maximum at the end of the switching period. This causes the audio to fade in. Also, the CPU 331 synchronizes the audio with the video of the moving image data by using the time stamp added to the audio data and the time stamp added to the moving image data.

See FIG. 7B. When the switching period has ended (YES in step S131), content transmission device 200 transmits moving image packets and audio packets to reproduction device 300 as follows. First, the CPU 261 captures the video displayed on the display unit 202 and the audio output from the speaker 203, and generates video data and audio data (step S144). Furthermore, CPU 261 generates moving image data (step S145), generates moving image packets (step S146), and causes the moving image packets to be transmitted to wireless communication unit 201 by TCP (step S147), in the same manner as steps S126 to S128.

CPU 261 also generates voice packets (step S148) in the same manner as in steps S109-110, and causes wireless communication section 201 to transmit the voice packets by UDP (step S149).

On the other hand, if the switching period has ended in reproduction device 300 (YES in step S143), reproduction device 300 receives the moving image packets and audio packets transmitted from content transmission device 200 (steps S150 and S151). The CPU 331 of the reproduction device 300 extracts the moving image data from the received moving image packet, decodes the moving image data to generate frame image data (step S152), and reproduces the image (step S153). Specifically, CPU 261 writes the generated video data to second frame memory 342 and outputs the video data in second frame memory 342 to display device 400 from external output unit 304 as a video signal. As a result, video reproduction is performed in the moving picture mode.

CPU 331 also extracts audio data from the received audio packet and reproduces the audio (step S154). That is, the CPU 261 generates an audio signal from the audio data and outputs it from the external output unit 304 to the display device 400 . At this time, the CPU 331 uses the time stamp added to the moving image data and the time stamp added to the audio data to synchronize the audio with the video. As described above, video is displayed on the display unit 401 of the display device 400 and audio is output from the speaker 402 .

CPU 331 determines whether or not an input has been given by the user to input unit 403 (step S155), and if an input has been given (YES in step S155), the input information is transmitted to content transmission device 200. (step S156). On the other hand, if the user has not given an input (NO in step S155), CPU 331 shifts the process to step S159.

The CPU 261 of the content transmission device 200 determines whether or not a second switching condition, which is a switching condition for transmission from moving image data to differential data, is satisfied (step S157).

Here, the second switching condition will be explained. The second switching condition includes the following three conditions (3) to (5), and the second switching condition is satisfied when any one of these conditions is satisfied.
(3) The predetermined first determination period is exceeded without the image change amount exceeding the predetermined reference value.
(4) A user's input to the screen is detected continuously for a predetermined second determination period or longer.
(5) The user gives an instruction to switch to the differential mode.

Condition (3) will be explained. As described above, the moving image mode is suitable for images that change greatly over time, and the difference mode is suitable for images that change less over time. Therefore, the content transmission device 200 detects the amount of video change, and if the amount of video change does not exceed the reference value continuously for the first determination period or longer, the transmission target is switched from moving image data to difference data.

Condition (4) will be explained. CPU 261 of content transmission device 200 monitors operations on input unit 403 based on input information given from playback device 300 . The CPU 331 detects a predetermined input to the image when the input unit 403 is operated continuously for the second determination period or longer. For example, the user may use the display device 400 as an electronic blackboard and draw characters on the screen with a stylus, that is, perform handwriting input. For such handwritten input, it is important to reflect the user's input on display with as little delay as possible. For this reason, video playback using differential data with less delay is more suitable than video playback using moving image data that causes a delay in video playback. Therefore, when the content transmission device 200 detects a user input to the video continuously for the second determination period or longer, the content transmission device 200 switches the transmission target from the moving image data to the difference data.

Condition (5) will be explained. In this embodiment, a user operation for instructing switching from the moving image mode to the difference mode is defined. For example, the user operation is an operation of tracing the screen (that is, the input unit 403) in a specific direction, or an operation of selecting a selection unit for selecting switching from the moving image mode to the difference mode included in the display screen. is realized by When this operation input is given to the input unit 403, the CPU 261 of the content transmission device 200 is given an instruction to switch from the moving image mode to the differential mode.

If the above second switching condition is not satisfied (NO in step S157), CPU 261 returns the process to step S144. As a result, the processes of steps S144 to S149 and step S157 are repeated, and moving image data and audio data are continuously transmitted. On the other hand, if the second switching condition is satisfied (YES in step S157), CPU 261 transmits a reception target switching request (reception target switching request) from moving image data to difference data to playback device 300 (step S158). ).

The CPU 331 of the playback device 300 determines whether or not the reception target switching request has been received (step S159). If reproducing device 300 has not received the reception target switching request (NO in step S159), CPU 331 returns the process to step S150. As a result, the processes of steps S150 to S156 and step S159 are repeated, and video reproduction based on moving image data and audio reproduction based on audio data are continuously performed.

After transmitting the reception target switching request, the CPU 261 of the content transmission device 200 transmits the difference data, the moving image data, and the audio data to the playback device 300 in a predetermined switching period in the same manner as in steps S121 to S131 (step S160 ~S170).

On the other hand, if a reception target switching request is received by reproduction device 300 (YES in step S120), CPU 331 starts a switching period (step S171). Further, the reproducing device 300 receives each of the difference packet, moving image packet, and audio packet transmitted from the content transmitting device 200 (steps S172 to S174). The CPU 331 sets the first transmittance α1, the second transmittance α2, and the audio output level SL (step S175). When the moving image mode is switched to the difference mode, the initial value of the first transmittance α1 is "0" and the initial value of the second transmittance α2 is "1". Also, the initial value of the audio output level SL is the same value as the audio output level in the moving picture mode.

Next, the CPU 331 extracts difference data from the received difference packet, adds the luminance values of the previous frame and the difference data for each pixel, and generates video data of the current frame (step S176). The generated video data is written to the first frame memory 341 . Furthermore, the CPU 331 multiplies the luminance value of each pixel of the video data stored in the first frame memory 341 by the first transmittance α1 to generate a first transparent image (step S177).

Further, CPU 331 extracts moving image data from the received moving image packet, decodes the moving image data, and generates frame image data (step S178). The generated video data is written to the second frame memory 342 . Furthermore, the CPU 331 multiplies the luminance value of each pixel of the video data stored in the second frame memory 342 by the second transmittance α2 to generate a second transparent image (step S179).

CPU 331 synthesizes the first transparent image and the second transparent image, and outputs a video signal of the synthesized image from external output unit 304 to display device 400 (step S180). As a result, video playback is performed in the transition mode.

The CPU 331 also extracts audio data from the received audio packet and reproduces the audio at the set audio output level SL (step S181). In other words, the CPU 331 generates an audio signal from the audio data and outputs it from the external output unit 304 to the display device 400 .

Next, CPU 331 determines whether or not the switching period has ended (step S182), and if the switching period has not ended (NO in step S182), the process returns to step S172. As a result, the processing of steps S172 to S182 is repeated, and the α-blending of the video is continuously executed. During this switching period, the first transmittance α1, the second transmittance α2, and the audio output level SL are gradually changed.

FIG. 11 is a graph showing changes in the first transmittance α1 and the second transmittance α2 when switching from the moving image mode to the difference mode. In FIG. 11, the vertical axis indicates transmittance and the horizontal axis indicates time. As shown in FIG. 11, the first transmittance α1 is 0 and the second transmittance α2 is 1 at the beginning of the switching period. As the switching period progresses, the first transmittance α1 gradually increases and the second transmittance α2 gradually decreases. At the end of the switching period, the first transmittance α1 becomes 1 and the second transmittance α2 becomes 0. By changing the first transmittance α1 and the second transmittance α2 in this manner, the image based on the moving image data is gradually switched to the image based on the differential data during the switching period.

During the switching period from the moving image mode to the difference mode, the sound output level SL changes similarly to the switching period from the difference mode to the moving image mode. That is, the audio output level SL is maximum at the beginning of the switching period, gradually decreases as the switching period progresses, and becomes 0 in the middle of the switching period. This causes the audio to fade out. Further, as the time progresses further from the middle of the switching period, the audio output level SL gradually increases and reaches the maximum at the end of the switching period. This causes the audio to fade in. Also, the CPU 331 synchronizes the audio with the video of the moving image data by using the time stamp added to the audio data and the time stamp added to the moving image data.

See FIG. 7D. When the switching period has ended (YES in step S170), CPU 261 of content transmission device 200 returns the process to step S105. As a result, the transmission target is switched from the moving image packet to the differential packet, and the differential packet and the audio packet are transmitted to the playback device 300 (steps S105 to S110).

On the other hand, if the switching period has ended in reproducing device 300 (YES in step S182), CPU 331 of reproducing device 300 returns the process to step S111. As a result, the mode is shifted to the difference mode, and the video is reproduced by the difference data.

With the above configuration, switching between the difference mode in which the image quality is low but the amount of data processed per unit time is small and the video mode in which the image quality is high but the amount of data processed per unit time is large can be switched. Both communication volume and high image quality can be achieved. In addition, since the video and the audio are synchronized in the moving image mode where the video and the audio are likely to be out of sync, the video and the audio can be prevented from being out of sync. In the difference mode, the amount of data is small, and the possibility of reproduction of video reproduction processing is small. Therefore, even if synchronization is not performed, the difference between video and audio is small.

In addition, the content transmission device 200 generates audio packets separately from the differential packets and the video packets, and transmits the audio packets when transmitting the differential packets and the video packets. That is, the audio data is shared between the difference mode and the moving image mode, and the audio data dedicated to the difference mode and the audio data dedicated to the moving image mode are not generated. Therefore, it is necessary to reduce the amount of communication without duplication of audio data for reproducing the same audio, and to generate audio data in different formats for each of the difference mode and the video mode. Therefore, the configuration of the content transmission device 200 is simplified. Similarly, the playback device 300 does not need to process different formats of audio data, and the configuration is simplified.

(Other embodiments)
In the above-described embodiment, a switching period is provided for switching the operation mode between the difference mode and the moving image mode, and during this switching period, the image reproduction by one of the difference data and the moving image data is gradually switched to the image reproduction by the other. Although the configuration for switching to is described, it is not limited to this. It is also possible to adopt a configuration in which the difference mode is directly switched to the moving image mode, or the moving image mode is directly switched to the difference mode without providing a switching period.

Also, in the above-described embodiment, the configuration in which the content transmission device 200 transmits both the difference data and the moving image data to the reproduction device 300 during the switching period has been described, but the configuration is not limited to this. For example, when the transmission target is switched, the data of the transmission target before switching may not be transmitted, and the data of the transmission target after switching may be transmitted. In this case, the video of the last one frame reproduced by the data before switching and the video generated by the new data to be transmitted received after switching are gradually switched by α-blending during the switching period. can be done.

In the above-described embodiment, the first switching condition includes conditions (1) and (2), and the second switching condition includes conditions (3) to (5). is not limited to The first switching condition may include only one of the conditions (1) and (2), and the second switching condition includes only one or two of the conditions (3) to (5). You can Also, the first switching condition and the second switching condition may include conditions other than conditions (1) and (2) and conditions (3) to (5).

Further, in the above-described embodiment, during the switching period, the audio reproduced with the video reproduced by one of the difference data and the moving image data is faded out, and the audio reproduced with the video reproduced by the other is faded out. has been described, but the configuration is not limited to this. It is also possible to adopt a configuration in which the sound is switched at one point in the switching period without increasing or decreasing the output level SL of the sound reproduced along with the video reproduced by the difference data and the moving image data. However, the configuration in which the audio is faded out and faded in is preferable because it can suppress the viewer's sense of discomfort.

In addition, in the above-described embodiment, the content transmission device 200 is provided with the functions of the change amount detection unit 217, the input detection unit 218, the switching instruction reception unit 219, the transmission target switching unit 220, and the switching request unit 221. Although stated, it is not limited to this. The reproduction device 300 is provided with the functions of a change amount detection unit, an input detection unit, a switching instruction reception unit, and a switching request unit, and detects the amount of change in the video reproduced by the reproduction device 300 and the input to the video from the user. , or when an instruction to switch video reproduction from the user is received, a request for switching the transmission target is transmitted to the content transmission device 200, and when the content transmission device 300 receives the switching request, the transmission target is changed. It is also possible to switch from one of the data and the video data to the other.

Also, in the above-described embodiment, a configuration has been described in which audio data is transmitted by UDP, and difference data and video data are transmitted by TCP, but the present invention is not limited to this. Audio data may be transmitted by TCP, and one or both of differential data and video data may be transmitted by UDP. However, by transmitting the difference data and video data by TCP, which performs response confirmation and retransmission processing, it is possible to suppress the interruption or disturbance of the video and reproduce high-quality video, so the difference data and video data Preferably, TCP is used for transmission.

Also, in the above-described embodiment, a configuration has been described in which non-compressed difference data and audio data are transmitted, and compressed moving image data is transmitted, but the present invention is not limited to this. One or both of the difference data and the audio data may be in compressed format, and the moving image data may be in non-compressed format. However, since the difference data and the audio data are in non-compressed format, there is no need for decompression processing in the playback device, and the playback load is small, so delays in video and audio between the content transmission device 200 and the display device 400 are suppressed. be able to. Also, by compressing the moving image data, the bit rate of the video can be improved, and high-quality video can be reproduced while suppressing the amount of communication.

Also, in the above-described embodiment, a configuration in which playback device 300 is used as a wireless access point has been described, but the present invention is not limited to this. For example, a playback device dedicated to playback of content may be configured to be capable of executing the above-described operations. Further, it is also possible to provide a playback device and a display device 400 separately, and instead of connecting them together, the playback device may include a display unit. The content transmission device 200 and the playback device 300 may not be configured to communicate via a wireless LAN, and may be configured to perform wired communication.

A playback device, a remote playback system, a playback method, and a computer program according to the present invention are a playback device, a remote playback system, a playback method, and a computer program that make a computer function as a playback device for playing back video and audio distributed by streaming. Useful.

REFERENCE SIGNS LIST 100 remote playback system 200 content transmission device 201 wireless communication unit 205 storage unit 206 control unit 210 content transmission program 217 change amount detection unit 218 input detection unit 219 switching instruction receiving unit 220 transmission target switching unit 261 CPU
262 memory 300 playback device 301 wireless communication unit 303 control unit 311 communication unit 312 first video playback unit 313 second video playback unit 314 audio playback unit 315 transition playback unit 331 CPU
341 first frame memory 342 second frame memory 400 display device

Claims (12)

a communication unit that receives difference data between two temporally preceding and following frames and moving image data;
a first video reproduction unit that reproduces video from a subsequent frame generated based on the difference data and the previous frame of the two frames;
a second video reproduction unit that reproduces video from a plurality of temporally continuous frames generated based on the moving image data;
with
The communication unit further receives a request to switch a reception target from one of the differential data and the moving image data to the other from a transmission source of the difference data and the moving image data,
The first video reproducing unit and the second video reproducing unit further reproduce video from one of the first video reproducing unit and the second video reproducing unit to the other when the switching request is received by the communication unit. toggling the
The communication unit receives the differential data using a connection-oriented protocol,
The communication unit receives the uncompressed difference data.
playback device. The communication unit receives the video data using a connection-oriented protocol.
2. The playback device according to claim 1 . The communication unit receives the compressed video data,
3. The playback device according to claim 2 . a communication unit that receives difference data between two temporally preceding and following frames and moving image data;
a first video reproduction unit that reproduces video from a subsequent frame generated based on the difference data and the previous frame of the two frames;
a second video reproduction unit that reproduces video from a plurality of temporally continuous frames generated based on the moving image data;
with
The communication unit further receives a request to switch a reception target from one of the differential data and the moving image data to the other from a transmission source of the difference data and the moving image data,
The first video reproducing unit and the second video reproducing unit further reproduce video from one of the first video reproducing unit and the second video reproducing unit to the other when the switching request is received by the communication unit. toggling the
the communication unit receives both the difference data and the moving image data at the same time when playback of the video is switched;
are simultaneously received in a predetermined switching period provided between a first period during which one of the first video reproducing unit and the second video reproducing unit reproduces video and a second period during which the other reproduces video; further comprising a transition reproducing unit that reproduces an image generated by one of the difference data and the moving image data while gradually changing the image generated by the other to an image generated by the other ;
having a first frame memory and a second frame memory,
The transition reproduction unit stores in the first frame memory a first transparent image obtained by performing transparency processing based on a predetermined first transmittance on an image generated from the difference data, and stores the moving image data. storing in the second frame memory a second transparent image obtained by performing transparency processing based on a second transmittance, which is a complement of the first transmittance, on the image generated by and the second transmission image,
playback device. a content transmission device that transmits differential data between two temporally preceding and following frames and moving image data;
with a playback device and
The playback device
a communication unit that receives the difference data and the video data;
a first video reproduction unit that reproduces video from a subsequent frame generated based on the difference data and the previous frame of the two frames;
a second video reproduction unit that reproduces video from a plurality of temporally continuous frames generated based on the moving image data;
with
The communication unit further receives from the content transmission device a request to switch a reception target from one of the difference data and the moving image data to the other,
The first video reproducing unit and the second video reproducing unit further reproduce video from one of the first video reproducing unit and the second video reproducing unit to the other when the switching request is received by the communication unit. toggling the
The content transmission device is
a change amount detection unit that detects a temporal change amount of the reproduced video;
a transmission target switching unit that switches a transmission target from one of the difference data and the video data to the other based on the temporal change amount of the video detected by the change amount detection unit;
a switching request unit that requests the playback device to switch a reception target from one of the differential data and the video data to the other;
having
Remote playback system. The content transmission device is
an input detection unit that detects an input to the image to be displayed;
a transmission target switching unit that switches a transmission target from one of the difference data and the video data to the other based on detection of the input by the input detection unit;
a switching request unit that requests the playback device to switch a reception target from one of the differential data and the video data to the other;
6. The remote playback system of claim 5 . The content transmission device is
a switching instruction reception unit that receives a switching instruction between video reproduction based on the difference data and video reproduction based on the moving image data;
a transmission target switching unit that switches a transmission target from one of the difference data and the video data to the other based on the switching instruction accepted by the switching instruction accepting unit;
a switching request unit that requests the playback device to switch a reception target from one of the differential data and the video data to the other;
6. The remote playback system of claim 5 . receiving difference data indicating a difference between two temporally preceding and succeeding frames;
Reproduce video with a later frame generated based on the received difference data and the previous frame of the two frames;
receiving moving image data when switching from the image based on the difference data to the image based on moving image data for generating a plurality of temporally continuous frames;
Stopping the reproduction of the video by the difference data,
playing back video from a plurality of temporally consecutive frames generated based on the received video data;
A playback method comprising:
receiving the differential data includes receiving the differential data using a connection-oriented protocol;
receiving the differential data includes receiving the uncompressed differential data;
how to play. receiving difference data indicating a difference between two temporally preceding and succeeding frames;
Reproduce video with a later frame generated based on the received difference data and the previous frame of the two frames;
receiving moving image data when switching from the image based on the difference data to the image based on moving image data for generating a plurality of temporally continuous frames;
Stopping the reproduction of the video by the difference data,
playing back video from a plurality of temporally consecutive frames generated based on the received video data;
A playback method comprising:
receiving both the difference data and the moving image data at the same time when playback of the video is switched;
During a predetermined switching period provided between a first period during which the image based on the difference data is reproduced and a second period during which the image based on the moving image data is reproduced, the image based on the difference data is changed to the image based on the moving image data. Playback with gradual changes,
The step-by-step reproduction includes storing in a first frame memory a first transparent image obtained by performing transparency processing based on a predetermined first transmittance on an image generated from the differential data, and storing in a second frame memory a second transparent image obtained by performing transparency processing based on a second transmittance, which is a complement of the first transmittance, on an image generated from moving image data; compositing the image with the second transmission image;
how to play. Receiving differential data between two temporally preceding and following video frames and video data,
receiving a request to switch a reception target from one of the differential data and the moving image data to the other from a transmission source of the difference data and the moving image data;
Reproduction of a first video image by a subsequent frame generated based on the difference data and the previous frame of the two frames, and a plurality of temporally continuous frames generated based on the moving image data. and switching playback of the video from one to the other when the switch request is received.
A playback method comprising:
receiving the differential data includes receiving the differential data using a connection-oriented protocol;
receiving the differential data includes receiving the uncompressed differential data;
how to play. Receiving differential data between two temporally preceding and following video frames and video data,
receiving a request to switch a reception target from one of the differential data and the moving image data to the other from a transmission source of the difference data and the moving image data;
Reproduction of a first video image by a subsequent frame generated based on the difference data and the previous frame of the two frames, and a plurality of temporally continuous frames generated based on the moving image data. and switching playback of the video from one to the other when the switch request is received.
A playback method comprising:
receiving both the difference data and the moving image data at the same time when playback of the video is switched;
One of the first video and the second video is switched to the other in a predetermined switching period provided between a first period during which the first video is reproduced and a second period during which the second video is reproduced. Playback with gradual changes,
In the stepwise change reproduction, a first transmission image obtained by performing transparency processing based on a predetermined first transmittance on the image of the first video is stored in a first frame memory, and the second transmission is performed. A second transparent image obtained by performing transparency processing based on a second transmittance, which is a complement of the first transmittance, on a video image is stored in a second frame memory. Combining the 2 transmission images,
how to play. A computer program for causing a computer to function as a content transmission device that transmits content including video,
a first video transmission means for transmitting difference data indicating a difference between two temporally preceding and following frames;
second video transmission means for transmitting video data;
a change amount detection unit that detects a temporal change amount of video played back at a transmission destination;
Transmission target switching means for switching transmission targets from one of the difference data and the moving image data to the other based on the temporal change amount of the video detected by the change amount detection unit ; and functioning the computer as switching request means for requesting switching of the receiving target from one of the
computer program.

Images