JP2018125690A - Information processing apparatus, information display, information processing method, information display method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce processing load for screen data transfer of a transmission source device.SOLUTION: An information processing apparatus creates movie data, and movie control information including movie layout information indicating layout of the movie data in a display space by an information display, and the information of the reproduction time of the movie data, creates still-picture control information including the still-picture data excepting the movie data in the display space, and the screen layout information indicating layout of the still-picture data in the display space, and creates transmission data based on the movie data, movie control information, the still-picture data and the still-picture control information, before being transmitted to the information display. After transmitting the transmission data, the information processing apparatus transmits the movie control information as the transmission data.SELECTED DRAWING: Figure 1A

Description

  The present invention relates to a screen data transfer technique between an information processing apparatus and an information display apparatus.

  Conventionally, video cables such as HDMI (High-Definition Multimedia Interface) and D-Sub (D-subminiature) cables are used as a method of displaying a personal computer screen as an information processing device on a projector as an information display device. The display method via was common. In recent years, it has become widespread that a function of communicating with a projector via a network is incorporated into a personal computer. Accordingly, screen data transfer for displaying a personal computer screen on a projector via a network has become widespread. Screen data transfer is a technique in which a personal computer screen is compressed as image data on the personal computer side and transmitted to a projector connected via a network. The projector displays the screen of the personal computer by expanding and displaying the received image data.

  On the other hand, the transmission source personal computer is MPEG4 or H.264. H.264 has a function of playing back compressed video data, and a video can be viewed on a personal computer screen. For this reason, personal computers often reproduce presentation materials including moving images on the screen of the personal computer while transmitting the compressed video data attached to the presentation materials. As a technique for transferring screen data including a moving image, a screen display method for transmitting and displaying screen data has been conventionally proposed (see Patent Document 1).

JP 2012-015868 A

  When using the screen display method disclosed in Patent Document 1, the information processing apparatus as the transmission source transmits all the screen data of the displayed information processing apparatus to the information display apparatus while reproducing the moving image compressed data. Needs to be encoded. When the performance of the information processing device is not high, there is a problem that the processing capability cannot catch up with the necessary encoding processing, and as a result, frame dropping occurs during reproduction of the information display device, and smooth moving image reproduction cannot be performed.

The present invention has been made in view of such a problem, and an object of the present invention is to reduce the processing burden for screen data transfer of a transmission source device.
With the goal.

  As one means for achieving the above object, an information processing apparatus of the present invention has the following configuration. That is, the information processing device creates first moving image data, moving image layout information indicating a layout of the moving image data in a display area by the information display device, and moving image control information including information on a reproduction time of the moving image data. Means for generating still image control information including still image data excluding the moving image data in the display area, and screen layout information indicating a layout of the still image data in the display area, and the moving image Transmission means for creating transmission data based on the data, the moving image control information, the still image data, and the still image control information, and transmitting the transmission data to the information display device. After the transmission, the transmission means uses the moving picture control information created by the first creation means as the transmission data. To send.

  According to the present invention, the processing load on the transmission source device is reduced, and even when the performance of the transmission source device is low, smooth video playback is possible on the transmission destination device.

The figure which shows an example of the function structure of the information processing apparatus 100 in 1st Embodiment, and the information display apparatus 140. FIG. 2 is a diagram illustrating an example of a hardware configuration of the information processing apparatus 100. FIG. The figure which shows an example of mounting of the screen transfer system which consists of the information processing apparatus 100 and the information display apparatus 140 in 1st Embodiment. 5 is a flowchart showing the overall operation of the information processing apparatus 100. 6 is a flowchart showing an operation of transferring screen data by the information processing apparatus 100. 5 is a flowchart showing an operation of extracting moving image information by the information processing apparatus 100. 4 is a diagram showing an example of transmission data 132 transmitted from the information processing apparatus 100. FIG. (A) is a display screen of information processing apparatus 100, (b) is a figure which shows the structure of the transmission data corresponding to (a). The flowchart which shows the operation | movement which displays the transferred screen data of the information display apparatus 100 in 1st Embodiment. The figure which shows an example of a function structure of the information processing apparatus 100 and the information display apparatus 140 in 2nd Embodiment. The figure which shows an example of mounting of the screen transfer system which consists of the information processing apparatus 100 and the information display apparatus 140 in 2nd Embodiment. 9 is a flowchart showing an operation of transferring a screen of compressed video data by the information processing apparatus 100 according to the second embodiment. The flowchart which showed the operation | movement which displays the transferred screen data by the information display apparatus 100 in 2nd Embodiment.

  Hereinafter, the present invention will be described in detail based on preferred embodiments with reference to the accompanying drawings. The configurations shown in the following embodiments are merely examples, and the present invention is not limited to the illustrated configurations.

[First Embodiment]
With reference to FIG. 1A and FIG. 1B, the structure of the information processing apparatus 100 and the information display apparatus 140 in 1st Embodiment which are one Embodiment of the screen transfer system which consists of an information processing apparatus of this invention and an information display apparatus is demonstrated. To do.

  FIG. 1A is a diagram illustrating an example of functional configurations of the information processing apparatus 100 and the information display apparatus 140 according to the present embodiment. The information processing apparatus 100 (for example, a personal computer) is a data transmission source apparatus, and can generate and send a screen to be displayed on the information display apparatus 140 by an operation of a user of the information processing apparatus 100. The information display device 140 (for example, a projector) is a data transmission destination device, and can display screen data received from the information processing device 100.

  Next, the configuration of the information processing apparatus 100 in the present embodiment will be described with reference to FIG. The receiving unit 101 receives one moving image compressed data 121 that is a local file stored in the information processing apparatus 100. The receiving unit 101 generates moving image data 111 from the received moving image compressed data 121.

  The screen creation unit 102 creates a video screen by expanding the video data 111 from the reception unit 101. The screen creation unit 102 creates a background screen for the moving image screen, for example, by a user operation or using a predetermined template. Subsequently, the screen creation unit 102 displays the background screen and the layout information of the background screen (information regarding the area (position) to be laid out (displayed)), the layout information of the video screen and the video screen, and the video source information (for example, video Display screen information 112 including information for identifying the compressed data 121 is generated.

  The composition unit 103 creates a display screen 113 for display on the display unit 174 (FIG. 1B) of the information processing apparatus 100. For this purpose, the synthesis unit 103 first lays out the background screen according to the layout information of the background screen included in the display screen information 112. The combining unit 103 further generates the display screen 113 by combining the moving image screen with the background screen in accordance with the moving image screen and the moving image layout information. Further, the composition unit 103 generates screen composition information 114 by extracting and combining the background screen and the moving image screen constituting the display screen 113 and the layout information of each screen. The screen composition information 114 further includes data of the entire screen (data of the entire screen that can be displayed), layout information thereof, and moving image source information.

  The display control unit 104 displays the display screen 113 generated by the combining unit 103 on the display unit 174 of the information processing apparatus. The display control unit 104 generates a transmission source display screen 131 to be displayed in accordance with the display unit 174 of the information processing apparatus.

  The area determination unit 105 determines a transmission destination display area to be displayed on the transmission destination information display device 140 from the screen composition information 114 from the synthesis unit 103. The display area of the transmission destination may be the same area as the display screen 113 or may be a partial area of the display screen 113. Then, the area determination unit 105 generates transmission destination display area information 115 by adding the determined transmission destination display area to the screen composition information 114.

  The moving image correspondence confirmation unit 110 acquires the decoding function information 155 from the moving image correspondence management unit 147 in the information display device 140, and confirms whether or not the transmission destination supports moving image compressed data. In addition, the moving image correspondence confirmation unit 110 generates moving image correspondence information 120 indicating a confirmation result.

  The drawing content determination unit 106 displays the display unit of the transmission destination information display device 140 based on the display area information 115 of the transmission destination from the region determination unit 105 and the moving image correspondence information 120 of the transmission destination from the moving image correspondence confirmation unit 110. The type and area of data to be drawn (displayed) 174 are determined. When it is determined by the moving image correspondence information 120 that the information display device 140 does not support moving image compressed data, the drawing content determination unit 106 draws still image data on the entire display area of the display unit 174 of the information display device 140. decide. On the other hand, when it is determined that the information display device 140 is compatible with the moving image compressed data, the drawing content determination unit 106 determines to draw the moving image compressed data in the region of the moving image screen on the display unit 174 of the information display device 140. Then, it is determined to draw still image data in an area other than the moving picture screen area.

  Furthermore, the drawing content determination unit 106 determines the area of the screen that has been determined to be the video compression data to be drawn, the entire screen data, the video source information, and the video layout information (the area (position) where the video compression data 121 is laid out (displayed)). ) Is generated. Further, the drawing content determination unit 106 generates still image screen information 116 including data of the entire screen, a background screen, and layout information thereof for an area determined to be drawn still image data.

  The still image data generation unit 107 determines an area for drawing (displaying) the still image data from the still image screen information 116 from the drawing content determination unit 106, and generates still image data. For this purpose, first, the still image data generation unit 107 compares the data of the entire screen of the still image screen information 116 with the data of the entire previous screen, and specifies an area having a data difference. Initially, the entire screen is an area having a data difference. The still image data generation unit 107 generates still image data only for an area having a difference as a still image data from the layout information of the background screen. The still image data generation unit 107 uses the still image data as screen data, and the screen data, the screen data size for displaying the screen data, and the screen layout information of the screen data (area (position) to be laid out (displayed)). Screen data information 118 consisting of information on

  The moving image data generation unit 109 determines an area for drawing (displaying) moving image compression data from the moving image screen information 117 from the drawing content determination unit 106, and generates moving image compression data. For this purpose, the moving image data generation unit 109 checks the presence / absence of an area in which the moving image compressed data is displayed from the data of the entire moving image screen information 117 and the moving image layout information. Subsequently, the moving image data generation unit 109 compares the moving image source information with the previous moving image compressed data and determines whether or not they are the same. In the same case, the compressed video data has not been transmitted since it has already been transmitted. Further, when the moving image compressed data is displayed, the moving image data generation unit 109 acquires the time stamp (reproduction time information) of the moving image compressed data on the current display screen. Then, the moving image data generation unit 109 generates moving image compression data, moving image compressed data information 119 including moving image compression data size, moving image layout information, and a time stamp.

  The transmission unit 108 transmits transmission data to the information display device 140 based on the screen data information 118 generated by the still image data generation unit 107 and / or the moving image compressed data information 119 generated by the moving image data generation unit 109. 132 is generated and transmitted. For example, in order to generate the transmission data 132, the transmission unit 108 first adds the number of generated screen data information 118 as the number of screen data information. Similarly, the number of moving image compressed data information 119 is added as the number of moving image compressed data information to generate transmission data 132. The configuration of the transmission data 132 will be described later with reference to FIG.

  Next, the configuration of the information display device 140 that is a transmission destination will be described. The receiving unit 141 receives transmission data 132 transmitted from the information processing apparatus 100 from the network. The reception unit 141 also generates reception data 148 from the received transmission data 132.

  The separation unit 142 separates the reception data 148 received by the reception unit 141 into screen data information 149 including screen data and layout information that are still image data, and moving image compression data information 150 including moving image compression data and layout information. . Then, the separation unit 142 sends the separated screen data information 149 to the still image data development unit 143. Further, the separation unit 142 sends the moving image compressed data information 150 to the moving image data reproduction unit 146.

  The still image data development unit 143 receives the screen data information 149 from the separation unit 142 and develops screen data (that is, still image data) as a background screen. The still image data development unit 143 holds screen information 151 including the developed screen data (background screen) and screen layout information.

  The moving image data reproducing unit 146 receives the moving image compressed data information 150 including the moving image compressed data separated by the separating unit 142 and develops it as a moving image screen. In addition, the moving image data reproduction unit 146 holds moving image screen information 152 including the expanded moving image screen and moving image layout information. Note that the moving image data reproduction unit 146 stores, as a moving image reproduction method, presence / absence information 154 of a decompression function of moving image compressed data, which means whether a moving image compressed data decoder is installed, from the time when the information display device 140 is activated. It shall be.

  The moving image correspondence management unit 147 generates decoding function information 155 from the presence / absence information 154 of the decompression function of the moving image compressed data from the moving image data reproduction unit 146. The decoding function information 155 indicates whether or not the information display device 140 is equipped with a moving image compressed data decoder.

  The synthesizing unit 144 generates a display screen 153 from the screen information 151 from the still image data developing unit 143 and the moving image screen information 152 from the moving image data reproducing unit 146. For this purpose, the synthesis unit 144 first lays out the background screen included in the screen information 151 along the screen layout information. The combining unit 144 combines the moving image screen of the moving image screen information 152 on the background screen along with the moving image layout information. Note that the composition is to superimpose the moving image screen on the screen data, and thereby the display screen 153 is generated.

  The display control unit 145 displays the display screen 153 generated by the combining unit 144 on the display unit 174 of the information display device 140. The display control unit 145 generates a transmission destination display screen 161 to be displayed in accordance with the display destination.

  FIG. 1B is a diagram illustrating an example of a hardware configuration of the information processing apparatus 100. The hardware configuration of the information display device 140 is the same as that in FIG. 1B.

  The storage unit 171 includes a memory such as a ROM (Read Only Memory) and a RAM (Random Access Memory). The storage unit 171 stores various information such as a program for performing various operations to be described later and communication parameters for wireless communication. In addition to the memory such as ROM and RAM, the storage unit 171 is a storage medium such as a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, a nonvolatile memory card, and a DVD. May be used. The storage unit 171 may include a plurality of memories.

  The control unit 172 includes a processor such as a CPU (Central Processing Unit) and an MPU (Micro Processing Unit). The control unit 172 controls the information processing apparatus 100 as a whole by executing a program stored in the storage unit 171. The control unit 172 may include a plurality of processors such as multi-cores, and the information processing apparatus 100 may be controlled by the plurality of processors.

  The input unit 173 receives various operations from the user. The display unit 174 performs various displays on the screen. Note that both the input unit 173 and the display unit 174 may be realized by one module like a touch panel. The display unit 174 is configured by an LCD (Liquid Crystal Display), for example. The display unit 174 may output information that can be visually recognized by the user and perform display control of various UIs (User Interfaces). When the information display device 140 is configured as a projector, the display unit 174 may function as a projection unit.

  The communication unit 175 controls wireless communication or wired communication. When performing wireless communication, the communication unit 175 controls the antenna 176 to transmit and receive wireless signals for wireless communication. The information processing apparatus 100 transmits content such as video data and moving image control information to the information display apparatus 140 via the communication unit 175. When the information display device 140 is configured as a projector, content such as video data and moving image control information are received from the information processing device 100 via the communication unit 175.

  Next, an example of a method for transferring screen data between the information processing apparatus 100 and the information display apparatus 140 in the present embodiment will be described with reference to FIG.

  FIG. 2 is a diagram illustrating an example of a screen transfer system including the information processing apparatus 100 and the information display apparatus 140 according to the present embodiment. In FIG. 2, an example in which a personal computer 201 as the information processing apparatus 100 and a projector 202 as the information display apparatus 140 are connected via a network 210 in the conference room 200, and the projector 202 displays the screen of the personal computer 201. It is shown.

  The transmission source screen 220 is a screen on the display unit 174 of the personal computer 201 enlarged for explanation. The user 230 of the personal computer 201 is looking at the display unit 174 of the personal computer 201, and the same screen as the transmission source screen 220 is displayed as the transmission destination screen 203 by the projector 202. On the transmission source screen 220, a playback screen 221 for moving image compressed data is laid out.

  Here, when the performance of the personal computer 201 is remarkably low, the moving image compressed data may be reproduced only by the projector 202 without reproducing the moving image compressed data on the transmission source screen 220. In this case, the moving image compressed data reproduction screen 221 on the transmission source screen 220 of the personal computer 201 remains black, for example, but the moving image is reproduced on the moving image compressed data reproduction screen on the transmission destination screen 203 of the projector 202. For this reason, there is no need to synchronize the video playback of the transmission source and the transmission destination by the time stamp. On the other hand, when the personal computer 201 reproduces the moving image compressed data and the projector 202 also reproduces the moving image compressed data, the personal computer 201 transmits the transmission source time stamp to the projector 202. Accordingly, the motivation for reproducing the moving image is secured at the transmission source (that is, the information processing apparatus 100) and the transmission destination (that is, the information display apparatus 140), and the reproduction screen 221 of the compressed moving data of the personal computer 201 and the compressed moving data of the projector 202 are obtained. The same scene is played back synchronously on the playback screen. In the present embodiment, as in the latter example, it is assumed that the personal computer 201 reproduces moving image compressed data, and the projector 202 also reproduces moving image compressed data.

  Next, the operation of the information processing apparatus 100 in the present embodiment will be described with reference to FIGS. FIG. 3 is a flowchart showing the overall operation of the information processing apparatus 100. In FIG. 3, it is assumed that there are a plurality of information display devices 140 as transmission destinations.

  In S301, the information processing apparatus 100 determines whether to start screen transfer from the information processing apparatus 100 to the information display apparatus 140. For example, when the image transfer processing software installed in the information processing apparatus 100 is activated by a user operation or a request from the information display apparatus 140, the information processing apparatus 100 performs screen transfer to the information display apparatus 140. Judge. If it is determined not to start screen transfer (No in S301), the process returns to S301 again, and the process of S301 is repeated until it is determined to start screen transfer.

  In S <b> 302, the information processing apparatus 100 acquires the capability of the destination information display apparatus 140 to which screen data is transferred to support moving image compressed data. This is processing performed by the moving image correspondence confirmation unit 110. The moving image correspondence confirmation unit 110 acquires the decoding function information 155 from the moving image correspondence management unit 147 of the information display device 140.

  In S303, the moving image correspondence confirmation unit 110 of the information processing apparatus 100 determines from the decoding function information 155 whether a moving image compressed data decoder is installed. If it is determined that they are compatible (Yes in S303), the process proceeds to S304. On the other hand, if it is determined that they are not compatible (No in S303), the process proceeds to S305.

  In S304, the information processing apparatus 100 registers the information display apparatus 140 in the information display list for sending the moving image compressed data. In S305, the information processing apparatus 100 registers the information display apparatus 140 in the information display list for sending only still image data because the information display apparatus 140 does not support moving image compressed data. In the case of FIG. 2, since the projector 202 supports decoding (reproduction) of moving image compressed data, the process of S304 is performed.

  When only the still image data is sent as in S305, the information processing apparatus 100 divides the entire screen into fine blocks, and sends blocks having a change in the screen at regular intervals to the information display device 140 as still image data. Examples of fine blocks of the entire screen include a 32 × 32 pixel square block and an area block in which a screen change has occurred. Further, the fixed interval is, for example, an interval of 100 ms or 60 fps (fps is the number of frames per second). This is the same method as the conventional screen transfer. The information display device 140 displays the same screen as that of the transmission source information display device by expanding and displaying the still image data. When the information processing apparatus 100 reproduces moving image compressed data, the screen always changes in the area where the moving image compressed data is reproduced, so that still image data must be created at regular intervals. In this case, the information processing apparatus 100 always creates still image data in addition to reproduction of moving image compressed data. When the performance of the information processing apparatus 100 is low, problems such as generation of still image data cannot be caught up and reproduction of moving image compressed data cannot be smoothly performed due to generation of still image data occur. As a result, it is difficult to reproduce smooth moving image compressed data on the information display device 140.

  In S306, the information processing apparatus 100 determines whether or not the decoding function information 155 of all information display apparatuses as transmission destinations has been acquired. If the destination information display device 140 remains (Yes in S306), the process returns to S302. When the decoding function information 155 of all the information display devices 140 has been acquired (No in S306), the process proceeds to S307. In the case of FIG. 2, since there is one projector 202 as the information display apparatus 140 of the transmission destination, the process proceeds to S307 without returning to S302.

  In step S <b> 307, the information processing apparatus 100 performs screen transfer processing for sending the screen of the information processing apparatus 100 to the information display apparatus 140. Detailed processing will be described with reference to the sequence shown in FIG.

  In S308, the information processing apparatus 100 determines whether to end the screen transfer process. For example, when the user of the information processing apparatus 100 ends the screen transfer processing software installed in the information processing apparatus 100 or when the information processing apparatus 100 receives another stop request, the information processing apparatus 100 It is determined that the screen transfer process is finished. If the screen transfer process is not terminated (No in S308), the process returns to S307 and continues the screen transfer process. When the screen transfer process ends (Yes in S308), the process of FIG. 3 ends.

  Next, the configuration of transmission data 132 generated by the transmission unit 108 of the information processing apparatus 100 will be described with reference to FIG. FIG. 6 shows a configuration of exemplary transmission data 132 in the screen transfer system shown in FIG. Note that the transmission data 132 illustrated in FIG. 6 can also be reception data 148 that is received and generated by the reception unit 141 of the information display device 140.

  FIG. 6A shows an example of the configuration of the transmission data 132 at the beginning of screen transfer, or when the moving image compressed data or moving image layout area is changed, or when a screen other than the moving image compressed data is changed. In FIG. 6 (a), transmission data 132 is an entire screen area 601 that is an entire screen area, screen data information 603 that is still image data information, a screen data information count 602, and a moving image that is information on moving image compressed data. It consists of compressed data information 605 and moving image compressed data information number 604. The screen data information number 602 indicates the number of screen data information that is still image data information.

  In order to suppress the amount of data transferred when the screen is changed, the still image data can be divided into a plurality of areas and have still image data for each area. Therefore, a plurality of screen data information exists when the update area in the entire screen is large, and conversely, there is a case where the screen data information does not exist without the update area. Therefore, the number of screen data information needs to be managed, and is included in the transmission data as the number of screen data information. The screen data information 603 includes screen layout information 611 that is an area of still image data in the entire screen, a screen data size 612 that indicates the size of the screen data 613 that is still image data, and screen data 613. Information including the screen layout information 611 and the screen data size 612 is control information (still image control information) for the screen data 613.

  The number of pieces of moving image compression data information 604 is the same as the number of pieces of screen data information 602, and indicates the number of pieces of moving image compression data information that is information of moving image compression data (1 in this embodiment). It is included in the transmission data in order to manage a case where a plurality of moving image compressed data is reproduced or a case where there is no moving image compressed data.

  The moving image compression data information 605 includes moving image layout information 621 that is an area of moving image compression data in the entire screen, and a time stamp 622 that means reproduction timing on the transmission side of the moving image compression data. In addition, the moving image compression data information 605 includes a moving image compression data size 623 indicating the size of the moving image compression data, and moving image compression data 624 that is the moving image compression data itself. Information including the moving image layout information 621, the time stamp, and the moving image compression data size is control information (moving image control information) for the moving image compression data 624. The moving image compressed data 624 is included in the transmission data 132 at the beginning of image transfer.

  Here, consider the transmission data 132 at the start of image transfer in the case of FIG. The entire screen area 601, screen layout information 611, and moving image layout information 621 are, for example, as shown in FIG. The screen data information number 602 is 1, and the screen data size 612 in the screen data information 603 is a size obtained by compressing the still image data in the area of the screen layout information 611, and is 500 KB when JPEG compression is performed. The screen data 613 is JPEG compressed screen data itself. Since the number of pieces of moving image compressed data information 604 is 1 and the time stamp 622 is the first, the time becomes 0:00:00 00 ms. The moving image compression data size 623 is the size of moving image compression data, and becomes 15 MB by MPEG4 or H264 compression with a playback time of 1 minute. The moving image compression data 624 is the moving image compression data itself by H264 compression. As a result, the size of the transmission data 132 at the start of image transfer is 16 MB.

  FIG. 6B is an example of the configuration of the transmission data 132 when the screen other than the moving image compressed data is not changed and only the moving image compressed data layout area is changed during the screen transfer. The transmission data 132 includes an entire screen area 601 that is an entire screen area, a screen data information number 602 that is the number of information of still image data, a moving image compressed data information 605 that is information of moving image compressed data, and a moving image compressed data information number. 604. Compared with FIG. 6A, the entire screen area 601 has the same value, but the screen data information count 602 indicates that there is no screen data information because there is no screen update area other than the compressed video data. It becomes the value of. The number of moving image compression data information 604 is the same value as in FIG. 6A (1 in the present embodiment), and the moving image compression data information 605 is being played back with the moving image layout information 621 and the time because the already transmitted moving image compression data is being reproduced. Only the stamp 622 is provided. The moving image compression data size 623 indicating the size of the moving image compression data is 0 because it is the same as the already transmitted moving image compression data. The moving image layout information 621 has the same value as in FIG. 6A, but the time stamp 622 is added for the elapsed time. Assuming a frame rate of 30 fps, the time becomes 00:00:00 33 ms after 33 ms in FIG. As a result, the transmission data 132 in the first and subsequent transfer processes has a size of 1 KB.

  FIG. 7A shows an entire screen 700 as an image of the transmission source screen 220 sent from the information processing apparatus 100 as the transmission source in FIG. 2 to the information display apparatus 140. The entire screen 700 includes a moving image screen 702 that reproduces and displays the moving image compressed data, and a background screen 701 other than the moving image compressed data. The background screen 701 excluding moving image compressed data is an area that is not frequently changed, such as the background of a presentation.

  7B corresponds to FIG. 7A, and the screen entire area 711, which is the entire screen area 601 in the transmission data 132 of FIG. 6, and the values of the screen layout information 611 and the moving image layout information 621 (each screen A layout area 712 and a moving image layout area 713) are shown. The entire screen area 711 is information of the entire screen 700 and is an area (0, 0, 1920, 1200) indicated by (X start coordinate, Y start coordinate, width, height) in this embodiment. . The screen layout area 712 is layout information of the background screen 701 other than the compressed video data, and is the same area as the entire screen area 711 (0, 0, 1920, 1200). The moving image layout area 713 is layout information of the moving image screen 702 that reproduces and displays the moving image compressed data in the entire screen 700, and is (320, 300, 1280, 720). In the present embodiment, it is assumed that the moving image layout area 713 is not changed in the single screen transfer process shown in FIG.

  Note that the method for specifying each region shown in FIG. 7B is an example, and each region may be specified by a method other than such coordinates.

  Next, the operation of the screen transfer process of the process of S307 in the flowchart of FIG. 3 by the information processing apparatus 100 will be described with reference to FIG. FIG. 4 is a flowchart showing the screen data transfer operation of the information processing apparatus 100.

  In S401, the composition unit 103 of the information processing apparatus 100 stores the display screen 113 that is a screen image. The synthesizing unit 103 further generates, from the stored display screen 113, screen composition information 114 obtained by extracting the background screen and the moving image screen and the respective layout information. The screen composition information 114 further includes data on the entire screen, layout information, and moving image source information.

  S402 is processing for extracting moving image information from the screen composition information 114, and is executed by the region determining unit 105, the drawing content determining unit 106, and the moving image data generating unit 109 of the information processing apparatus 100. Detailed processing will be described with reference to the sequence shown in FIG.

  The processing from S403 to S406 is executed by the drawing content determination unit 106 of the information processing apparatus 100. In S403, the drawing content determination unit 106 compares the data of the entire screen of the previous screen composition information 114 with the data of the entire current screen. That is, the drawing content determination unit 106 compares the previous display screen 113 with the current display screen 113. In step S404, the drawing content determination unit 106 determines whether there is a difference area as a result of the comparison processing in step S403. If there is no difference area (No in S404), the process proceeds to S409. If there is a difference area (Yes in S404), the process proceeds to S405. In the case of FIG. 7, in the first transfer process, the entire screen area 711 becomes a difference area, but in the subsequent transfer process, only the moving image layout area 713 is updated, and the process proceeds to S405.

  In step S405, the drawing content determination unit 106 determines whether the difference area is only in the layout area of the moving image compressed data. That is, in the case of FIG. 7, the drawing content determination unit 106 checks whether there is a difference area within the moving image layout area 713. If the moving image compressed data is being reproduced, the difference area is within the moving image layout area 713. If the difference area is within the moving image layout area 713 (Yes in S405), the process proceeds to S406. If the difference area is included outside the moving image layout area, the drawing content determination unit 106 determines that the contents of the area other than the moving image compressed data have been updated. In this case (No in S405), the process proceeds to S407. In the case of FIG. 7, in the first transfer process, the entire screen area 711 is a difference area, so the process proceeds to S407. However, since the subsequent transfer processing is updated only in the moving image layout area 713, the processing proceeds to S406.

  In S406, when the difference area is only in the layout area of the compressed video data, the drawing content determination unit 106 determines whether the current video layout area is the same as the previous video layout area. If the moving image layout area has changed from the previous time, the drawing content determination unit 106 determines that the contents of the area other than the moving image compressed data have been updated (Yes in S406), and the process proceeds to S407. If the moving image layout area is the same as the previous time and the difference area is only in the moving image compressed data layout area, the drawing content determination unit 106 determines that the contents of the area other than the moving image compressed data are not updated ( No in S406), the process proceeds to S409. In the case of FIG. 7, since the moving image layout area 713 is not changed after the first transfer process, the process proceeds to S409.

  S407 is processing of the still image data generation unit 107 of the information processing apparatus 100. At this time, since the difference area includes areas other than the moving image compressed data, the still image data generation unit 107 compresses the difference area other than the moving image compressed data into still image data. At this time, the still image data generation unit 107 also generates screen layout area information indicating which area of the still image data is in the entire screen area. In the case of FIG. 7, the still image data generation unit 107 compresses the entire screen area 711 into still image data during the first transfer process. In addition, the still image data generation unit 107 also creates a screen layout area 712 as the entire screen area.

  S408 is processing of the transmission unit 108 of the information processing apparatus 100. The transmission unit 108 adds the still image data, the screen layout area information, and the screen data generated by the still image data generation unit 107 in S407 to the transmission data including the moving image compression data information 119 generated by the moving image data generation unit 109 in S402. The number of information is added to generate transmission data 132. In the case of FIG. 6A, the transmission unit 108 includes a screen layout information 611 that includes the screen layout area 712 created in S <b> 407, a still image data size as a screen data size 612, and a screen composed of screen data 613. Data information 603 is created. The number of created screen data information is created as the number of screen data information 602. Further, the transmission unit 108 creates transmission data 132 to which the created screen data information 603 and the number of screen data information 602 are added. The data content of the transmission data 132 is, for example, the content described in FIG. 6A, and the number of screen data information is 1 in the first transfer process.

  S409 is processing of the area determination unit 105 of the information processing apparatus 100, and stores the current screen image as the previous screen image and uses it for comparison with the next screen image.

  S410 is a process of the transmission unit 108 of the information processing apparatus 100, and transmits the transmission data 132 including the moving image compressed data information 119 to all the information display devices 140 corresponding to the moving image compressed data. In the case of FIG. 6A, in the first transfer process, the transmission unit 108 transmits the transmission data 132 of FIG. 6A to the information display device 140. Thereafter, the transmission unit 108 transmits the transmission data 132 of FIG. 6B to the information display device 140. In S411, the information processing apparatus 100 waits until the next frame and ends the process.

  Next, with reference to FIG. 5, the process of S402 in the flowchart of FIG. FIG. 5 is a flowchart showing an operation of extracting moving image information by the information processing apparatus 100 according to this embodiment.

  In S501, the area determination unit 105 of the information processing apparatus 100 determines whether there is an area in which video is reproduced in the entire screen from the screen composition information 114 including the data of the entire screen, layout information, and moving image source information. to decide. If there is an area where the video is being played back (S501 Yes), the process proceeds to S502. If there is no area where the video is played back (No in S501), the process ends. In the case of FIG. 7, since the moving image layout area 713 is in the entire screen area 711 in both the first transfer process and the subsequent transfer processes (Yes in S501), the process proceeds to S502.

  S502 is processing of the drawing content determination unit 106 of the information processing apparatus 100. The drawing content determination unit 106 determines whether or not the video source in the area where the video is being reproduced is moving image compressed data. In addition, the drawing content determination unit 106 determines from the decoding function information 155 of the transmission destination from the moving image correspondence confirmation unit 110 whether the information display device 140 of the transmission destination supports moving image compressed data. If the video source is moving image compressed data and the information display device 140 supports moving image compressed data (Yes in S502), the process proceeds to S503. Here, when the video source is a video from an external terminal such as an HDMI (registered trademark) terminal or an analog terminal, the drawing content determination unit 106 determines that the video source is not video compression data. If the information display device 140 does not support moving image compression data (No in S502), the drawing content determination unit 106 determines that moving image compression data cannot be transmitted, and ends the processing. In the cases of FIGS. 2 and 7, the moving image source is the compressed moving image data 121 that is a local file in the information processing apparatus 100 in both the first transfer processing and the subsequent transfer processing. In addition, since the information display device 140 (projector 202) also supports compressed video data, the process proceeds to S503.

  S503 to S508 are processes of the moving image data generation unit 109 of the information processing apparatus 100. In step S503, the moving image data generation unit 109 extracts the moving image compressed data of the video source being reproduced. In step S <b> 504, the moving image data generation unit 109 extracts, as moving image compressed data information, a time stamp to be reproduced from the moving image compressed data and moving image layout information that is a layout area of the moving image compressed data. In the cases of FIGS. 2 and 7, in the first transfer process, the layout area is a value (380, 300, 1280, 720) indicated by the moving image layout area 713, and the time stamp is 0: 00: 00: 00. In the subsequent transfer process, the layout area is the same as the layout area in the first transfer process, and the time stamp becomes 0: 00: 00: 33 milliseconds, and thereafter increases by 33 milliseconds. If the video has not been played yet, the time stamp shows a negative value. This is an example, and specific determination information may be added.

  In step S505, the moving image data generation unit 109 determines whether the previously extracted moving image compressed data extracted is the same as the current moving image compressed data. For example, the moving image data generation unit 109 compares the moving image compressed data with the file name or binary data. In the case of different moving image compression data (Yes in S505), the process proceeds to S506. In the case of the same moving image compression data (No in S505), the process proceeds to S507. In the case of FIG. 2, the moving image data generation unit 109 determines that the moving image compressed data is different in the first transfer process, and the process proceeds to S506. In the subsequent transfer processing, since the moving image source is the same, the processing proceeds to S507.

  In S506, since the previously extracted moving image compressed data and the current moving image compressed data are different, the moving image data generation unit 109 calculates the moving image compressed data from the new moving image compressed data, the moving image compressed data size, the time stamp, and the moving image layout information. Information and the number of compressed video data information are created as transmission data. The moving image data generation unit 109 sends the transmission data to the transmission unit 108. In the case of FIGS. 6 and 7, the moving image layout information 621 is (380, 300, 1280, 720) of the moving image layout area 713 from FIG. The time stamp 622 is 0: 00: 00: 00, the moving image compression data size 623 is 15 MB, and the moving image compression data 624 is the moving image compression data 121 itself. The moving image data generation unit 109 generates the moving image compression data information 605 configured as described above and the moving image compression data information number 604 that is the number of generations of moving image compression data information as 1, and the transmission unit 108 adds the transmission data 132 to the transmission data 132.

  In S507, since the previously extracted moving image compressed data and the current moving image compressed data are the same, the moving image compressed data is already transmitted and becomes unnecessary, and the moving image data generation unit 109 saves the time stamp and moving image compressed data information that is moving image layout information. And the number of compressed video data information are created as transmission data. The moving image data generation unit 109 sends the transmission data to the transmission unit 108. In the cases of FIGS. 6 and 7, the moving image layout information 621 is (380, 300, 1280, 720) of the moving image layout area 713 from FIG. The time stamp 622 is 0 hour 00 minute 00 second 33 milliseconds, and the moving image compressed data size 623 is 0 because the moving image compressed data is the same. The moving image data generation unit 109 generates the moving image compression data information 605 configured as described above and the moving image compression data information number 604 that is the number of generations of moving image compression data information as 1, and the transmission unit 108 adds the transmission data 132 to the transmission data 132.

  In S508, the moving image data generation unit 109 stores the moving image compressed data and moving image compressed data information 119, which are transmission data this time, as the previous moving image compressed data and moving image compressed data information, and the next moving image compressed data and moving image compressed data information Used for comparison. As described above, data to be transmitted in the screen transfer process in the information processing apparatus 100 is created.

  Next, the operation of the display method of the transfer screen data of the information display device 140 will be described with reference to FIG. FIG. 8 is a flowchart showing an operation of displaying the transferred screen data by the information display device 140 according to the present embodiment. The information display device 140 receives and displays the transmission data 132 transmitted from the information processing device 100.

  In step S <b> 801, the information display apparatus 140 determines whether to display a screen transferred from the information processing apparatus 100, for example, according to an instruction from the user of the information processing apparatus 100. When not displaying (No in S801), the process returns to S801, and the process of S801 is repeated until it is displayed. When displaying (Yes in S801), the process proceeds to S802.

  In step S <b> 802, the reception unit 141 of the information display device 140 determines whether the transmission data 132 has been received from the information processing device 100. If not received (No in S802), the receiving unit 141 repeats the process of S802. If received (Yes in S802), the reception unit 141 generates reception data 148 from the received transmission data 132, outputs the reception data 148 to the separation unit 142, and proceeds to S803.

  In step S803, the separation unit 142 and the still image data expansion unit 143 of the information display device 140 perform expansion processing of screen data that is outside the reproduction area of the moving image compressed data. First, the separation unit 142 separates the reception data 148 output from the reception unit 141, separates the screen data information 149 and the moving image compressed data information 150, if present, and the screen data information 149 into the still image data expansion unit 143. Send to.

  The still image data expansion unit 143 decodes the screen data information 149 and expands the screen data 613 in the entire screen area based on the screen layout information 611. When there are a plurality of update areas in the entire screen, there are a plurality of pieces of screen data information. In such a case, the still image data expansion unit 143 decodes all screen data information and performs expansion processing. The still image data expansion unit 143 expands the entire screen area or an equivalent large area at the start of display or when the entire display screen is updated, but when only the playback area of the moving image compressed data is updated. There are few areas to be newly developed.

  Referring to FIG. 6, at the time of the first transfer process, the still image data development unit 143 separates the received data 148 of FIG. 6A into the entire screen area 601, the screen data information count 602, and the screen data information 603. . If the entire screen area 601 is configured as shown in FIG. 7, the display size is specified as (0, 0, 1920, 1200) from the entire screen area 711. Further, the screen data information number 602 is one. The screen layout area 712 is also the same as the display size (0, 0, 1920, 1200). Therefore, the still image data expansion unit 143 expands (decodes) the screen data 613 in this area. During the transfer process after the first transfer process, the still image data expansion unit 143 separates the received data 148 of FIG. 6B into the entire screen area 601, the screen data information count 602, and the screen data information 603. The entire screen area 711 has the same value as the first transfer process. Since the screen data information number 602 is 0, the still image data expansion unit 143 determines that there is no screen data information and does not perform the decoding process.

  In step S <b> 804, the separation unit 142 of the information display device 140 determines whether the moving image compressed data information 119 exists in the reception data 148 output from the reception unit 141. If it exists (Yes in S804), the separation unit 142 sends the moving image compressed data information 150 to the moving image data reproduction unit 146, and proceeds to S805. If it does not exist (No in S804), the separation unit 142 determines that there is no reproduction of the moving image compressed data and that the entire screen is only the screen data 613, and the process proceeds to S809.

  Referring to FIG. 6, in the first transfer process, both the transmission data 132 in FIG. 6A and the transmission data 132 in FIG. There is one piece of information 605. Therefore, the separation unit 142 separates the transmission data 132 into the entire screen area 601 and the moving image compressed data information 605 serving as the reproduction area of the moving image compressed data, and the process proceeds to S805.

  S805 to S807 are processing of the moving image data reproduction unit 146 of the information display device 140. In step S805, the moving image data reproduction unit 146 determines whether moving image compressed data exists in the moving image compressed data information. If there is no moving image compression data in the moving image compression data information (No in S805), the moving image data reproduction unit 146 determines that it has already been received, such as during reproduction, and proceeds to S807. When the moving image compression data 624 exists in the moving image compression data information 605 (Yes in S805), the moving image data reproduction unit 146 determines that reproduction is newly performed, and proceeds to S806. The moving image compressed data 624 may be received only once during the reproduction period.

  Referring to FIG. 6, at the time of the first transfer process, the moving image data reproduction unit 146 determines that moving image compression data 624 exists because the moving image compression data size 623 in FIG. 6A is 15 MB, and the process proceeds to S806. In the transfer processing after the first transfer processing, the moving image compressed data size 623 in FIG. 6B is 0, so the moving image data reproducing unit 146 determines that there is no moving image compressed data, and the process proceeds to S807.

  In S806, the moving image data reproducing unit 146 decodes the moving image compressed data of the moving image compressed data information. Referring to FIG. 6, in the first transfer process, the moving image data reproduction unit 146 decodes the compressed moving image data 624 in FIG.

  In step S807, the moving image data reproducing unit 146 corrects the reproduction timing of the moving image compressed data currently being decoded in accordance with the time stamp of the moving image compressed data information. With this modification, the moving image data reproduction unit 146 matches the timing of the moving image reproduction data on the display screen of the information processing apparatus 100 that is the transmission source and the display screen of the information display apparatus 140 that is the transmission destination. Referring to FIG. 6, in the first transfer process, the moving image data reproduction unit 146 matches the time stamp 622 in FIG. 6A and in the subsequent transfer process to the time stamp 622 in FIG. 6B.

  S <b> 808 is processing of the combining unit 144 of the information display device 140. The combining unit 144 generates the display screen 153 by combining the background screen based on the still image data developed in S803 and the moving image screen of the moving image compressed data in S807 according to the moving image layout area of the moving image compressed data information.

  6 and 7, in the first transfer process, the moving image layout information 621 in FIG. 6A and the moving image layout information 621 in FIG. 6B in the subsequent transfer process are stored in the same moving image layout area 713. (380, 300, 1280, 720). As illustrated in FIG. 7, the combining unit 144 combines the moving image screen 702 with the background screen 701 to generate the entire screen 700.

  S809 is processing of the display control unit 145 of the information display device 140. The display control unit 145 displays the screen data based on the still image data developed in S803 or the display screen 153 in S808 as the transmission destination display screen 161. Is displayed (played back). After the display, the process returns to S802 to wait for reception of the next transmission data 132.

  As described above, when the information processing apparatus 100 connected via the network displays a screen on which the compressed video data is being reproduced on the information display apparatus 140, the compressed video data, the time stamp, and the video layout are first displayed from the information processing apparatus 100. Information is transmitted to the information display device 140. Thereafter, the information processing apparatus 100 can reduce the processing load on the information processing apparatus 100 by transmitting only the time stamp for screen synchronization and the moving image layout information to the information display apparatus 140. As a result, smooth moving image reproduction can be realized by both the information processing apparatus 100 and the information display apparatus 140.

  Specifically, conventionally, the information processing apparatus sends all data to be transmitted as still image data, and during the first transfer process, the transmission data 132 is 500 KB. For a minute, the size is 300 KB with one transfer. If the playback frame rate of the compressed video data is 30 fps, transfer processing of 1800 times per minute is required to update the screen with the same performance. The total amount of transmission data for 1 minute is 1800 times × 30 × 300 KB + 500 KB, which is 540.5 MB. In addition to this processing, the information processing apparatus must also reproduce the moving image compressed data, so that the load on the information processing apparatus as the transmission source becomes very high. When the performance of the information processing apparatus at the transmission source is low, the screen transfer interval must be reduced from 30 fps to 10 fps, etc. As a result, the moving image in the reproduction area of the moving image compressed data on the information display device at the transmission destination It will not be smooth.

  On the other hand, according to the present embodiment, at the time of the first transfer process, the transmission data 132 is 16 MB. However, for one minute of the subsequent transfer process, the size becomes 1 KB in one transfer, which is a very small size. If the playback frame rate of the compressed video data is 30 fps, transfer processing of 1800 times per minute is required to update the screen with the same performance. The total amount of transmission data for 1 minute is 1800 × 1 KB + 16 MB, which is 17.8 MB. In addition, transmission data in subsequent transfer processing is a time stamp and a moving image layout area for reproduction synchronization with the information processing apparatus in the moving image compressed data. Therefore, even if the screen transfer interval is lowered to 10 fps, the moving image in the reproduction area of the moving image compressed data in the information display apparatus of the transmission destination becomes smooth.

[Second Embodiment]
Hereinafter, the second embodiment of the present invention will be described focusing on differences from the first embodiment. However, the technical scope of the present invention is not limited to this embodiment.

  FIG. 9 is a diagram illustrating an example of functional configurations of the information processing apparatus 100 and the information display apparatus 140 according to the present embodiment. As a difference from the first embodiment of FIG. 1A, the information processing apparatus 100 receives the moving image compressed data 121 stored therein, but the information processing apparatus 100 in this embodiment is a content on an external network. The moving image compressed data 901 is input. The moving image compressed data that is the content may be continuous data such as a stream, or may be file data. The information processing apparatus 100 acquires the compressed moving image data 901 on the network from the network by specifying a URL (Uniform Resource Locator) or an IP (Internet Protocol) address where the data is arranged.

  Further, the information display device 140 is newly provided with a moving image receiving unit 902 as a difference from the first embodiment. The moving image receiving unit 902 receives the moving image compressed data information 903 indicating the storage location of the moving image compressed data to be reproduced from the moving image data reproducing unit 146. The moving image compressed data information 903 includes a moving image compressed data address, moving image layout information, and a time stamp. The moving image compressed data address is a URL or an IP address. The moving image receiving unit 902 acquires the moving image compressed data 901 on the network from the moving image compressed data address of the moving image compressed data information 903. Then, the moving image receiving unit 902 sends the acquired moving image compressed data 904 to the moving image data reproducing unit 146.

  The separation unit 142 sends the moving image compressed data information 903 from the reception data 148 that is the same as the transmission data 132 in the first embodiment to the moving image reception unit 902 and the moving image data reproduction unit 146. The moving image data reproducing unit 146 reproduces the moving image compressed data 904 to be reproduced from the moving image receiving unit 902, performs decoding according to the time stamp of the moving image compressed data information 150, and sends the moving image screen information 152 to the synthesizing unit 144.

  FIG. 10 is a diagram illustrating an example of a screen transfer system including the information processing apparatus 100 and the information display apparatus 140 according to the present embodiment. As a difference from FIG. 2 described in the first embodiment, it shows that the PC 201 and the projector 202 in the conference room 200 reproduce and display the compressed video data file 1000 on the network including the Internet. .

  The transmission data 132 in this embodiment is the same as the transmission data 132 in FIG. 6 described in the first embodiment. However, in the transmission data 132 of FIG. 6A, the moving image compression data 624 of the moving image compression data information 605 contains the moving image compression data address in this embodiment. The moving image compression data address is a URL or an IP address indicating the location of the moving image compression data file on the network. The display screen of this embodiment is the same as the screen shown in FIG. 7A described in the first embodiment.

  Next, with reference to FIG. 11, the process of S402 in the flowchart of FIG. 4 in the information processing apparatus 100 in the present embodiment will be described. FIG. 10 is a flowchart showing an operation of extracting moving image information by the information processing apparatus 100 according to this embodiment. Hereinafter, this flowchart will be described focusing on differences from FIG. 5 described in the first embodiment.

  S501 to S502 are the same as those in FIG. S1101 to S1102 are processes of the moving image data generation unit 109 of the information processing apparatus 100. In step S1101, the moving image data generation unit 109 determines whether the moving image compressed data of the video source being reproduced is a local file in the information processing apparatus 100 or a file on the network. Whether the moving image compressed data is a local file or a network file is determined based on whether the moving image compressed data size 623 or the content of the moving image compressed data is a character string. This is an example, and specific determination information may be added. If the moving image compressed data is a local file (Yes in S1101), the process proceeds from S503 to S504. If the moving image compressed data is a file on the network (No in S1101), the process proceeds to S1102.

  In step S1102, the moving image data generation unit 109 extracts the URL or IP address of the moving image compression data file as the moving image compression data address because the moving image compression data is a file on the network. After extraction, the process proceeds to S504. Step S504 is the same as that described in FIG. In S505 to S506, the processing itself is the same as that described in FIG. However, the moving image compressed data is the local file data itself in the first embodiment, but in this embodiment, the moving image compressed data is a moving image compressed data address. Henceforth, S507 to S508 is the same as the description of FIG.

  As a result, the information processing apparatus 100 only needs to transmit the moving image compressed data on the network only to the moving image compressed data address, and thus the amount of data for transmission can be reduced.

  Next, the operation of the transfer screen data display method of the information display device 140 will be described with reference to FIG. FIG. 12 is a flowchart showing an operation of displaying the transferred screen data by the information display device 140 according to this embodiment. Hereinafter, this flowchart will be described focusing on differences from FIG. 8 described in the first embodiment.

  Steps S801 to S805 are the same as those in FIG. S1201 is processing of the moving image data reproduction unit 146 of the information display device 140. The moving image data reproduction unit 146 analyzes the moving image compression data 624 of the moving image compression data information 605 from the separation unit 142 and determines whether the content of the moving image compression data 624 is a moving image compression data address from the moving image compression data size 623 or the like. . The moving image compression data address is the URL or IP address of the moving image compression data file, and the content of the data is a character string. For this reason, the value of the moving image compression data size 623 is at most several KB level, which is extremely small compared to the moving image compression data itself or the screen data which is still image data. For this reason, the load of the information processing apparatus 100 can be reduced. If the moving image compression data is a moving image compression data address (Yes in S1201), the process proceeds to S1202. If different (No in S1201), the process proceeds to S806 because it is the moving image compressed data itself.

  S1202 is a process of the moving image receiving unit 902 of the information display device 140. The moving image receiving unit 902 receives the moving image compressed data information 903 from the moving image data reproducing unit 146, and acquires the moving image compressed data from the specified URL or IP address with the moving image compressed data 624 as the moving image compressed data address. Then, the moving image receiving unit 902 sends the acquired moving image compressed data 904 to the moving image data reproducing unit 146 of the information display device 140. The moving image compressed data to be sent may be a single file, or data little by little like streaming. After reception, the process proceeds to S806. Henceforth, S806 to S809 are the same as description of FIG.

  As described above, when a screen on which the information processing device is reproducing the moving image compressed data on the network is displayed on the information display device via the network, the address of the moving image compressed data is first sent from the information processing device to the information display device. A time stamp and moving image layout information are also transmitted from the information processing apparatus to the information display apparatus. Thereafter, by transmitting only the time stamp for screen synchronization and the moving image layout information to the information display device, smooth moving image reproduction can be performed while suppressing the load on the information processing device.

[Other Embodiments]
In the above embodiment, the transmission destination information display device 140 has been mainly described as one unit, but a plurality of units may be used. In the above embodiment, the number of compressed video data 121 is one, but a plurality of pieces may be used.

  Further, the present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus execute the program. It can also be realized by a process of reading and executing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

100 Information processing device, 140 Information display device

Claims (13)

An information processing apparatus,
First creating means for creating moving image data, moving image layout information indicating a layout of the moving image data in a display area by the information display device, and moving image control information including information on a reproduction time of the moving image data;
Second creation means for creating still image control information including still image data excluding the moving image data in the display area and screen layout information indicating a layout of the still image data in the display area;
Transmission means for creating transmission data based on the moving image data, the moving image control information, the still image data, and the still image control information and transmitting the transmission data to the information display device;
Have
After the transmission data is transmitted by the transmission unit, the transmission unit transmits the moving image control information created by the first creation unit as the transmission data. When the video layout information changes,
The first creation means creates video control information including the changed video layout information and the playback time information of the video data,
The second creation means creates the still image control information including the still image data and the screen layout information based on the changed moving image layout information,
The transmission means includes the moving image control information created after the moving image layout information is changed, and the still image data. The information processing apparatus according to claim 1, wherein the still image control information is transmitted as the transmission data. A display control means for displaying the moving image data and the still image data on a display unit of the information processing apparatus;
The first creation means determines the reproduction time information of the moving image data to be included in the moving image control information so as to be synchronized with the reproduction time of the moving image data displayed on the display unit. The information processing apparatus according to claim 1 or 2. The information display device further comprises a determination means for determining whether the moving image data can be displayed;
4. The transmission device according to claim 1, wherein the transmission unit transmits the transmission data to the information display device that is determined to be capable of displaying the moving image data by the determination unit. 5. Information processing device.   The information processing apparatus according to claim 1, wherein the moving image data is data stored inside the information processing apparatus.   The moving image data is data acquired by the information processing apparatus via a network, and the moving image data indicates information on which the moving image data is arranged in the network. The information processing apparatus according to any one of the above.   The information processing apparatus according to claim 6, wherein the information in which the moving image data is arranged is a URL or an IP address.   An information display device comprising: a reproducing unit that reproduces the moving image data and the still image data based on the transmission data received from the information processing device according to claim 1.   9. The information display apparatus according to claim 8, further comprising an acquisition unit configured to acquire the moving image data from the network when the moving image data is data acquired via the network in the information processing apparatus. An information processing method,
A first creating step of creating moving image data, moving image layout information indicating a layout of the moving image data in a display area by the information display device, and moving image control information including information on a reproduction time of the moving image data;
A second creation step of creating still image control information including still image data excluding the moving image data in the display area and screen layout information indicating a layout of the still image data in the display area;
A first transmission step of creating transmission data based on the moving image data, the moving image control information, the still image data, and the still image control information and transmitting the transmission data to the information display device;
In the first transmission step, after the transmission data is transmitted, a second transmission step of transmitting the moving image control information created in the first creation step as the transmission data;
An information processing method characterized by comprising:   An information display method including a reproduction step of reproducing the moving image data and the still image data based on the transmission data received from the information processing apparatus according to claim 1.   A program for causing a computer to function as the information processing apparatus according to any one of claims 1 to 7. A program for causing a computer to function as the information display device according to claim 8 or 9.

Images