JP2001309318A - Device and method for controlling image information and computer readable storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce image deterioration in the case of receiving EPG data and each piece of image data from the Internet besides a television signal and arranging the data on the screen to be displayed. SOLUTION: Sound data are made to coincide with image data in timing, subsequently combined with the image data in a compressing part/synthesizing part 10, packeted by an isochronous packeting part 11 and sent to a linking part 13. Internet data inputted from a modem 23, written as graphic moving image data in a frame memory after format conversion, packeted by an isochronous/asynchronous packeting part 17 and sent to the part 13. EPG data are also processed in the same manner, subsequently packeted and outputted to the part 13. The configuration information of each image data is also prepared and sent to the part 13. The part 13 combines each image data and the configuration information time sequentially and outputs the image data and the configuration information.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a system for inputting a plurality of types of image information including a television signal, arranging them on one screen, and displaying and recording the image information. The present invention relates to a method and a computer-readable storage medium used in the method.

[0002]

2. Description of the Related Art In recent years, television systems have also become multifunctional, displaying not only television moving picture information transmitted from a broadcasting station, but also information from the Internet, and an electronic program guide (EPG) included in television signals. Can be displayed. In this case, these image and audio information are displayed in association with each other. For example, the broadcast contents are confirmed on the graphic screen of the EPG, and the broadcast contents of the program selected from the contents are displayed, and the information related to the program is displayed via the Internet. Not only that, it is integrated with the related image and audio data and displayed.

[0003] In addition, a home network is also required to have an apparatus for transmitting and displaying these integrated image data and audio data to another display and recording device for display and recording. However, these multiple image sources have different properties. For example, most of the image data of a broadcast television is a natural moving image. These images are compressed and transferred and recorded by irreversible compression encoding such as MPEG. However, since the high-frequency components of the images are relatively small, even if they are decoded and reproduced again, the image quality is hardly degraded.

On the other hand, since EPG data and graphic image data obtained via the Internet contain many characters and graphics, when compressed by irreversible compression coding such as MPEG, high-frequency components are reproduced when reproduced. Is removed, resulting in a blurred image and unsatisfactory transfer and storage.

FIG. 6 shows an image information control device used in a conventional multifunctional television receiver. In FIG. 6, reference numeral 1 denotes a front part of a television receiver, which includes a decoder for demodulating a television signal from a broadcasting station and decoding an image or audio signal compressed by MPEG or the like. Images from broadcasters are
After being converted into a predetermined digital signal format by the input unit 2, it is written to one plane of the frame memory 4 in accordance with a write memory address determined by the frame memory control unit 3.

On the other hand, the EPG information extracted from the television signal in the former stage 1 is also input as digital image data to the frame memory control unit 3 and the frame memory 4
Is written on another plane (a plane different from the television image).

Further, the image data input from the Internet is converted into predetermined digital image data by the control unit 5 via the modem 23 and the input / output control unit 7, and then converted into the frame memory via the frame memory control unit 3. 4 is written to another memory plane.

The image data written on each of the three planes of the frame memory 4 is output after being synthesized by the frame memory controller 3 under the control of the controller 5. Then, the compression unit / D / A unit 101
Compressed by EG or the like, digitally output, packetized by the isochronous packet unit 11,
After being converted into serial data by the serial conversion unit 19 via the, the data is transferred to a receiving device 22 including a recording device and a receiver, and is recorded and displayed.

In some cases, the data is D / A-converted by the compression unit / D / A unit 101, converted into an analog signal, and sent to the receiving device 22.

[0010]

However, in the above conventional example, the combined image data is compressed by the same compression method or converted into analog data.
Graphic images such as G data and Internet data that contain many figures and characters have a problem that the high-frequency portion is lost and the image quality is reduced. Also, when the screen movement of the graphic image becomes large, the graphic image contains many high-frequency components, and if transferred without compression, the image data amount becomes enormous, making transfer and recording virtually impossible. There was a problem.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problem, and has as its object to transfer various kinds of image data without deteriorating the image quality.

[0012]

To achieve the above object, an image information control device according to the present invention comprises a plurality of image input means for inputting different types of image data;
Creating means for creating configuration information relating to the configuration when each image data input from each of the image input means is arranged on a screen; and a plurality of image data input from each of the image input means according to the type. Compression means for compressing, packetizing means for packetizing each of the compressed or non-compressed image data and the created configuration information, and output means for combining and outputting the name packets in time series I have.

In the image information control method according to the present invention, a procedure for inputting a plurality of different types of image data, and configuration information relating to a configuration when the input image data are arranged on a screen are created. A procedure for compressing the input image data according to the type thereof, a procedure for packetizing the compressed or uncompressed image data and the created configuration information, and a procedure for packetizing the name packet. And a process of sequentially combining and outputting.

In the storage medium according to the present invention,
A process of inputting a plurality of image data of different types, a process of creating configuration information relating to a configuration when each of the input image data is arranged on a screen, and a process of converting the input image data according to the type. And a program for executing a process of packetizing the compressed or uncompressed image data and the created configuration information, and a process of combining and outputting the name packets in time series. I remember.

[0015]

DESCRIPTION OF THE PREFERRED EMBODIMENTS A first embodiment of the present invention described below compresses or decompresses input image data according to its type, and then converts the image data to a different digital signal transfer protocol. It is characterized in that it is transferred by. In other words, natural moving image data is important for real-time processing and has few high-frequency components in the image. I do. Also, graphic image data of characters and graphics does not change significantly on the screen, but the lack of high frequency components due to compression affects the image quality, so handshake communication with the receiving side is performed without compression and natural image data is sent. Not using a period
Packetize and transfer.

Further, according to the second embodiment of the present invention, when there is a screen change in graphic image data, real-time transfer is also important. It is characterized by.

Further, according to the third embodiment of the present invention, when the movement of the screen of the graphic image data becomes large, the transfer protocol can be changed to a transfer protocol that can be transferred in real time. It is characterized in that screen data is output periodically.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing an image information control device when applied to the multifunctional television receiver according to the first to third embodiments of the present invention. In FIG. 1, reference numeral 1 denotes a front part of a television receiver, which includes a decoder for demodulating a television signal from a broadcasting station and decoding an image and an audio signal compressed by MPEG or the like. The broadcasting station also has a function of separating and extracting electronic program guide (EPG) data transmitted together with image data.

Reference numeral 2 denotes an input section for converting an input signal into a predetermined digital signal format, and 3 controls writing and reading to and from a frame memory 4 in accordance with an instruction from a control section 5,
A frame memory control unit for generating a memory address, synthesizing each plane information of the frame memory 4 and outputting the same, 4 a frame memory having at least one plane hierarchy, 5 a control unit having a CPU, 6 a frame memory 4
Display position, display effect (alpha blending, wipe, etc.) of the image data written on the hierarchical plane
An image window management unit that generates overlay information, audio information, and the like in real time.

Reference numeral 7 denotes an input / output control unit for executing input / output of still image data and audio data from the modem 23 and a printer (not shown) in accordance with an instruction from the control unit 5; Conversion unit to convert to the format of
9 is an asynchronous packet unit for converting data into packets by handshake communication, and 10 is an
A compression unit / synthesis unit that performs compression by a compression encoding method such as PEG and synthesizes with audio data.

Reference numeral 11 denotes an isochronous packet unit for packetizing data at predetermined fixed time intervals, and 12 a function of packetizing data at predetermined fixed time intervals and a function of converting data into packets by handshake communication. An isochronous / asynchronous packet unit 13 that has both of them and that can switch between them is a link unit 13 that links and outputs a plurality of packet data in time series.

Reference numeral 14 denotes a motion detection unit for detecting a moving portion (line or pixel) from the continuous graphic moving image data. Reference numeral 15 denotes a motion frame memory control for controlling writing and reading of a frame memory required for motion detection. , 16 is a motion frame memory, 17 is an isochronous / asynchronous which has both a function of packetizing data at predetermined fixed time intervals and a function of converting data into packets by handshake communication. This is the packet part.

Reference numeral 18 denotes a refresh unit for issuing an instruction for forcibly outputting image data of the motion frame memory 16 for at least one screen, for example, one frame, and 19 converts parallel data to serial data and outputs a predetermined signal. This is a serial conversion unit for converting to a level. These signal levels and data formats are based on IEEE 1394 and USB (U
universal serial bus).

Reference numeral 20 denotes a signal received from the apparatus, which converts a serial signal into a parallel signal, decomposes each image and audio packet signal, separately decodes and synthesizes each image data, and displays it. The receiving device 21 is a recording device that receives a signal input from the present device and records the signal on a recording medium. The recording device 21 may record each image and audio data after separating them, or may record them while being combined.

Reference numeral 23 denotes a modem for inputting and outputting Internet information from a telephone line. Reference numeral 24 denotes a pointing device such as a mouse, which is used when a user operates a screen, such as a display position of an input image and an overlapping order. An output control unit 25 converts image data from the frame memory control unit 3 into data suitable for the display 26.
Reference numeral 6 denotes a display unit that displays the combined image data. 2
Reference numeral 7 denotes an audio output unit.

Next, the basic operation of the present apparatus will be described. The image data (natural moving image data) and the EPG data of the television signal output from the television front section 1 are converted into a desired data format by the input section 2 and then converted into the frame memory 4 via the frame memory control section 3. Are written to different planes. The image data (natural moving image data) read from the frame memory 4 is compressed by the compression unit / combining unit 10 via the frame memory control unit 3 and the control unit 5.

The audio data of the television signal is transmitted to the control unit 5.
After the timing is adjusted with the image data described above, the data is input to the compression / synthesis unit 10 via the input / output control unit 7,
The image data is combined with the image data. The synthesized image and audio data are packetized at regular intervals by the isochronous packet unit 11 and sent to the link unit 13. The audio data is also input to the audio output unit 27, and the audio is reproduced.

On the other hand, Internet data obtained via the modem 23 and the input / output
After being converted into a predetermined format by U, it is written as graphic moving image data via the frame memory control unit 3 to a plane of the frame memory 4 different from the above-mentioned plane. The graphic moving image data is uncompressed by the motion detecting unit 14 and the frame memory control unit 1.
The packets are packetized at regular intervals by the isochronous / asynchronous packet unit 17 via the communication unit 5 and output to the link unit 13.

Further, the EPG data written in the frame memory is also processed in the same manner as the graphic image data, then packetized and output to the link unit 13. At the time of the above basic operation, the motion detection unit 14
Does not operate, and the motion frame memory 16 is not used.

A communication control line is provided between the television front section 1 and the control section 5 so that information on each image data written in the frame memory 4, such as resolution, the number of colors, position data to be displayed, etc., is transmitted to the front of the television. It is sent from the unit 1 to the control unit 5. The control unit 5 determines the frame memory 4 based on the information and the information on the graphic image data generated by the control unit 5.
The configuration information indicating the configuration content of each image data output from is generated and sent to the image window management unit 6. This transmission timing may be always at a fixed interval, or when a change occurs in the configuration content. The image window management unit 6 sends the graphic image data, its audio data, and the configuration information to the isochronous / asynchronous packet unit 12 at regular time intervals and packetizes them.

FIG. 2 shows an example of the contents of the image configuration. In this example, the frame memory 4 has three layers, and the layer 1 has a television image (natural moving image) from a broadcasting station.
Is written. At this time, from the TV front part 1,
Display size (X1-X0) on TV screen, (Y3-Y
The value of 1) is sent to the control unit 5. (X0, Y0), (X1, Y1), (X
2, Y2), (X3, Y3) and a layer number indicating the order of the display hierarchy are also sent. These are based on data transmitted to the control unit 5 via the input / output control unit 7 based on the displacement of the pointing device 24 caused by the operation of the user.
Is generated.

The generated configuration information including the position information, the layer number and the like is transmitted to the frame memory control unit 3, and based on the data, the television image data is displayed in the position and layer on the screen as shown in FIG. You.

Similarly, the Internet information obtained via the modem 23 is stored in the control unit 5 in the position information (X
4, Y4), (X5, Y5), (X6, Y6), (X
7, Y7), a layer number 2 is generated, and transmitted to the frame memory control unit 3.

Further, the position information for the image and the layer number for the EPG are obtained from the EPG data sent from the television front section 1 and its size information by the control unit 5 (X8, Y).
8), (X9, Y9), (X10, Y10), (X1
1, Y11) and layer number 3.

These layer numbers also indicate the order of superposition when the windows of the respective images overlap, and the smaller the layer number is, the higher the superposition.
When the alpha blending (the effect of performing a weighting operation for each pixel of the overlapping portion to make it look transparent) is performed between the overlapping windows, the control unit 5 transmits the information (alpha value) to the image window. Management unit 6
To

Further, the audio data can be changed so that the volume data can be changed by the layer number of the superimposition.
A volume control code is added to the audio data.

The control unit 5 outputs the position information, the layer number, the alpha value, the volume data, the audio data, and the like to the image window management unit 6 when each information changes or at regular intervals. The image window management unit 6 organizes each data based on the data.

The image window management section 6 has an interface for receiving these data from an external network, and can be used in combination with data from the control section 5 or mixed with data from the control section 5. Further, data may be received via the serial conversion unit 19, the link unit 13, and the isochronous / asynchronous socket unit 12.

The image and audio configuration data created by the image window management unit 6 is output to an isochronous / asynchronous packet unit 12, packetized in an asynchronous mode using handshake communication, and then output to a link unit 13. Linked.

If the image changes drastically, the image window management unit 6 sends a switching signal to the isochronous / asynchronous packet unit 12 to change to an isochronous mode (a mode in which packetization is performed at regular time intervals and handshake communication is not performed). Is output to change the packetization mode.

The still image data to be printed is generated by the control unit 5, converted in format by the conversion unit 8 via the input / output control unit 7, output to the asynchronous packet unit 9, packetized and linked. 13 is output.
After each packet is linked in time series in the link unit 13, it is output to the serial conversion unit 19, and performs level conversion and parallel-serial conversion applicable to an external network.

The serial data output from the present apparatus is transferred to the receiver 20 or the recording device 21, and is transmitted to the receiver 20.
On the side, processing reverse to the above description, that is, serial-parallel conversion, data decomposition, control of developing each image data from the image and audio configuration data to the frame memory, writing each image data to the frame memory 4, and the above-mentioned volume It outputs audio data based on the control data, reads out and displays each image data written in the frame memory 4. Also,
The recording device 21 records each image data and audio data on a recording medium.

Next, as a second embodiment of the present invention,
The operation when the graphic moving image data has increased the definition and the data amount has increased will be described. The graphic moving image data written in a certain layer of the frame memory 4 is transmitted to the motion detection unit 1 via the frame memory control unit 3.
4 The motion detection unit 14 compares the transmitted data at least in units of one frame, and detects a portion having motion.

When the moving part exceeds a certain threshold value, the fact is transmitted to the moving frame memory control unit 15. The graphic moving image data is written into the motion frame memory 16 by the motion frame memory control unit 15 via the motion detection unit 14.

The motion detecting section 14 also sends the position information of the portion where the motion has occurred to the motion frame memory control section 15. The moving frame memory control unit 15 reads out the graphic moving image data of the moving part from the moving frame memory 16 based on the position information, and outputs the read moving image data to the isochronous / asynchronous packet unit 17.

FIG. 3 is a timing chart when graphic moving image data is transferred in the asynchronous mode.
Tp is a certain time, and is determined in advance between transmission and reception of the present system. This time Tp is equal to IEEE
1394 or the time specified by the USB.

Ti is the time reserved for the isochronous transfer mode, and the time of Ti is divided and used by a plurality of isochronous transfer packets. This Ti is always reserved time, and retry transfer is not performed in order to attach importance to real-time performance.

Ta is a time reserved for the asynchronous transfer mode. Normally, transmission is performed while performing handshake communication between the transmission side and the reception side. Therefore, even when a transfer error occurs, retry transfer is possible.

FIG. 3 shows an example in which natural moving image packet data as a television image is transmitted in the isochronous transfer mode, and graphic moving image data, image / audio configuration data, and still image data of the printer are transferred in the asynchronous transfer mode. I have.

At the time Ti, real-time images and sounds are transmitted, and at the time Ta, graphic moving images which do not require relatively real-time are transmitted. During the time Ta, the transfer is performed in the handshake mode and the retry transfer is performed at the time of the error transfer. Therefore, if the graphic moving image is transferred as it is, the transfer end time is later than the isochronous transfer of the image having the real-time property.

In the case where the graphic moving image data is highly defined in this way, if only the moving part is transferred, the data transfer amount during the Ta time is reduced, and the transfer is performed in the asynchronous transfer mode even in the isochronous mode. The time difference from the case where the image is compressed is small, the synthesized image can be transferred without causing any trouble, and the graphic moving image is not irreversibly compressed, so that the image quality does not deteriorate.

Next, a third embodiment of the present invention will be described. This embodiment is effective when the graphic moving image data is displayed as a three-dimensional graphic moving image or when the screen scrolls. If the graphic moving image uses three-dimensional graphics and the image moves a lot, the data transfer amount of the moving part increases, and if handshake transfer such as asynchronous transfer is performed,
Further, the data transfer amount increases, so that the transfer is not completed within a certain time, and the temporal relationship with other moving images is shifted.

Therefore, when the change in the motion of the graphic moving image exceeds a certain value, the motion detecting unit 14 sends a signal for changing the transfer mode to the isochronous mode to the control unit 5 and the motion frame memory control unit 15. Output.

FIG. 4 shows an example of the transfer timing according to the present embodiment. Here, the case where the change of the configuration data of a plurality of images also increases is taken as an example. During the isochronous transfer time Ti, the natural moving image packet data, the image / audio configuration packet data, and the graphic moving image packet data are sent, and during the asynchronous transfer mode time Ta, only the still image packet data for the printer is transferred.

As described above, when the movement of the graphic image becomes large, the transfer mode is switched to deal with it.
However, in this case, since error processing cannot be performed (because data is transferred within a certain period of time and there is no time to retransmit even if there is an error), if there is an error transfer and that part is not overwritten, an error There is a problem that the image data with the error remains on the receiving side.

In order to avoid the above problem, in the present embodiment, the control unit 5 forcibly forces the refresh unit 18 to change the screen at least one frame at regular time intervals in the isochronous mode transfer. Instruction to transfer data. The refresh unit 18 instructs the motion frame memory control unit 15 to read all data in the motion frame memory 16.

Thus, even if a transfer error occurs,
If the refresh unit operates once, the screen of the error part is overwritten, and the time interval for performing the refresh operation is shortened to such a degree that the user does not notice, so that the deviation does not become a problem.

FIG. 5 is a diagram showing the relationship between the graphic moving image data transfer method and image movement according to the present embodiment.
Since there is a lot of movement during screen scrolling or 3D moving images, isochronous transfer (no error processing) is performed, and the above-described refresh operation is periodically performed to prevent transfer errors.
Graphic moving image data with little motion, such as EPG, performs asynchronous transfer that can perform error processing including retransmission.

The isochronous / asynchronous transfer may be the one employed in IEEE 1394, or the isochronous or bulk transfer employed in USB.

Next, a storage medium according to another embodiment of the present invention will be described. The system of FIG. 1 described above
Although it can be constituted by hardware, it can also be constituted by a computer system having a CPU and a memory. When constituted by a computer system, the memory constitutes a storage medium according to the present invention. The storage medium stores a program for executing the operations and processes described in the above embodiments.

As the storage medium, ROM, R
Semiconductor memory such as AM, optical disk, magneto-optical disk,
A magnetic storage medium or the like may be used, and these may be a CD-ROM,
The present invention may be applied to an FD, a magnetic card, a magnetic tape, a nonvolatile memory card, or the like.

Therefore, by using this storage medium in a system or apparatus other than the system shown in FIG. 1, the system or the computer reads out and executes the program code stored in the storage medium.
The same functions as those of the above embodiments can be realized, the same effects can be obtained, and the object of the present invention can be achieved.

An OS running on a computer
When performing part or all of the processing, or after the program code read from the storage medium is written to the memory provided in the extended function board or the extended function unit connected to the computer inserted in the computer, Even when the CPU or the like provided in the above-mentioned extended function board or extended function unit performs part or all of the processing based on the instruction of the program code, the same functions as those of the above-described embodiments can be realized and the same functions can be realized. Effect can be obtained,
The object of the present invention can be achieved.

[0064]

As described above, according to the present invention,
Since there is no need to compress all image data input from a plurality of image information sources or convert it to analog data as in the past, many figures and characters such as EPG (electronic programmable guide) and Internet data are used. Transfer without loss of image quality
Can be displayed and recorded.

Further, even when the screen movement of the graphic image becomes large, by changing the transfer method in accordance with the increase of the image data amount, the synthesized plural images can be prevented from being displaced by the transfer time difference. ,
Can be transferred.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an image information control device according to an embodiment of the present invention.

FIG. 2 is a configuration diagram of each image data on a screen at the time of input from a plurality of image sources.

FIG. 3 is a transfer timing chart of a graphic moving image according to the first embodiment of the present invention.

FIG. 4 is a transfer timing chart of a graphic moving image according to a second embodiment of the present invention.

FIG. 5 is a configuration diagram illustrating switching of graphic moving image transfer.

FIG. 6 is a block diagram showing a conventional image information control device.

[Explanation of symbols]

Reference Signs List 1 TV front section 3 Frame memory control section 4 Frame memory 5 Control section 6 Image window management section 7 Input / output control section 9, 11 Asynchronous packet section 10 Compression section / combining section 12, 17 Isochronous / asynchronous packet section 13 Link section 14 Movement Detector 15 Frame memory controller for motion 16 Frame memory for motion 20 Receiver 21 Recording device

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Shuntaro Araya 3-30-2 Shimomaruko, Ota-ku, Tokyo Inside Canon Inc. (72) Inventor Hideaki Yui 3-30-2 Shimomaruko, Ota-ku, Tokyo Canon Incorporated F term (reference) 5C025 BA11 BA25 BA28 CB08 DA01 DA05 5C063 AB03 AC01 AC02 CA23 CA36 DA07 DA13 EB01 EB33

Claims (14)

[Claims] 1. A plurality of image input means for inputting different types of image data, and a generating means for generating configuration information relating to a configuration when each image data input from each of the image input means is arranged on a screen. Compression means for compressing a plurality of image data input from each of the image input means according to the type thereof; and packetizing means for packetizing each of the compressed or non-compressed image data and the created configuration information. And an output means for combining and outputting the name packets in a time-series manner. 2. An error processing unit for performing error processing on each of the image data, a detection unit for detecting a motion of each of the image data, and a re-output of data obtained by performing error processing on the image data in accordance with the detected motion. 2. The image information control device according to claim 1, further comprising a switching unit that switches whether to perform the operation. 3. The motion detecting means performs a motion detection when the input image data is graphic moving image data, and the packetizing means packetizes the detected motion part and transmits the packet by handshake communication. 3. The image information control device according to claim 2, wherein the image information is output while performing. 4. When the amount of movement of the graphic moving image data increases, the packetizing means outputs a packet at regular time intervals without performing the handshake communication, and at least outputs the graphic moving image data at regular time intervals. 4. The image information control device according to claim 3, further comprising output means for outputting one screen. 5. The image information control device according to claim 1, wherein said compression means performs compression when the input image data is natural moving image data. 6. The image information control device according to claim 1, further comprising information input means for inputting the configuration information of one or more image data. 7. The method according to claim 1, wherein said packetizing means is IEEE 139.
4 or USB (Universal Serial B)
The image information control device according to claim 1, wherein the image information control device uses a user interface (usa). 8. The image information control apparatus according to claim 1, wherein the configuration information includes at least one of a display position of each image data on a screen, an overlay layer number, and an overlay blending value. 9. The image information control device according to claim 1, wherein the configuration information includes at least one of a display position of each image data on a screen, an overlay layer number, and an overlay blending value. 10. An image according to claim 1, further comprising display control means for arranging each of said compressed or uncompressed image data on the screen based on said configuration information and displaying it on a display means. Information control device. 11. The image information control apparatus according to claim 1, further comprising one or more audio input means for inputting audio data accompanying the input image data. 12. The image information control device according to claim 10, wherein the configuration information includes volume control data of the input voice. 13. A procedure for inputting a plurality of different types of image data, a procedure for creating configuration information relating to a configuration when each of the input image data is arranged on a screen, A procedure of compressing each of the compressed or uncompressed image data and the created configuration information into packets, and a procedure of combining and outputting the name packets in time series. An image information control method, comprising: 14. A process of inputting a plurality of different types of image data; a process of creating configuration information relating to a configuration when the input image data is arranged on a screen; Processing according to the type, processing of packetizing the compressed or uncompressed image data and the created configuration information, and processing of combining and outputting the name packets in time series. A computer-readable storage medium storing a program to be executed.