JP2004312584A - Image processing method and image display system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing method and image display system for compressing an image into low capacitance at high speed while preventing deterioration of the image. <P>SOLUTION: An entire picture is captured in a step S11, and a change area in the picture is calculated in a step S21. An application under displaying an image to be captured in a step S13 is then detected. In accordance with the application, the selection of a compression scheme in a step S15 is optimized, thereby compressing the image into the low capacitance at high speed while preventing the deterioration of the image. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image processing method and an image display system, and more particularly, to an image compression method for compressing an image generated by an image generation device, and displaying an image after compressing the image at a transmission source and then expanding the image at a reception destination. The present invention relates to an image display system.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, an image processing method has been devised in which an image is transmitted by dynamically changing a compression method according to image information and communication channel conditions. For example, a method (first method) described in Patent Document 1 is known. This method includes switches SW1 and SW2 for switching the compression method, a predictive encoder 122, a variable length encoder 123, and an MPEG encoder 124, as shown in an encoding device 121 in FIG. . In this method, first, a moving area is detected from the areas read from the memory. Next, by performing a histogram process on the detected area, it is determined whether the image is close to a natural image, and the switches SW1 and SW2 are switched based on the determination result. Thus, in the case of a moving image, MPEG encoding can be performed by the MPEG encoder 124, and in other cases, the prediction encoder 122 and the variable-length encoder 123 can compress the moving image.
[0003]
Patent Document 2 discloses an invention in which, when image data is transmitted via a network, the data transmission speed in the network is calculated, and the image data to be transmitted is reduced based on the calculated data transmission speed ( Second method) is shown. In the present invention, a method is disclosed in which image data is resized or the compression ratio is changed so that the data amount is reduced and the transmission time is constant.
[0004]
[Patent Document 1]
JP-A-10-145796, page 9 to page 10 [Patent Document 2]
JP 2001-36655 A
[Problems to be solved by the invention]
However, in the first method, it takes time to determine a natural image area in order to determine from a histogram, and erroneous determination frequently occurs, so that it is difficult to increase the speed. In the second method, high-speed transfer can be realized because the compression ratio of an image is changed according to the transmission speed, but image distortion is likely to occur because the compression ratio is changed while ignoring the features of the image. There were drawbacks.
[0006]
An object of the present invention is to solve the above-mentioned conventional problems and to provide an image processing method and an image display system for compressing an image at a high speed and a smaller capacity while suppressing the deterioration of the image.
[0007]
[Means for Solving the Problems]
The invention according to claim 1 of the present application is an image processing method for compressing an image of an image signal generating device, wherein the image signal generating device detects a program for generating an image for each area displayed by the image signal generating device. And a step of selecting a compression method according to the detected program, and a step of compressing the image generated by the image signal generating device according to the selected compression method.
[0008]
The invention according to claim 2 of the present application is the image processing method according to claim 1, wherein the compression step is a method of compressing an image generated by the image signal generating device for each display area.
[0009]
According to a third aspect of the present invention, in the image processing method of the first aspect, the step of selecting a compression method in accordance with the detected program includes, when there are a plurality of detected programs, selecting one of the plurality of programs. , A step of selecting a compression method.
[0010]
According to a fourth aspect of the present invention, in the image processing method according to any one of the first to third aspects, the image signal generating device is a personal computer.
[0011]
The invention according to claim 5 of the present application is an image processing method for compressing an image of an image signal generating device, wherein the image signal generating device detects a program for generating an image for each area displayed by the image signal generating device. And a step of selecting a compression ratio according to the detected program, and a step of compressing the image generated by the image signal generating device according to the selected compression ratio.
[0012]
According to a sixth aspect of the present invention, in the image processing method of the fifth aspect, the compression step is a method of compressing an image generated by the image signal generator for each display area.
[0013]
According to a seventh aspect of the present invention, in the image processing method according to the fifth aspect, the step of selecting a compression rate according to the detected program includes selecting one of the plurality of programs when there are a plurality of detected programs. , A compression ratio is selected.
[0014]
According to an eighth aspect of the present invention, in the image processing method according to the fifth aspect, the step of selecting the compression ratio includes, when the detected program is moving image reproduction software, the compression ratio is set if the bit rate of the reproduced moving image is high. It is characterized in that the compression ratio is increased when the bit rate is small and the bit rate is low.
[0015]
According to a ninth aspect of the present invention, in the image processing method according to any one of the fifth to eighth aspects, the image signal generating device is a personal computer.
[0016]
According to a tenth aspect of the present invention, in the image processing method of the ninth aspect, the compressing step is JPEG compression.
[0017]
The invention according to claim 11 of the present application includes an image signal generating device that generates an image signal, a transmitting device that transmits the generated image signal, and a display device that receives and displays an image transmitted from the transmitting device. In the image display system, the image generating device includes a compression unit that captures an image to be generated and compresses the image in a compression method selected according to an operating program, and the transmission device includes a selected compression method. The display device transmits a compressed image, and the display device includes an expansion unit that expands the transmitted compressed image in accordance with a compression method.
[0018]
According to a twelfth aspect of the present invention, in the image display system of the eleventh aspect, the image signal generating device is a personal computer.
[0019]
According to a thirteenth aspect of the present invention, in the image display system according to the eleventh or twelfth aspect, the image display device is a projector.
[0020]
BEST MODE FOR CARRYING OUT THE INVENTION
(Embodiment 1)
An image processing method according to the first embodiment of the present invention will be described. In the present embodiment, it is assumed that images displayed on a personal computer (hereinafter, referred to as PC) are continuously captured and compressed. FIG. 2 is a block diagram illustrating a configuration of the image processing apparatus according to the present embodiment, and FIG. 3 schematically illustrates a screen of a PC.
[0021]
In FIG. 2, a CPU 11 is connected to a bus line 12. A memory 13 and a display 14 are connected to the bus line 12, and an input unit 15 and a graphic adapter 16 are also connected to form an image reproducing apparatus.
[0022]
FIG. 3 also shows that a screen 21 on the display 14 of the personal computer 10 has a display area 22 for document creation software (hereinafter referred to as software) A and a display area 23 for moving image display software E. An area 24 indicates a background area where the document creation software A and the moving image reproduction software E are not displayed.
[0023]
Next, the image processing method of the present embodiment will be described in detail with reference to the flowchart of FIG. First, in step S11, the entire contents of the screen 21 of the PC are loaded into the memory of the PC. In step S12, the content of the entire captured screen is compared with the content previously captured, the coordinates of the change point are calculated, and the change area of the screen is calculated. FIG. 4 is a diagram for explaining an example of a method of calculating a change area. Here, the crosses in FIG. 4 indicate pixels where the previously captured screen and the currently captured screen have changed. The change area is obtained by calculating a rectangle that includes all the pixels where the screen has changed. FIG. 4 shows an example of the calculated change area 25. However, at the time of the first capture, since there is no screen captured last time, the entire screen is set as the change area.
[0024]
In step S13, when the calculated change area 25 is completely included in the display area 22 of the document creation software A on the screen of the PC, for example, the document creation software A is detected as a changing application. . When the change area 25 overlaps with the background area 24, the display area 22 of the document creation software A, and the display area 23 of the moving image reproduction software E as shown in FIG. , Detect the application and perform the subsequent processing steps.
[0025]
In step 14, the detected application is searched from the list. FIG. 5 shows an example of this list. When no application is detected, and in the case of the background area 24, other is selected in the list of FIG.
[0026]
This list is a list in which programs registered in advance and compression methods are associated with each other. For example, if the detected program is the document creation software A, the list is referred to, and in step 15, PNG compression is selected as the compression method. In the case of a program for creating a document, the displayed image often has a small number of colors, and by using PNG compression, the capacity after compression can be suppressed at high speed without distortion. Therefore, in the list of FIG. 5, the document creation software A and the document creation software D are PNG compressed. In the case of image display software, natural images are often handled, so JPEG compression suitable for natural images is used. In the case of moving image display software, a natural image is often correlated with the previous screen. Therefore, a compression method using an inter-frame difference such as MPEG compression is selected. Further, in the case of other applications or background images, a high-speed compression method with little distortion on average for various images, for example, compression of vector quantization is selected. According to the list of FIG. 3 searched as described above, one of PNG compression (step S16), JPEG compression (step S17), MPEG compression (step S18), and vector quantization (step S19) is selected. It is assumed that such selection is performed for each area of the image and compressed.
[0027]
By detecting which application is displaying the image to be compressed in this manner, an optimal compression method can be selected. Therefore, according to the present embodiment, it is possible to compress an image at a high speed and a smaller capacity while suppressing the deterioration of the image.
[0028]
In the present embodiment, a method has been described in which an application is detected by dividing it into portions, and compression is performed separately. However, for example, a method in which the change area 25 in FIG. Good. In this case, the compression method is selected, for example, in the list of FIG. 5, by giving priority to the registration programs in order from the top. If the change area 25 is included in the document creation software A even a little, PNG compression is selected. If it is not included in the creation software A but is included in the image display software B at all, a method of selecting JPEG compression may be used.
[0029]
Also, in the present embodiment, examples of document creation software, image display software, and moving image display software have been described as applications to be registered, but the present invention is not limited to this. The list in FIG. 5 may be registered in advance, or may be rewritable by the user.
[0030]
In addition, an example of a method of capturing the entire screen and detecting a change point has been described. However, an application that is operating in response to a notification of a running application or a screen update notification of an operating application is detected and operated. It is good also as a method.
[0031]
Also, in the present embodiment, PNG compression, JPEG compression, MPEG compression, and vector quantization have been described as examples of compression methods. However, a method of reducing the number of colors or a method of reducing the number of pixels of an image may be used. It is not limited.
[0032]
(Embodiment 2)
Next, an image processing method according to the second embodiment of the present invention will be described. An apparatus for implementing this image processing method is the same as in the above-described embodiment. In this embodiment, the compression ratio is determined by the software used for each area of the screen and the transmission speed.
[0033]
Next, the image processing method will be described in detail with reference to the flowchart of FIG. First, in step 61, the contents of the entire screen of the PC are loaded into the memory of the PC. In step S62, as in the first embodiment, the contents of the entire captured screen are compared with the previously captured contents to calculate the coordinates of the change points. The area is defined as a change area. However, in the first time, since there is no content captured last time, the entire screen is set as the change area. In step S63, the changed application is detected. The processing up to this point is the same as in the first embodiment. For example, if the calculated change area is included in the area on the screen of the PC, for example, where the moving picture reproduction software I is displayed, the moving picture reproduction software I is detected as the changing program. In step 64, it is determined whether the detected application is moving image reproduction software. If not, the application is searched from the list in step S65. FIG. 7 shows an example of this list. If no application is detected, the other is selected in the list of FIG. In the case of the moving image reproducing software I, the bit rate of the moving image being reproduced is detected in step S66. Then, in step S67, a compression ratio is selected. For example, when a 64 kbps moving image is reproduced, a JPEG compression ratio of 50 is selected from the list in FIG. If the detected application is the document creation software G, the compression rate 90 is selected in step S67 with reference to the list in FIG. After selecting the compression ratio in this way, JPEG compression is performed in step S68.
[0034]
As described above, the optimum JPEG compression ratio can be selected by detecting which application displays the image to be compressed in which region. Therefore, high-speed processing can be realized by lowering the compression ratio and reducing image distortion in software including high-frequency components in an image such as document creation software, and increasing the compression ratio in image display software. In the case of video playback software, by changing the compression rate according to the bit rate, assigning a bit rate more than necessary by increasing the JPEG compression rate for images with a low bit rate and large image distortion. And speeding up is possible. Conversely, for an image with a high bit rate and small image distortion, the JPEG compression ratio can be reduced to prevent image degradation. As described above, according to the present invention, by detecting which application is displaying an image to be compressed, an optimal compression ratio can be selected, and the capacity of the image can be quickly increased while suppressing the deterioration of the image. Can be compressed smaller.
[0035]
In the present embodiment, a method has been described in which an application is detected by dividing the application into parts and compression is performed on each application. However, a method may be employed in which the change area is compressed at a time without being divided. In this case, the compression method may be selected from among the lowest compression rates among the applications included in the change area, or from the average value or the intermediate value, but is not limited to this.
[0036]
Also, in the present embodiment, examples of document creation software, image display software, and moving image display software have been described as applications to be registered, but the present invention is not limited to this. 7 and 8 may be registered in advance or may be rewritable by the user. Also, in the case of moving image display software, an example in which the compression rate is changed according to the bit rate has been described. However, the compression rate may be changed depending on the format of the software being reproduced, for example, and the present invention is not limited to this. In addition, an example of a method of capturing the entire screen and detecting a change point has been described. However, an application that is operating in response to a notification of a running application or a screen update notification of an operating application is detected and operated. You may do so. Further, although an example of JPEG compression has been described in the present embodiment, the present invention is not limited to this.
[0037]
(Third embodiment)
FIG. 9 is a configuration diagram of a display system according to Embodiment 3 of the present invention. In FIG. 9, reference numeral 91 denotes a projector, 92 denotes a screen for displaying an image of the projector, 93 denotes a PC, and 94 denotes a wireless LAN transceiver. This display system first loads an image displayed on the PC 93 into a memory. Next, the captured image is compressed and transmitted to the projector 91 via the wireless LAN transceiver 94. The projector 91 receives the compressed image, decompresses the image, and displays the image on the screen 92. Thus, the same image as the screen on the PC 93 can be displayed on the screen 92. Here, the PC 93 is an image signal generating device for generating an image signal, and the wireless LAN transceiver 94 constitutes a transmitting device for transmitting the image signal thus generated. The projector 91 constitutes a display device for receiving and displaying the transmitted image.
[0038]
FIG. 10 is a block diagram showing a specific internal configuration. 10, 101 to 110 are processing blocks on the PC side, and 111 to 115 are processing blocks on the projector side. The PC 93 includes an operation application detection unit 101, a time monitoring unit 102 for monitoring a predetermined time, an operation unit acquisition unit 103 for acquiring an area in which an operation application is displayed in an image memory, and an entire screen acquisition unit for acquiring an entire screen in a memory. 104, an update area calculating means 105 for comparing the previous screen with the current screen and calculating an updated area, an update record obtaining means 106 for obtaining an image of the update area, and an application detected by the operation application detecting means 101 A compression method selecting means 107 for determining a compression method by means of an image, an image compression means 108 for compressing an image, an image information transmitting means 109 for transmitting image information obtained by the update area calculating means 105 and the compression method selecting means 107 to the projector, Compressed image transmission for transmitting a compressed image to the projector 91 It is configured to include a means 110. Here, the image signal transmission unit 109 and the compressed image transmission 110 are realized by the wireless LAN transceiver 94.
[0039]
On the other hand, the projector 91 includes an image information receiving unit 111 for receiving image information, a compressed image receiving unit 112 for receiving a compressed image, an image expanding unit 113 for expanding a compressed image based on information of the image information receiving unit 111, It includes a previous screen updating means 114 for updating only the part updated from the previous screen to the image memory based on the image information obtained by the information receiving means 111, and a display means 115 for displaying information of the image memory. I have.
[0040]
The operation of the display system configured as described above will be described. First, the operation application detection unit 101 detects an operation application by acquiring a command to be drawn by a PC application on a screen. When the running application detecting unit 101 detects an operating application on the PC 93, or when there is no running application and the time monitoring unit 102 elapses a predetermined time such as one second, the update area calculating unit 101 At 105, the previous screen and the current screen are compared. The comparison between the previous screen and the current screen in the update area calculation means 105 is such that, when a running application is detected, only the area to be drawn on the screen by the application is fetched into the memory by the operation section fetch means 103. ,Compare. When a certain time has elapsed, the entire screen is fetched into the memory by the entire screen fetching means 104 and compared.
[0041]
As a result of the comparison, the update area calculation unit 105 acquires the coordinates of the change point, and calculates a rectangular area including all of the coordinates as the change area. The data of the image of the rectangular area is acquired by the update area acquisition means 106 and is passed to the image compression means 108. In addition, using the information of the operating application, the compression method is selected by the compression method selection means 107 and passed to the image compression means 108. The image compression means 108 compresses the image based on the information. A method of selecting a compression method will be described later. The upper left coordinates and lower right coordinates of the rectangular area calculated by the update area calculation means 105 and the information on the compression method selected by the compression method selection means 107 are transmitted to the projector 91 by the image information transmission means 109. The image data compressed by the image compressing means 108 is transmitted to the projector 91 by the compressed image transmitting means 110.
[0042]
In the projector 91, the image information is received by the image information receiving unit 111, the compressed image is obtained from the PC 93 by the compressed image receiving unit 112, and the image is expanded by the image expanding unit 113 based on the image information. Thereafter, the image data expanded to the position of the image memory of the projector 91 corresponding to the coordinates obtained by the image information of the image information receiving unit 111 is updated by the updating unit 114, and the display unit 115 updates the image data of the image memory of the projector 91. Display information. With the above configuration, the same image as the image displayed on the PC 93 can be projected by the projector 91.
[0043]
Next, the operation of the compression method selection means 107 will be described in detail. The compression method selection means 107 selects a compression method from a list as shown in FIG. 11, for example, as in the first embodiment, according to the application detected by the operation application detection means 101. In the area where the application is not detected, it is assumed that others are selected in the list of FIG.
[0044]
The list shown in FIG. 11 is associated with a compression method according to the type of application. For example, if the detected program is the document creation software M, the list is referred to and the PNG compression method is selected. In the case of a program for creating a document, the displayed image often has a small number of colors, and the capacity after compression can be suppressed at high speed and without distortion by using PNG compression. The software M and the document creation software P are associated with PNG compression. In the case of image display software, natural images are often handled. Therefore, JPEG compression suitable for natural images is used. In the case of moving image display software, natural images often have a correlation with the previous screen. A compression method using an inter-frame difference such as MPEG compression is selected. JPEG compression is selected for the background image.
[0045]
Therefore, according to the present invention, an optimal compression method can be selected according to the operating application, so that even when the transmission capacity of the transmission path is small, there is provided an image display system that operates at a high speed while suppressing image degradation. can do.
[0046]
Note that, in the present embodiment, examples of document creation software, image display software, and moving image display software have been described as applications to be registered, but the present invention is not limited to this. The list in FIG. 11 may be registered in advance, or may be rewritable by the user.
[0047]
Also, the method of detecting a running application in response to a screen update notification of a running application has been described. A method of determining the running application by capturing the entire screen and detecting a change area is described. It may be.
[0048]
In this embodiment, examples of compression formats such as PNG compression, JPEG compression, and MPEG compression have been described. However, a method of reducing the number of colors or a method of reducing the number of pixels of an image may be used, and the present invention is not limited thereto. .
[0049]
【The invention's effect】
As described above, according to the present invention, it is possible to provide an image processing method and an image display system for compressing an image at high speed and with a smaller capacity while suppressing the deterioration of the image.
[Brief description of the drawings]
FIG. 1 is a flowchart illustrating an operation of an image processing method according to a first embodiment of the present invention; FIG. 2 is a block diagram illustrating an image processing apparatus that executes the first embodiment of the present invention; FIG. 4 is a schematic diagram for explaining an operation in the first embodiment; FIG. 4 is a schematic diagram for explaining a method of detecting a changed application in the first embodiment of the present invention; FIG. 5 is a compression diagram in the first embodiment of the present invention; FIG. 6 is a flowchart illustrating an operation of an image processing method according to the second embodiment of the present invention. FIG. 7 is a diagram illustrating a method of selecting a compression ratio according to the second embodiment of the present invention. FIG. 8 is a list for explaining a method of selecting a compression ratio of moving image software according to the second embodiment of the present invention. FIG. 9 is a configuration diagram for explaining a third embodiment of the present invention. 】Book FIG. 11 is a block diagram for explaining the operation of Embodiment 3 of the present invention. FIG. 11 is a list for explaining a method of selecting a compression method in Embodiment 3 of the present invention. FIG. 12 is a block diagram of a conventional image processing apparatus. Block diagram shown [Explanation of reference numerals]
22 Display area of the document creation software A 23 Display area 24 of the video playback software E Background area 25 Change area 91 Projector 92 Screen 93 Personal computer (PC)
94 wireless LAN transceiver 101 operating application detecting means 102 time monitoring means 107 compression method selecting means 108 image compressing means 109 compressed image transmitting means 112 compressed image receiving means 113 image expanding means

Claims (13)

An image processing method for compressing an image of an image signal generating device, wherein for each region to be displayed by the image signal generating device, a step of detecting a program by which the image signal generating device generates an image,
Selecting a compression method according to the detected program;
Compressing the image generated by the image signal generator according to the selected compression method. The image processing method according to claim 1, wherein the compression step is a method of compressing an image generated by the image signal generating device for each display area. 2. The method according to claim 1, wherein the step of selecting a compression method according to the detected program is a step of, when there are a plurality of detected programs, selecting one of the plurality of programs and selecting a compression method. The image processing method described in the above. The image processing method according to claim 1, wherein the image signal generator is a personal computer. An image processing method for compressing an image of an image signal generating device, wherein for each region to be displayed by the image signal generating device, a step of detecting a program by which the image signal generating device generates an image,
Selecting a compression ratio according to the detected program;
Compressing the image generated by the image signal generating device according to a selected compression ratio. The image processing method according to claim 5, wherein the compression step is a method of compressing an image generated by the image signal generation device for each display area. 6. The method according to claim 5, wherein the step of selecting a compression ratio according to the detected program is a step of selecting one of the plurality of detected programs and selecting a compression ratio when there are a plurality of detected programs. The image processing method described in the above. The step of selecting the compression ratio includes:
6. The image processing method according to claim 5, wherein when the detected program is moving image reproduction software, the compression ratio is reduced if the bit rate of the reproduced moving image is high, and the compression ratio is increased if the bit rate is low. The image processing method according to claim 5, wherein the image signal generating device is a personal computer. The image processing method according to claim 9, wherein the step of compressing is JPEG compression. An image signal generator for generating an image signal;
A transmitting device for transmitting the generated image signal,
A display device that receives and displays the image transmitted from the transmitting device,
The image generating device includes a compression unit that captures an image to be generated and compresses the image using a compression method selected according to a running program.
The transmitting device transmits the selected compression method and the compressed image,
The image display system according to claim 1, wherein the display device includes an expansion unit that expands the transmitted compressed image in accordance with a compression method. The image display system according to claim 11, wherein the image signal generator is a personal computer. The image display system according to claim 11, wherein the image display device is a projector.

Images