JP2016219973A - Image processor and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform compression and expansion with simple processing during storing a frame composed of image signals in a storage device.SOLUTION: In an image processor, a storage part supplies a compressed frame being a frame subjected to compression by thinning-out for eliminating a portion of a plurality of image signals in a frame composed of a plurality of image signals. An expansion part expands the compressed frame by restoring the thinned image signals in the supplied compressed frame on the basis of thinning-out information being thinning-out information of each frame.SELECTED DRAWING: Figure 2

Description

  The present technology relates to an image processing device and an image processing method. Specifically, the present invention relates to an image processing apparatus and an image processing method for temporarily storing an image signal in a storage device in the course of image processing.

  Conventionally, an image processing system that processes a stream supplied via the Internet or the like and converts it into an image signal for display is used. When an image signal for display is generated from this stream, various image processing, for example, resolution conversion processing for converting the resolution of the image in accordance with the display device is performed. For each of these processes, an image signal processed in units of frames, which are image signals for one screen, is stored in the memory. Since this frame has a relatively large capacity, it is stored in a large capacity DRAM (Dynamic Random Access Memory). Further, when displaying a moving image, it is necessary to sequentially process time-series frames, and access to the DRAM increases. As a result, there is a problem that power consumption for DRAM access increases. In view of this, a system has been proposed in which a frame is compressed and stored in a DRAM, read out, decompressed and used for the above-described image processing (see, for example, Patent Document 1).

JP 2010-251882 A

  The above-described conventional technique reduces the capacity of the frame by compressing the data before storing it in the DRAM, thereby reducing the power consumption for accessing the DRAM. This compression is performed by thinning the image signal, and two frames generated by different thinning methods are stored in the DRAM. When these are expanded, the original frame is restored by performing super-resolution processing from the two frames. Here, the super-resolution processing is processing for generating one high-resolution frame from a plurality of low-resolution frames. A method is used in which a high-resolution frame is estimated from a low-resolution frame by repeatedly performing an operation. As described above, the above-described conventional technology has a problem that the processing becomes complicated because the two compressed frames are restored by the super-resolution processing.

  The present technology has been created in view of such a situation, and an object thereof is to perform compression and expansion when a frame is stored in a storage device by simple processing.

  The present technology has been made to solve the above-described problems, and a first aspect of the present technology is compression by thinning out a part of the plurality of image signals in a frame composed of the plurality of image signals. The storage unit that supplies the compressed frame that is the frame that has been subjected to the restoration, and the thinned image signal in the supplied compressed frame is restored based on the thinning information that is the thinning information for each frame Accordingly, the image processing apparatus includes a decompression unit that decompresses the compressed frame. As a result, the compressed frame is supplied from the storage unit and expanded.

  The first aspect further includes a thinning information generation unit that generates the thinning information, and a compression unit that performs the compression of the frame by the thinning based on the thinning information and stores the compressed frame in the storage unit. May be. This brings about the effect that the frame is compressed by the thinning-out.

  Further, in this first aspect, the thinning information generation unit generates a thinning rate that is a ratio of the number of image signals to be thinned out with respect to the number of image signals included in the frame as the thinning information, The compression unit may perform the thinning by determining the number of image signals to be thinned based on the thinning rate. This brings about the effect that thinning is performed based on the thinning rate.

  In the first aspect, the compression unit may generate a plurality of arrangement patterns, which are information on the arrangement of the image signals to be thinned out, on a line obtained by dividing the frame in the horizontal direction for each line in a predetermined order. The compression may be performed by applying to the thinning, and the decompression unit may perform the decompression by applying the arrangement pattern to the restoration for each line in the predetermined order. This brings about the effect that the arrangement pattern is applied to the thinning and the restoration in a preset order.

  In the first aspect, the thinning information generation unit may generate the thinning rate and thinning position information that is information in the predetermined order as the thinning information. As a result, the thinning rate and the thinning position information are generated as thinning information.

  In the first aspect, the thinning information generation unit may generate, as the thinning information, the thinning position information that is different for each of the plurality of frames that are continuous with the thinning rate. This brings about the effect | action that the said thinning | decimation position information different for every said continuous several said flame | frame is produced | generated.

  In the first aspect, the compression unit may perform the compression based on the arrangement pattern in which the image signals to be thinned are arranged not adjacent to each other in the horizontal direction. This brings about the effect | action that the said image signal used as the said thinning-out object becomes the arrangement | positioning which is not adjacent in a horizontal direction.

  The first aspect further includes a frame mixing unit that generates a new frame by mixing the second frame, which is a frame different from the frame, and the frame based on a predetermined mixing ratio, The thinning information generation unit may generate the thinning rate of the frame and the second frame as the thinning information based on the mixing ratio. This brings about the effect that the thinning rate of the frame and the second frame based on the mixing ratio is generated as the thinning information.

  In the first aspect, the information processing apparatus further includes a thinning information generation unit that generates the thinning information. The storage unit stores the frame, and the frame is stored by the thinning based on the thinning information. You may compress and supply to the said expansion | extension part. This brings about the effect | action that the said flame | frame is compressed in a memory | storage part.

  In addition, according to a second aspect of the present technology, the thinning is performed on the compressed frame, which is the frame that is compressed by thinning to remove a part of the plurality of image signals in a frame including a plurality of image signals. The image processing method includes a decompression procedure for decompressing the compressed frame by restoring the image signal based on the thinning information which is the thinning information for each frame. As a result, the compressed frame is supplied from the storage unit and expanded.

  According to the present technology, it is possible to achieve an excellent effect of performing compression and expansion when storing a frame in a storage device by a simple process. Note that the effects described here are not necessarily limited, and may be any of the effects described in the present disclosure.

It is a figure showing an example of composition of an image processing device in a 1st embodiment of this art. It is a figure showing an example of functional composition of an image processing device in a 1st embodiment of this art. It is a figure showing an example of processing of image processing part 100 in a 1st embodiment of this art. It is a figure which shows an example of the compression method in embodiment of this technique. It is a figure showing an example of thinning information in a 1st embodiment of this art. It is a figure which shows the example of the arrangement | positioning pattern (thinning-out rate 1/8) in embodiment of this technique. It is a figure which shows the example of the arrangement pattern (thinning-out rate 2/8) in embodiment of this technique. It is a figure showing an example of arrangement pattern (thinning rate of 3/8) in an embodiment of the present technology. It is a figure showing an example of arrangement pattern (thinning rate of 4/8) in an embodiment of this art. It is a figure showing an example of composition of compression part 130 in a 1st embodiment of this art. It is a figure which shows an example of the expansion | extension method in embodiment of this technique. It is a figure showing an example of a restoration method of an image signal in an embodiment of this art. It is a figure showing an example of composition of decompression part 140 in a 1st embodiment of this art. It is a figure showing an example of compression processing in a 1st embodiment of this art. It is a figure showing an example of decompression processing in a 1st embodiment of this art. It is a figure showing an example of the compression method in a 2nd embodiment of this art. It is a figure showing an example of thinning information in a 2nd embodiment of this art. It is a figure showing an example of composition of an image processing device in a 3rd embodiment of this art. It is a figure showing an example of composition of frame processing part 150 in a 3rd embodiment of this art. It is a figure showing an example of composition of resolution conversion part 160 in a 3rd embodiment of this art. It is a figure showing an example of composition of OSD image generation part 170 in a 3rd embodiment of this art. It is a figure showing an example of composition of OSD image display part 180 in a 3rd embodiment of this art. It is a figure which shows the function structural example of the image process part 100 in 4th Embodiment of this technique. It is a figure showing an example of composition of storage part 200 in a 5th embodiment of this art. It is a figure showing an example of processing of image processor 100 in a 5th embodiment of this art.

Hereinafter, modes for carrying out the present technology (hereinafter referred to as embodiments) will be described. The description will be made in the following order.
1. First embodiment (example when using a thinning rate as thinning information)
2. Second embodiment (example in which thinning rate and thinning position information are used as thinning information)
3. Third embodiment (example in the case of using a plurality of compression units and expansion units)
4). Fourth embodiment (example in which two frames are mixed and displayed)
5. Fifth embodiment (example when the storage unit includes a compression unit)

<1. First Embodiment>
[Configuration of image processing apparatus]
FIG. 1 is a diagram illustrating a configuration example of an image processing device according to the first embodiment of the present technology. This image processing apparatus includes an image processing unit 100 and a storage unit 200.

  The image processing unit 100 processes an input stream to generate a frame which is an image signal for one screen, and outputs a display frame converted into a format suitable for the display device 300. The output display frame is displayed on the display device 300.

The storage unit 200 stores data such as frames generated in the course of processing in the image processing unit 100. The storage unit 200 supplies the compressed frame to the image processing unit 100 as will be described later. The storage unit 200 can be a DRAM that can be accessed at high speed, such as SDRAM (Synchronous DRAM).
[Configuration of image processing unit]

  The image processing unit 100 in FIG. 1 includes a demultiplexer 102, a processor 103, a storage unit interface 104, a video decoder 120, a compression unit 130, a decompression unit 140, a frame processing unit 150, and a resolution conversion unit 160. Is provided. The image processing unit 100 further includes an on-screen display (OSD) image generation unit 170, an OSD image display unit 180, and a mixing unit 190. These are connected to each other by a bus 101.

  The demultiplexer 102 performs demultiplexing. This demultiplexing is a process of separating multiplexed image data, audio data, and caption data from a stream.

  The processor 103 controls the entire image processing unit 100. The processor 103 controls each unit of the image processing unit 100 based on the firmware. The firmware is stored in a ROM (Read Only Memory) (not shown).

  The storage unit interface 104 is an interface for exchanging with the storage unit 200.

  The video decoder 120 converts the image data separated by the demultiplexer 102 into a plurality of continuous frames. The video decoder 120 converts the image data into frames by decoding the encoded image data.

  The compression unit 130 compresses frames based on thinning information described later. Here, reducing the data amount of the frame is referred to as compression. The compressed frame (hereinafter referred to as a compressed frame) is stored in the storage unit 200. Details of compression in the compression unit 130 will be described later.

  The decompression unit 140 decompresses the compressed frame held in the storage unit 200 based on the thinning information. Here, returning the compressed frame to the original frame is referred to as decompression. Details of expansion in the expansion unit 140 will be described later.

  The frame processing unit 150 processes a frame. This frame processing includes, for example, interlace / progressive conversion processing for changing the frame scanning method between interlace and progressive, noise reduction processing for removing noise contained in the frame, and the like.

  The resolution converter 160 converts the resolution of the frame. The resolution conversion unit 160 converts the frame resolution to the display device 300 resolution when the frame resolution and the display device 300 resolution are different.

  The OSD image generation unit 170 generates an on-screen display (OSD) image. Here, the OSD image is a user interface such as a subtitle portion, an icon, and a menu, and is an image that is displayed by being superimposed on an image supplied by a stream or the like. This OSD image is also a kind of frame, and is processed as a target of compression by the compression unit 130 and expansion by the expansion unit 140.

  The OSD image display unit 180 converts the resolution of the OSD image generated by the OSD image generation unit 170 into the resolution of the display device 300.

  The mixing unit 190 generates a single frame by mixing the frame and the OSD image whose resolution has been converted by the resolution conversion unit 160 and the OSD image display unit 180, respectively. The generated frame is output to the display device 300 as a display frame. As a mixing method in the mixing unit 190, alpha blend that mixes a frame and an OSD image based on a predetermined mixing ratio can be used. The mixing ratio is determined by the processor 103 based on the frame image and the like.

  Note that the demultiplexer 102 and the like in the figure are realized by dedicated hardware, but can also be realized as processing by software executed by the processor 103.

  Next, data exchange between the image processing unit 100 and the storage unit 200 will be described. When storing a frame or the like in the storage unit 200, the processor 103 generates a command for instructing writing (write command) and outputs the command to the storage unit 200. Along with this command, a write destination address and write target data are output to the storage unit 200. The storage unit 200 interprets the write command and stores the write target data at the write destination address. On the other hand, when a frame or the like is read from the storage unit 200, the processor 103 generates a command (read command) that instructs reading, and outputs the command to the storage unit 200. Along with this command, a read destination address is output to the storage unit 200. The storage unit 200 interprets the read command and outputs the data stored at the read destination address to the image processing unit 100. In this manner, data is exchanged between the image processing unit 100 and the storage unit 200.

  FIG. 2 is a diagram illustrating a functional configuration example of the image processing apparatus according to the first embodiment of the present technology. This figure shows the functional configuration of each unit of the image processing unit 100 and the storage unit 200 and the flow of data such as frames. The image processing unit 100 shown in the figure is different from the image processing unit 100 described in FIG. 1 in that a thinning information generation unit 110 is provided. The thinning information generation unit 110 generates thinning information, which will be described later, based on a frame, and supplies the thinning information to the compression unit 130 and the decompression unit 140. The processing of the thinning information generation unit 110 is executed by software, for example, by the processor 103 described with reference to FIG.

  The stream input to the image processing apparatus is separated into image data, audio data, and caption data by the demultiplexer 102 and stored in the storage unit 200. Of these, the audio data is converted into audio and output by a speaker or the like (not shown).

  On the other hand, the image data is read from the storage unit 200 and input to the video decoder 120. The video decoder 120 converts the image data into frames and stores them in the storage unit 200. Next, this frame is read from the storage unit 200 and input to the frame processing unit 150. Frames processed by the frame processing unit 150 (hereinafter referred to as processed frames) are compressed by the compression unit 130 and stored in the storage unit 200. At this time, the thinning information generation unit 110 generates thinning information to be described later based on the frame, and the compression unit 130 performs compression based on the generated thinning information. Next, the compressed processed frame (hereinafter referred to as a processed compressed frame) is read from the storage unit 200, decompressed by the decompressing unit 140, and converted into a processed frame again. The resolution of the processed converted frame is converted by the resolution conversion unit 160 and input to the mixing unit 190.

  The caption data is read from the storage unit 200 and input to the OSD image generation unit 170. An OSD image is generated based on the caption data. The OSD image generation unit 170 also generates an OSD image when displaying icons and menus. These generated OSD images are compressed by the compression unit 130 and stored in the storage unit 200. Also at this time, compression by the compression unit 130 is performed based on the thinning information generated by the thinning information generation unit 110. Next, the compressed OSD image is read from the storage unit 200, decompressed by the decompression unit 140, and input to the OSD image display unit 180. The OSD image display unit 180 converts the resolution of the OSD image and inputs it to the mixing unit 190.

  The processed frame whose resolution has been changed and the OSD image are mixed by the mixing unit 190 and output as a display frame.

  As described above, the processed frame and the OSD image are compressed by the compression unit 130 and stored in the storage unit 200. The compressed frames are read from the storage unit 200 and then expanded by the expansion unit 140. As a result, the amount of data transferred to and from the storage unit 200 can be reduced, and power consumption can be reduced.

  FIG. 3 is a diagram illustrating an example of processing of the image processing unit 100 according to the first embodiment of the present technology. This figure shows the flow of processing in each part of the functional configuration diagram described in FIG. In the drawing, the processing of the portion denoted as “DRAM” represents processing for storing a frame or the like in the storage unit 200 or processing for reading a frame or the like from the storage unit 200.

  First, the stream input to the image processing unit 100 is demultiplexed by the demultiplexer 102, and the image data separated from the stream is stored in the storage unit 200. Next, the image data stored in the storage unit 200 is read out, decoded by the video decoder 120 to generate a frame, and stored in the storage unit 200. Next, the frame stored in the storage unit 200 is read, and the thinning information generation unit 110 generates thinning information based on the read frame. The generated thinning information is used for subsequent compression processing in the compression unit 130 and expansion processing in the expansion unit 140. The frame read from the storage unit 200 is processed by the frame processing unit 150 to generate a processed frame, and is compressed by the compression unit 130 and stored in the storage unit 200. Next, the processed compressed frame read from the storage unit 200 is expanded by the expansion unit 140 and the resolution is converted by the resolution conversion unit 160.

  On the other hand, the OSD image is generated by the OSD image generation unit 170, compressed by the compression unit 130, and stored in the storage unit 200. Next, the compressed OSD image read from the storage unit 200 is expanded by the expansion unit 140, and the resolution is converted by the OSD image display unit 180.

  Finally, the processed frame and the OSD image whose resolution has been changed are mixed by the mixing unit 190.

[Compression method]
FIG. 4 is a diagram illustrating an example of a compression method according to the embodiment of the present technology. Circles shown in the figure represent image signals in a frame. Thus, the frame is configured by arranging image signals in two dimensions. A plurality of image signals when a frame is divided in the horizontal direction corresponds to a line. The frame shown in the figure has n lines. The numbers assigned to the image signals in the figure represent the positions from the beginning of each line.

  The compression in the embodiment of the present technology is performed by thinning out image signals. Here, the thinning is to remove a part of a plurality of image signals constituting a frame. An image signal 501 indicated by a dotted line in the image signal in FIG. Compression can be performed by removing the image signal to be thinned out and converting the image signal into a frame composed of only another image signal 502. The thinning can be performed based on a thinning rate that is a ratio of the number of image signals to be thinned out to the number of image signals included in the frame. In the figure, one image signal is thinned out for every eight image signals, and the thinning rate becomes 1/8. The higher the thinning rate, the smaller the number of image signals in the compressed frame, and the higher the compression rate. Further, the thinning rate can be set to the same value for all the lines constituting the frame.

  This thinning rate corresponds to the thinning information described above, and is generated for each frame by the thinning information generation unit 110 described with reference to FIG. At this time, the thinning information generation unit 110 can generate thinning information based on the resolution of the frame and the like. For example, it is possible to generate thinning information with a high thinning rate when a frame has a high resolution or when a plurality of images are displayed like a multi-screen, and with a low thinning rate when the frame has a low resolution.

  In the figure, the image signals 501 to be thinned out in the line are arranged so as not to be adjacent in the horizontal direction. Thereby, it is possible to prevent the image signals 501 to be thinned out from being concentrated on the line. As will be described later, when the compressed frame is converted into the original frame, a process of restoring the thinned image signal is performed. This is because when the restored image signal is concentrated at a specific position in the line, the image is noticeably deteriorated and the image quality is lowered. In the figure, the position of the image signal 501 to be thinned is shifted for each line. Thereby, it is possible to prevent the image signals to be thinned out from concentrating on a specific column of the frame. This is to prevent the restored image signal from being concentrated on a specific column. As shown in the figure, when the thinning rate is 1/8, there are eight combinations of the arrangement of image signals to be thinned and other image signals. Here, this combination is referred to as an arrangement pattern. By thinning out the eight arrangement patterns in order from the first line in the frame, the position of the image signal 501 to be thinned can be shifted for each line.

[Thinning information]
FIG. 5 is a diagram illustrating an example of thinning information according to the first embodiment of the present technology. As shown in the figure, the thinning rate for each frame is recorded as thinning information. This thinning information may be stored in the storage unit 200 after being compressed by the compression unit 130. When the decompression unit 140 decompresses the compressed frame, the thinning information corresponding to the compressed frame can be read from the storage unit 200 and used. It should be noted that this thinning-out information can use a method of holding together with image information for each frame, for example, image parameters such as display position and color format. Further, the thinning information can be added to the compressed frame and stored in the storage unit 200. In this case, the compression unit 130 adds thinning information to the compressed frame, and the decompression unit 140 extracts the thinning information from the compressed frame and performs decompression. Furthermore, it is possible to adopt a configuration in which thinning information is added for each line of the frame. In this case, thinning information is generated for each line. As a result, the thinning rate can be changed for each line.

[Placement pattern]
FIG. 6 is a diagram illustrating an example of an arrangement pattern (thinning rate 1/8) according to the embodiment of the present technology. In the figure, a represents an example of an arrangement pattern when the thinning rate is 1/8. This is an example in which one thinning target image signal 501 is arranged for eight image signals as in FIG. In the figure, b represents an example of an arrangement pattern when the thinning rate is 1/8. As described above, when the thinning rate is 1/8, there are eight arrangement patterns. These eight arrangement patterns are sequentially applied to each line, and thinning is performed.

  In the figure, b represents an example in which thinning is performed by applying an arrangement pattern in the order in which the image signal 501 to be thinned is shifted to the right for each line. The arrangement pattern can be identified by the thinning position information. In FIG. 6B, 1 to 8 are assigned as thinning position information to each of the eight arrangement patterns. The order of application of these arrangement patterns to each line in the frame can be determined in advance, and the compression and decompression of the frame can be performed based on this order. For example, it can be determined that the arrangement pattern corresponding to the thinning position information 1 to 8 is applied to each line in this order. In this case, an arrangement pattern corresponding to the thinning position information 1 to 8 is applied in order from the first line of the frame, and is used for compression and expansion.

The position of the image signal to be thinned out in each line can be calculated by the following equation.
N = ((a + b−2)% 8 + 1) + 8 × n (Expression 1)
However, N represents the position from the head in each line of the image signal to be thinned. a represents the position of the image signal to be thinned out according to the thinning rate. In the example of the figure, a = 1. n represents 0 and a positive integer. That is, n = 0, 1, 2,. Note that% is an operator representing a remainder operation.

  FIG. 7 is a diagram illustrating an example of an arrangement pattern (thinning rate 2/8) according to the embodiment of the present technology. In the figure, “a” represents an example of an arrangement pattern when the thinning rate is 2/8, and two image signals to be thinned are arranged for eight image signals. B in the figure represents an example of an arrangement pattern when the thinning rate is 2/8. The thinning-out position information b in FIG. 10 is the same as that in FIG. Also in the figure, the position of the image signal to be thinned out in each line can be calculated by Equation 1. However, a = 1 and 5.

  FIG. 8 is a diagram illustrating an example of an arrangement pattern (thinning rate 3/8) according to the embodiment of the present technology. In the figure, “a” represents an example of an arrangement pattern when the thinning rate is 3/8, and three image signals to be thinned are arranged for eight image signals. B in the figure represents an example of an arrangement pattern when the thinning rate is 3/8. The position of the image signal to be thinned out in each line can be calculated by setting a = 1, 3, 5 in Equation 1.

  FIG. 9 is a diagram illustrating an example of an arrangement pattern (thinning rate 4/8) according to the embodiment of the present technology. In the figure, “a” represents an example of an arrangement pattern when the thinning rate is 4/8, and four image signals to be thinned are arranged for eight image signals. B in the figure represents an example of an arrangement pattern when the thinning rate is 4/8. The position of the image signal to be thinned out in each line can be calculated by setting a = 1, 3, 5, 7 in Equation 1.

  As described above, in order to arrange the image signals 501 to be thinned out in the line so as not to be adjacent in the horizontal direction, the thinning rate needs to be 4/8, that is, ½ at the maximum. For this reason, the arrangement pattern of the figure becomes the maximum thinning-out rate in the embodiment of the present technology. In FIG. 6B, the image signal is thinned out for each line in the oblique direction, and the resolution in the oblique direction is halved. On the other hand, in the vertical and horizontal directions of the image signal, the same resolution as before the thinning is maintained, and the expansion processing described later can be easily performed. In many natural images, the resolution in the vertical and horizontal directions is higher than the resolution in the oblique direction. Therefore, the compression and expansion method according to the embodiment of the present technology is a method suitable for a system that processes natural images.

[Configuration of compression unit]
FIG. 10 is a diagram illustrating a configuration example of the compression unit 130 according to the first embodiment of the present technology. The compression unit 130 includes an image signal removal unit 131 and an image signal rearrangement unit 132.

  The image signal removal unit 131 removes the image signal to be thinned out from the input frame based on the thinning rate in the input thinning information. The image signal removing unit 131 applies the arrangement pattern corresponding to the thinning rate to thinning for each line in a predetermined order. At this time, the image signal removal unit 131 can employ a method of holding the input frame in a line memory or the like and deleting data at an address corresponding to the image signal to be thinned out.

  The image signal rearrangement unit 132 rearranges the image signals not to be removed at the position of the image signal removed by the image signal removal unit 131 to reduce the frame size. The rearrangement unit 132 can perform processing of moving the image signal data that is not the removal target on the line memory so as to fill the address data corresponding to the thinning target image signal in the line memory.

  It is also possible to compress the data by overwriting the data of the address corresponding to the image signal to be thinned out in the line memory with the data of the image signal not to be thinned out and rearranging the data.

[Extension method]
FIG. 11 is a diagram illustrating an example of the decompression method according to the embodiment of the present technology. First, an empty image signal 503 is inserted at the position of the image signal to be thinned based on the arrangement pattern. Next, the image signal thinned out based on the image signal around the inserted image signal 503 is restored. The restored image signal 504 is overwritten at the position of the image signal 503. As a result, the frame can be expanded.

[Restore image signal]
FIG. 12 is a diagram illustrating an example of the image signal restoration method according to the embodiment of the present technology. The figure shows an example in the case of restoring the image signal at the position of the empty image signal 503, and images are obtained from the image signals (image signals 505 to 508) arranged on the upper, lower, left and right sides of the empty image signal 503. In this example, the signal 504 is restored by interpolation. As this method, a method of calculating an average value of the image signals 505 to 508 and interpolating the image signal 504 can be used. Further, a method is used in which a weighted average obtained by weighting based on the correlation between the upper and lower image signals (image signals 505 and 506) and the left and right image signals (image signals 507 and 508) is calculated and the image signal 504 is interpolated. You can also. Specifically, the difference between the upper and lower image signals and the left and right image signals is calculated, and the weighted average is calculated by weighting the image signal with the smaller difference (upper and lower image signals or left and right image signals). can do. Thereby, interpolation based on the correlation can be performed. Further, even in the case of the thinning-out rate 4/8 described in FIG. 9, interpolation based on the correlation can be performed using the upper, lower, left, and right image signals of the image signal to be restored.

[Configuration of the extension unit]
FIG. 13 is a diagram illustrating a configuration example of the decompression unit 140 according to the first embodiment of the present technology. The decompression unit 140 includes an image signal insertion unit 141 and a restoration unit 142.

  The image signal insertion unit 141 inserts an empty image signal at the position of the input image signal that is thinned out based on the input thinning information. The image signal insertion unit 141 applies the arrangement pattern for each line based on the same order as that applied to the thinning in the compression unit 130 described above, and grasps the position of the thinned image signal. Then, an empty image signal 503 is inserted at this position. At this time, the image signal insertion unit 141 can take a method of holding the input compressed frame in a line memory or the like and inserting empty data at the position of the thinned image signal.

  The restoration unit 142 restores the thinned image signal. As described above, the restoration unit 142 interpolates the value of the image signal 504 by performing calculation using the upper, lower, left, and right image signals of the empty image signal 503, and sets the address corresponding to the empty image signal 503. It can be restored as data.

[Compression processing]
FIG. 14 is a diagram illustrating an example of the compression processing according to the first embodiment of the present technology. The compression process of FIG. 9 is executed when the compression unit 130 compresses a frame. First, the thinning information generation unit 110 generates thinning information based on a frame to be compressed (step S901). Next, based on the thinning information, the compression unit 130 removes the image signal to be thinned in the first line of the frame (step S902). At this time, the arrangement pattern corresponding to the thinning rate of the thinning information is applied in a predetermined order. Next, the image signal is rearranged by the compression unit 130 (step S903). Next, it is determined whether or not processing has been performed for all the lines of the frame (step S904). If the next processing target line exists (step S904: Yes), the arrangement pattern is changed (step S905), and the processing from step S902 is executed again. On the other hand, when the process is completed for all lines (step S904: No), the compression unit 130 ends the compression process.

[Extension processing]
FIG. 15 is a diagram illustrating an example of the decompression process according to the first embodiment of the present technology. The decompression process in FIG. 10 is executed when the decompression unit 140 decompresses a frame. First, thinning information of a compressed frame to be decompressed is acquired (step S951). This is performed by reading out the thinning information corresponding to the compressed frame from the storage unit 200. Next, based on the thinning information, the decompression unit 140 inserts an empty image signal at the position of the thinned image signal in the first line of the frame (step S952). At this time, the arrangement pattern corresponding to the thinning rate of the thinning information is applied in a predetermined order. Next, the image signal thinned out by the decompression unit 140 is restored (step S953). Next, it is determined whether or not processing has been performed for all lines of the frame (step S954). If the next processing target line exists (step S954: Yes), the arrangement pattern is changed (step S955), and the processing from step S952 is executed again. On the other hand, when the processing is completed for all lines (step S954: No), the decompression unit 140 terminates the decompression processing.

  As described above, in the first embodiment of the present technology, it is possible to compress and expand a frame by thinning and interpolation of image signals, which are relatively simple processes.

<2. Second Embodiment>
In the above-described embodiment, compression and decompression are performed in the application order of the same arrangement pattern in a frame to which the same thinning rate is applied. On the other hand, in the second embodiment of the present technology, the arrangement pattern application order is changed for each of a plurality of consecutive frames. As a result, deterioration of image quality is prevented.

[Compression method]
FIG. 16 is a diagram illustrating an example of a compression method according to the second embodiment of the present technology. This figure corresponds to the case where the thinning rate is 1/8 and shows the application order of the arrangement pattern in the first to fourth frames which are continuous frames. In the first frame in the figure, as in FIG. 6, the arrangement pattern is applied in the order of the thinning position information 1 to 8, and these are applied in order from the first line. In the subsequent second frame, although the arrangement pattern application order does not change in the order of the thinning position information 1 to 8, the arrangement pattern of the thinning position information 2 is applied to the first line. Similarly, in the third frame, the arrangement pattern of the thinning position information 3 is applied to the first line. As described above, the arrangement pattern of the thinning position information 1 to 8 is circulated and applied to each line of the frame, while the arrangement pattern starting from each frame is shifted and applied to each line. As described above, in the second embodiment of the present technology, the application order of the arrangement pattern is changed for each frame. Thereby, the thinning-out target image signal can be distributed to different positions in consecutive frames, and deterioration in image quality can be prevented.

  The application order of the arrangement pattern can be identified by, for example, thinning position information corresponding to the arrangement pattern applied to the first line. In the example shown in the figure, the application order of the arrangement pattern can be identified by corresponding the thinning position information 1 to 4 to the first to fourth frames, respectively.

[Thinning information]
FIG. 17 is a diagram illustrating an example of thinning information according to the second embodiment of the present technology. As shown in the figure, the thinning rate and the thinning position information for each frame are recorded as the thinning information.

  Since the other configuration of the image processing apparatus is the same as that of the first embodiment of the present technology, the description thereof is omitted.

  As described above, according to the second embodiment of the present technology, the positions of the thinned image signals in a plurality of consecutive frames are dispersed, so that the image quality can be improved.

<3. Third Embodiment>
In the first embodiment described above, compression and decompression are performed by the compression unit 130 and the decompression unit 140. On the other hand, in the third embodiment of the present technology, processing is performed by a plurality of compression units and decompression units, respectively. Thereby, the processing of the image processing unit 100 can be performed at high speed.

[Configuration of image processing apparatus]
FIG. 18 is a diagram illustrating a configuration example of an image processing device according to the third embodiment of the present technology. This image processing apparatus does not need to include the compression unit 130 and the expansion unit 140 as compared with the image processing apparatus described with reference to FIG. This is because the frame processing unit 150, the OSD image generation unit 170, the resolution conversion unit 160, and the OSD image display unit 180 have a compression or expansion function, as will be described later.

[Configuration of frame processing unit]
FIG. 19 is a diagram illustrating a configuration example of the frame processing unit 150 according to the third embodiment of the present technology. It is assumed that the frame processing unit 150 according to the third embodiment of the present technology performs interlace / progressive conversion and noise reduction as frame processing. The frame processing unit 150 includes an interlace progressive conversion (IPC) unit 152, a noise reduction unit 153, a compression unit 154, and a line memory 155.

  The line memory 155 is a memory that temporarily holds a frame. The IPC unit 152 changes the frame scanning method between interlaced and progressive. The IPC unit 152 performs processing by holding the frame on the line memory. For example, when the frame scanning method is converted from interlaced to progressive, two consecutive frames are held in the line memory 155. Next, conversion is performed by taking out each line of these frames in order and combining them into one frame.

  The noise reduction unit 153 removes noise included in the frame. The noise reduction unit 153 can perform, for example, a two-dimensional noise reduction process that attenuates high-frequency components of image signals belonging to the same frame. In this two-dimensional noise reduction process, a frame is held in the line memory 155, and an average value of adjacent image signals is calculated to generate a new image signal, thereby leveling the image signal within the frame and removing noise. I do. This method is mainly used for removing noise from the moving image part. In addition, for example, it is possible to perform a three-dimensional noise reduction process that removes noise using image signals of successive frames. In this three-dimensional noise reduction process, a plurality of frames are held in the line memory 155, and an average value between frames of each image signal is calculated to generate a new frame. As a result, the image signal is leveled in successive frames to remove noise. This method is mainly used to remove noise from still image portions.

  The compression unit 154 performs frame compression based on the thinning information. The compression unit 154 performs processing such as thinning of the image signal described with reference to FIG. At this time, the frame is held in the line memory 155, and image signals are removed and rearranged on the line memory 155.

[Configuration of resolution converter]
FIG. 20 is a diagram illustrating a configuration example of the resolution conversion unit 160 according to the third embodiment of the present technology. The resolution conversion unit 160 includes an expansion unit 161, a resolution conversion processing unit 162, and a line memory 165.

  The line memory 165 is a memory that temporarily holds a frame. The decompression unit 161 decompresses the compressed frame based on the thinning information. The decompression unit 161 performs processing such as insertion of an image signal described in FIG. At this time, the compressed frame is held in the line memory 165, and an image signal is inserted and restored on the line memory 165.

  The resolution conversion processing unit 162 performs processing for converting the resolution in the same manner as the resolution conversion unit 160 described with reference to FIG. For example, when a high-resolution stream image (for example, a 4K image) is displayed on a display device that supports 2K images, the resolution conversion processing unit 162 performs resolution conversion. In this case, the conversion is performed by halving the number of frame image signals. As this conversion processing, for example, an average value of two image signals adjacent in the horizontal and vertical directions of the original frame (high-resolution image) is calculated, and this value is converted into a new frame image signal. be able to. At this time, the resolution conversion processing unit 162 stores the original frame in the line memory 165 and performs the above-described processing on the line memory 165. In order to perform such processing, the line memory 165 is configured with a storage capacity capable of holding a frame. By using the line memory 165 in common with the above-described decompression unit 161, the interpolation processing described in FIG. 12 can be performed with high accuracy. More specifically, in FIG. 12, the image signals adjacent to the image signal 503 (image signals 505 to 508) are used when calculating the weighted average obtained by performing weighting based on the correlation. On the other hand, in the decompression unit 161 according to the third embodiment of the present technology, the line memory 165 having a large storage capacity can be used, so that the correlation detection range can be expanded. For this reason, more accurate correlation can be detected, and highly accurate interpolation processing can be performed.

[Configuration of OSD image generation unit]
FIG. 21 is a diagram illustrating a configuration example of the OSD image generation unit 170 according to the third embodiment of the present technology. The OSD image generation unit 170 includes an OSD image generation processing unit 171, a compression unit 174, and a line memory 175.

  The line memory 175 is a memory that temporarily holds an OSD image. The OSD image generation processing unit 171 performs the same processing as the OSD image generation unit 170 described with reference to FIG. The OSD image generation processing unit 171 generates the OSD image (caption) by combining the input caption data with the image of the caption display area built in the OSD image generation unit 170. At this time, the image of the caption display area is held on the line memory 175, and the caption data is combined on the line memory 175.

  The compression unit 174 compresses a frame (OSD image). The operation of the compression unit 174 is the same as that of the compression unit 154 described in FIG.

[Configuration of OSD image display unit]
FIG. 22 is a diagram illustrating a configuration example of the OSD image display unit 180 according to the third embodiment of the present technology. The OSD image display unit 180 includes an expansion unit 181, an OSD image resolution conversion unit 182, and a line memory 185.

  The line memory 185 is a memory that temporarily holds an OSD image. The decompression unit 181 decompresses a frame (OSD image) compressed based on the thinning information. The operation of the expansion unit 181 is the same as that of the expansion unit 161 described in FIG.

  The OSD image resolution conversion unit 182 performs the same processing as the OSD image display unit 180 described with reference to FIG. The OSD image resolution conversion unit 182 converts the OSD image generated by the OSD image generation unit 170 into an OSD image having a resolution suitable for the display device by deleting an image signal. At this time, the OSD image resolution conversion unit 182 stores the OSD image in the line memory 185 and performs processing for converting the resolution on the line memory 185.

  Since the other configuration of the image processing apparatus is the same as that of the first embodiment of the present technology, the description thereof is omitted.

  As described above, the frame processing unit 150 and the OSD image generation unit 170 according to the third embodiment of the present technology each include the compression units 154 and 174, and the resolution conversion unit 160 and the OSD image display unit 180 each include the decompression unit 161. And 181 are incorporated. The compression unit and decompression unit perform processing using a line memory included in the frame processing unit 150 and the like. Therefore, it is possible to individually perform compression and expansion processing in the frame processing unit 150 and the like without increasing the number of line memories necessary for compression and expansion processing. When displaying a moving image, it is necessary to sequentially process consecutive frames. When the frame processing unit 150 or the like individually performs compression and decompression processing and performs storage and reading with respect to the storage unit 200, processing of each unit in the image processing unit 100 can be made into pipeline processing, Processing can be performed at high speed.

<4. Fourth Embodiment>
In the above-described first embodiment, processing is performed for one frame. On the other hand, the fourth embodiment of the present technology proposes a case where two frames are mixed and displayed.

  FIG. 23 is a diagram illustrating a functional configuration example of the image processing unit 100 according to the fourth embodiment of the present technology. This figure describes the thinning information generation unit 110, compression unit 130, decompression unit 140, frame processing unit 150, resolution conversion unit 160, OSD image display unit 180, and mixing unit 190 in the image processing unit 100. is there. The image processing unit 100 according to the fourth embodiment of the present technology simultaneously processes two pieces of image data to generate two frames, mixes these into one frame, and outputs the mixed frame to the display device 300. This mixing is performed by the mixing unit 190 based on a mixing ratio described later. This mixing ratio is supplied by the processor 103.

  The thinning information generation unit 110 generates thinning information of each frame based on a mixture ratio and two frames (frame 1 and frame 2), which will be described later, and supplies them to the compression unit 130 and the expansion unit 140.

  The frame processing unit 150 performs processing for two frames. The compression unit 130 compresses the two processed frames and the OSD image, and stores them in the storage unit 200. The decompression unit 140 reads out the two processed compressed frames and the compressed OSD image from the storage unit 200 and performs decompression. The resolution converter 160 converts the resolution of the two processed frames.

  The mixing unit 190 mixes the two resolution-converted frames and the resolution-converted OSD image to generate a display frame. The mixing unit 190 includes a frame mixing unit 191 and a frame OSD image mixing unit 192.

  The frame OSD image mixing unit 192 generates a display frame by mixing the frame and the OSD image, similarly to the mixing unit 190 described in FIG.

The frame mixing unit 191 performs mixing of two frames. The frame mixing unit 191 mixes two resolution converted frames based on a predetermined mixing ratio. This mixing can be performed as follows.
Q = α × P1 + (1−α) × P2
However, P1 and P2 are the image signals of the frame before mixing, Q is the image signal of the frame after mixing, and α is the mixing ratio. In this way, the frame mixing unit 191 multiplies each frame by α or 1-α, and adds these to mix the two frames in a translucent state. Such mixing is referred to as alpha blending. When the mixing ratio is 0.5, the ratio of the two frames to be mixed is equal. However, when one frame is displayed thinner than the other frame, mixing is performed by reducing the ratio of the frame. The thinning rate can be increased for the thinly displayed frame. This is because the display quality is not noticeable even when the number of image signals to be thinned is large because the display is thin.

  Therefore, the thinning information generation unit 110 generates a thinning rate for two frames based on the mixing ratio. Specifically, the thinning rate of a frame in which a low mixing ratio is set is set to a value higher than the thinning rate of the other frame and is supplied to the compression unit 130 and the like. As a result, the thinning-out frame thinning rate can be increased and the frame capacity can be reduced.

  In the image processing unit 100 described above, the compression unit 130, the decompression unit 140, the frame processing unit 150, and the resolution conversion unit 160 process two frames at the same time. However, the image processing unit 100 described with reference to FIG. Two sets may be arranged. This is because two frames can be processed simultaneously. The other configuration of the image processing unit 100 is the same as the configuration of the image processing unit described with reference to FIG.

  As described above, according to the fourth embodiment of the present technology, when alpha blending is performed on two frames, by increasing the thinning rate of thinly displayed frames and storing them in the storage unit 200, The power consumption during access can be further reduced.

<5. Fifth embodiment>
In the first embodiment described above, the image processing unit 100 includes the compression unit 130 and the expansion unit 140, and the storage unit 200 stores the compressed frame. On the other hand, in the fifth embodiment of the present technology, the storage unit 200 includes a compression unit, and performs compression when outputting a frame to the image processing unit 100. This reduces the frequency of compression processing.

  FIG. 24 is a diagram illustrating a configuration example of the storage unit 200 according to the fifth embodiment of the present technology. The storage unit 200 includes an image processing unit interface 210, a control unit 220, a column address decoder 240, a row address decoder 250, a memory cell array 260, and a compression unit 230.

  The image processing unit interface 210 is an interface for exchanging with the image processing unit 100. The image processing unit interface 210 interprets the command output from the image processing unit 100 and outputs an address, data, and thinning information based on the command.

  The memory cell array 260 is configured by two-dimensionally arranging memory cells that store data. In writing and reading data, a desired memory cell is selected, and data is stored and stored data is read. The memory cells arranged in two dimensions are connected to row and column wirings in an XY matrix, and each memory cell can be selected by selecting these wirings.

  The control unit 220 outputs the address output from the image processing unit interface 210 by separating it into a row address and a column address.

  The column address decoder selects a column wiring of the memory cell array 260 based on the column address.

  The row address decoder selects a row wiring of the memory cell array 260 based on the row address.

  The compression unit 230 compresses data (output data) read from the memory cell array 260 based on the thinning information. The configuration of the compression unit 230 is the same as that of the compression unit 130 described in FIG.

  The image processing unit 100 according to the fifth embodiment of the present technology does not need to include the compression unit 130 described in FIG. Since the other configuration of the image processing apparatus is the same as that of the image processing apparatus described with reference to FIG.

  The exchange of frames between the storage unit 200 and the image processing unit 100 in FIG. When the image processing unit 100 stores a frame in the storage unit 200, a write command is output from the image processing unit 100, and a frame that is write data is output along with this. The storage unit 200 interprets the write command and writes the frame data to the memory cell of the memory cell array 260 corresponding to the write destination address. As a result, the frame is stored.

  When the image processing unit 100 reads a frame from the storage unit 200, the image processing unit 100 outputs a read command. The storage unit 200 interprets the read command, reads a frame from the memory cell of the memory cell array 260 corresponding to the read destination address, and then performs compression by the compression unit 230 to generate a compressed frame. The compressed frame is output to the image processing unit 100. The compressed frame output from the storage unit 200 is decompressed by the decompression unit 140 of the image processing unit 100. As described above, the image processing apparatus according to the fifth embodiment of the present technology performs compression when a frame or the like is read from the storage unit 200.

  Once an OSD image is generated, it is rarely changed. For this reason, it is possible to use a method in which an OSD image is generated and stored in the storage unit 200 when the system is started, and is read out and mixed with a frame as necessary to generate a display frame. This is to reduce the OSD image generation process. Therefore, in such a case, the image processing apparatus according to the fifth embodiment of the present technology is used, an uncompressed OSD image is stored in the storage unit 200, and based on a thinning rate according to a stream to be displayed. Then, the data is compressed in the storage unit 200 and transferred to the image processing unit 100. Thereby, it is possible to select and compress an appropriate thinning rate according to the frame to be displayed.

  FIG. 25 is a diagram illustrating an example of processing of the image processing unit 100 according to the fifth embodiment of the present technology. This figure shows the flow of processing in each part of the image processing unit 100 as in FIG. Demultiplex processing and decoding are performed on the input stream to generate a frame, which is stored in the storage unit 200. This frame is read out, and based on this, the thinning information generation unit 110 generates thinning information. The frame read from the storage unit 200 is processed by the frame processing unit 150 to generate a processed frame and stored in the storage unit 200. Next, the resolution of the processed frame read from the storage unit 200 is converted by the resolution conversion unit 160.

  On the other hand, the OSD image is generated in advance by the OSD image generation unit 170 and stored in the storage unit 200. The stored OSD image is compressed and read by the compression unit 230 of the storage unit 200 based on the thinning information generated by the thinning information generation unit 110 described above, and is expanded by the expansion unit 140. The OSD image display unit 180 converts the resolution of the expanded OSD image.

  Finally, the processed frame and the OSD image whose resolution has been changed are mixed by the mixing unit 190.

  As described above, according to the fifth embodiment of the present technology, a frame stored in the storage unit 200 in an uncompressed state is compressed based on a thinning rate as necessary and output from the storage unit 200. It is possible to prevent the display quality of the frame from being deteriorated.

  As described above, according to the embodiment of the present technology, the compression and decompression processing when the frame is stored in the storage unit 200 can be performed by thinning and interpolation of the image signal, which are simple processing.

  The above-described embodiment shows an example for embodying the present technology, and the matters in the embodiment and the invention-specific matters in the claims have a corresponding relationship. Similarly, the invention specific matter in the claims and the matter in the embodiment of the present technology having the same name as this have a corresponding relationship. However, the present technology is not limited to the embodiment, and can be embodied by making various modifications to the embodiment without departing from the gist thereof.

  Further, the processing procedure described in the above embodiment may be regarded as a method having a series of these procedures, and a program for causing a computer to execute these series of procedures or a recording medium storing the program. You may catch it. As this recording medium, for example, a CD (Compact Disc), an MD (MiniDisc), a DVD (Digital Versatile Disc), a memory card, a Blu-ray disc (Blu-ray (registered trademark) Disc), or the like can be used.

  In addition, the effect described in this specification is an illustration to the last, Comprising: It does not limit and there may exist another effect.

In addition, this technique can also take the following structures.
(1) A storage unit that supplies a compressed frame, which is the frame that has been compressed by thinning to remove a part of the plurality of image signals in a frame configured by a plurality of image signals;
An image processing unit comprising: a decompression unit that decompresses the compressed frame by restoring the thinned image signal in the supplied compressed frame based on the thinning information that is the thinning information for each frame. apparatus.
(2) a thinning information generation unit for generating the thinning information;
The image processing apparatus according to (1), further comprising: a compression unit that performs the compression of the frame by the decimation based on the decimation information and stores the compression in the storage unit.
(3) The thinning information generation unit generates, as the thinning information, a thinning rate that is a ratio of the number of image signals to be thinned with respect to the number of image signals included in the frame.
The image processing device according to (2), wherein the compression unit performs the thinning by determining the number of image signals to be thinned based on the thinning rate.
(4) The compression unit applies a plurality of arrangement patterns, which are information on the arrangement of the image signals to be thinned, on the line obtained by dividing the frame in the horizontal direction to the thinning for each line in a predetermined order. Perform the compression,
The image processing apparatus according to (3), wherein the expansion unit performs the expansion by applying the arrangement pattern to the restoration for each line in the predetermined order.
(5) The image processing device according to (4), wherein the thinning information generation unit generates the thinning rate and thinning position information that is information in the predetermined order as the thinning information.
(6) The image processing device according to (5), wherein the thinning information generation unit generates the thinning position information that is different for each of the plurality of frames that are continuous with the thinning rate as the thinning information.
(7) The image processing device according to (4) or (5), wherein the compression unit performs the compression based on the arrangement pattern in which the image signals to be thinned are arranged not adjacent to each other in the horizontal direction. .
(8) a frame mixing unit that generates a new frame by mixing the second frame, which is a frame different from the frame, and the frame based on a predetermined mixing ratio;
The image processing device according to any one of (3) to (7), wherein the thinning information generation unit generates the thinning rate of the frame and the second frame as the thinning information based on the mixing ratio.
(9) further comprising a thinning information generation unit for generating the thinning information;
The image processing apparatus according to (1), wherein the storage unit stores the frame, performs the compression of the stored frame by the thinning based on the thinning information, and supplies the compressed frame to the expansion unit.
(10) The thinned image signal in the compressed frame, which is the frame that has been compressed by thinning to remove a part of the plurality of image signals in a frame composed of a plurality of image signals, for each frame An image processing method comprising a decompression procedure for decompressing the compressed frame by restoring based on the thinning information that is the thinning information.

DESCRIPTION OF SYMBOLS 100 Image processing part 102 Demultiplexer 103 Processor 104 Storage part interface 110 Decimation information generation part 120 Video decoder 130,154,174,230 Compression part 131 Image signal removal part 132 Image signal rearrangement part 140,161,181 Expansion part 141 Image Signal insertion unit 142 Restoration unit 150 Frame processing unit 152 IPC unit 153 Noise reduction unit 155, 165, 175, 185 Line memory 160 Resolution conversion unit 162 Resolution conversion processing unit 170 OSD image generation unit 171 OSD image generation processing unit 180 OSD image display Unit 182 OSD image resolution conversion unit 190 mixing unit 191 frame mixing unit 192 frame OSD image mixing unit 200 storage unit 210 image processing unit interface 20 control unit 240 a column address decoder 250 row address decoder 260 memory cell array 300 display

Claims (10)

A storage unit that supplies a compressed frame that is the frame that has been compressed by thinning to remove a part of the plurality of image signals in a frame constituted by a plurality of image signals;
An image processing unit comprising: a decompression unit that decompresses the compressed frame by restoring the thinned image signal in the supplied compressed frame based on the thinning information that is the thinning information for each frame. apparatus. A decimation information generating unit for generating the decimation information;
The image processing apparatus according to claim 1, further comprising: a compression unit that performs the compression of the frame by the decimation based on the decimation information and stores the compression in the storage unit. The thinning information generation unit generates, as the thinning information, a thinning rate that is a ratio of the number of image signals to be thinned with respect to the number of image signals included in the frame,
The image processing apparatus according to claim 2, wherein the compression unit performs the thinning by determining the number of image signals to be thinned based on the thinning rate. The compression unit applies the plurality of arrangement patterns, which are information on the arrangement of the image signals to be thinned, on the line obtained by dividing the frame in the horizontal direction to the thinning for each line in a predetermined order. Done
The image processing apparatus according to claim 3, wherein the expansion unit performs the expansion by applying the arrangement pattern to the restoration for each line in the predetermined order.   The image processing apparatus according to claim 4, wherein the thinning information generation unit generates the thinning rate and thinning position information that is information in the predetermined order as the thinning information.   The image processing apparatus according to claim 5, wherein the thinning information generation unit generates the thinning position information that is different for each of the plurality of frames that are continuous with the thinning rate as the thinning information.   The image processing apparatus according to claim 4, wherein the compression unit performs the compression based on the arrangement pattern in which the image signals to be thinned are arranged not adjacent to each other in the horizontal direction. A frame mixing unit that generates a new frame by mixing a second frame that is different from the frame and the frame based on a predetermined mixing ratio;
The image processing apparatus according to claim 3, wherein the thinning information generation unit generates the thinning rate of the frame and the second frame as the thinning information based on the mixing ratio. A thinning information generator for generating the thinning information;
The image processing apparatus according to claim 1, wherein the storage unit stores the frame, performs the compression of the stored frame by the thinning based on the thinning information, and supplies the compressed frame to the expansion unit.   In the frame composed of a plurality of image signals, the thinned image signal in the compressed frame, which is the frame that has been compressed by thinning to remove a part of the plurality of image signals, is subjected to the decimation for each frame. An image processing method comprising a decompression procedure for decompressing the compressed frame by restoring based on thinning information that is information.

Images