JP2007243500A - Image processing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To multiplex much moving images with an original moving image without sharply degrading the image quality of the original moving image. <P>SOLUTION: In an original moving image and a multiplexing moving image, two continuous frames are processed as a pair and the pixel values of the same positions of the frames in the pair are averaged, to process images so that each two frames are constituted of the same image. In the first frame out of the frames, constituting the pair in the case of multiplexing the original moving image, the original moving image and the multiplexing image are added to each other; and in the second frame, the multiplexing image is subtracted from the original moving image. On the receiving side of the moving images, the original moving image is obtained, by executing additional operations between the frames of the pair and the original moving image, and the moving image are reproduced by executing subtraction operation. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image processing apparatus, and more particularly to an image processing apparatus that multiplexes a moving image with a broadcast original moving image and extracts the original moving image and the moving image from the multiplexed moving image.

  Conventionally, the most basic method of multiplexing a moving image or data with respect to the original moving image is to assign the lower bits (about 1 to 4 bits) of the pixel value at each position of the original moving image as an area for multiplexing, and data in that area Is replaced with a moving image or data for multiplexing (this is not related to the known invention in the literature).

  Further, as a pixel replacement type data multiplexing technique, a method is known in which a large amount of data is multiplexed in a noise-like area of an original image and a small amount of data is multiplexed in other portions (see, for example, Non-Patent Document 1). ).

  Furthermore, a technique for realizing high-efficiency compression using the correlation between adjacent pixels is also known (see, for example, Patent Document 1). In this image processing apparatus, a plurality of input image signals are grouped in units of a certain number of frames, and each frame in the group is divided into a plurality of blocks. Then, after smoothing a partial image obtained by collecting pixels at the same position in the block as a sub-sampling image, the sub-sampling image is combined with sub-sampling images of other frames in the same group and arranged in time series.

IPSJ Magazine Vol.44 No.3 Mar. 2003 (pages 237-238) Japanese Patent Laying-Open No. 2003-32666 (page 3, FIG. 1)

  However, in the basic method described above, some bits of each pixel value of the original moving image are used for the moving image or data for multiplexing. Therefore, if the number of bits is increased, the data of the original moving image is changed. There is a problem that the image quality of the original moving image is significantly deteriorated.

  In addition, the technique described in Non-Patent Document 1 has a problem that the ratio of noise-like regions varies depending on the original image, and is not suitable for multiplexing a moving image or data at a certain rate on the original moving image.

  Further, in the technique described in Patent Document 1, the correlation between adjacent pixels is used by arranging the partial images obtained by dividing the frames in combination with the partial images of other frames in time series. There is a problem that, depending on the material of the image, there may be a case where the efficiency is good and a case where it is bad.

  Accordingly, an object of the present invention is to provide an image processing apparatus capable of multiplexing a moving image having a larger amount of data without causing significant image quality degradation of the original moving image when multiplexing the moving image on the original moving image. is there.

  The first image processing apparatus of the present invention generates an image obtained by averaging two consecutive frames of the original moving image as a pair in the image processing device that multiplexes the moving image with respect to the original moving image. An averaging processing unit (10 in FIG. 1) that generates an image obtained by averaging two frames as a pair, adds a pair of moving image frames to the pair of moving image frames, and also adds a pair of moving images from the pair of moving image frames. A moving image multiplexing unit (30 in FIG. 1) that subtracts frames and outputs the addition output and the subtraction output alternately as a multiplexed moving image is provided.

  More specifically, the averaging processing unit (10 in FIG. 1) detects a frame interval of the input original moving image and operates a 1-bit counter to generate a transmission side timing for each frame. A counter (11 in FIG. 1), a first memory (12 in FIG. 1) that delays the original moving image by one frame in synchronization with the transmission side timing, and a moving image that is input in synchronization with the transmission side timing are 1 A second memory (13 in FIG. 1) that delays by a frame, a first adder (14 in FIG. 1) that adds a one-frame delayed image from the first memory to the original image, and every two counts of transmission side timing A first selector (16 in FIG. 1) for selecting the output of the first adder, a second adder (15 in FIG. 1) for adding a one-frame delayed image from the second memory to the moving image, and transmission side timing 2nd count every 2 counts Composed of a second selector for selecting the output of the vessel (17 in Fig. 1).

  In addition, the moving image multiplexing unit (30 in FIG. 1) adds an output of the second selector to the output of the first selector (31 in FIG. 1), and outputs the output of the second selector from the output of the first selector. A subtracter (32 in FIG. 1) for subtracting and a selector (33 in FIG. 1) for selecting the output of the adder or subtractor according to the count value of the frame counter.

  Further, the image processing apparatus includes a bit number adjusting unit that is between the averaging processing unit and the moving image multiplexing unit and adjusts the number of bits of each pixel value of the multiplexed moving image to be the number of bits of the original moving image and the moving image. (20 in FIG. 1).

  More specifically, the bit number adjusting unit (20 in FIG. 1) shifts the output of the first selector in the LSB direction by the number of bits of the pixel value that is increased by addition / subtraction in the averaging processing unit and the moving image multiplexing unit. A first bit shifter (21 in FIG. 1) and a second bit shifter (22 in FIG. 1) that shifts the output of the second selector in the LSB direction by the number of bits of the pixel value increased by addition and subtraction in the averaging processing unit and the moving image multiplexing unit. ).

  According to a second image processing apparatus of the present invention, in the image processing apparatus for reproducing an original moving image and a moving image from the above-described multiple moving images, a means for adding two consecutive frames of the multiple moving images, and a continuous of the multiple moving images It has a moving image separation unit (40 in FIG. 2) comprising means for adding two frames, means for outputting the addition output as an original moving image, and means for outputting the subtraction output as a moving image.

  More specifically, the moving image separation unit (40 in FIG. 2) detects a frame interval of multiple moving images and operates a 1-bit counter to generate a reception side timing for each frame. (41 in FIG. 2), a memory (42 in FIG. 2) that delays the multiple moving image by one frame in synchronization with the reception side timing, and an adder (42) that adds the one frame delayed image from the memory to the multiple moving image 44) in FIG. 2, a subtracter (45 in FIG. 2) for subtracting one frame delayed image from the memory from the multiplex moving image, and a first selector for selecting the output of the adder every two counts of the reception side timing ( 46) of FIG. 2 and a second selector (47 of FIG. 2) for selecting the output of the subtractor every two counts of the reception side timing.

  In the present invention, an interval between frames of a moving image is detected by a frame counter, and timing for processing is generated. By using a frame counter and a frame delay memory, a moving image delayed by one frame can be generated. By passing the moving image and the moving image delayed by one frame through an adder or subtracter, Averaging, extraction of original moving images or multiple moving images can be performed. Further, by passing the original moving image and the multiplexed moving image through an adder or subtracter, a moving image in which both are multiplexed can be generated.

  According to the present invention, when a moving image is multiplexed onto the original moving image, the two consecutive frames are averaged as a pair for each of the original moving image and the moving image, and addition and subtraction are performed on the averaged original moving image and the moving image. Therefore, the original moving image and the moving image can maintain a data amount of 1 bit reduction after being multiplexed, and a moving image having a larger data amount can be maintained without causing a significant deterioration in image quality of the original moving image. The effect that it can multiplex can be acquired.

[Description of configuration]
Next, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing the transmission side of the image processing apparatus of the present invention. This transmission side apparatus multiplexes an original moving image and a moving image and outputs a multiplexed moving image. As shown in FIG. 1, an averaging processing unit 10, a bit number adjusting unit 20, and a moving image multiplexing unit 30 are used. It is configured.

  The averaging processing unit 10 generates an image obtained by averaging two consecutive frames as a pair for each of the original moving image and the moving image. The bit number adjusting unit 20 increases by averaging in the averaging processing unit 10, and further increases by multiplexing in the moving image multiplexing unit 30, so that the number of bits of each pixel value of the multiplexed moving image is increased. Adjust to the number of bits of the original moving image and moving image. The moving image multiplexing unit 30 generates a multiplexed moving image by adding and subtracting the processed original moving image output from the bit number adjusting unit 20 and the moving image.

  The averaging processing unit 10 includes a frame counter 11, two memories 12 and 13, two adders 14 and 15, and two selectors 16 and 17. The frame counter 11 detects the frame interval of the input original moving image and operates the 1-bit counter to generate the transmission side timing for each frame.

  The memory 12 delays the original moving image by one frame in synchronization with the transmission side timing, and the memory 13 delays the input moving image by one frame in synchronization with the transmission side timing. The adder 14 adds the one-frame delayed image from the memory 12 to the original moving image, and the adder 15 adds the one-frame delayed image from the memory 13 to the moving image. The selector 16 selects the output of the adder 14 every two counts of the transmission side timing, and the selector 17 selects the output of the adder 15 every two counts of the transmission side timing.

  The bit number adjusting unit 20 includes two 2-bit shifters 21 and 22. The image after the addition in the adders 14 and 15 of the averaging processing unit 10 has a pixel value increased by 1 bit, and the final moving image multiplexing unit 30 also passes through the adder 31 so that the final multiplexed video It is necessary to match the number of bits of the pixel value of the image with the number of bits of the original moving image and the moving image. Therefore, the 2-bit shifter 21 shifts the output of the selector 16 by 2 bits in the LSB direction, and the 2-bit shifter 22 shifts the output of the selector 17 by 2 bits in the LSB direction.

  The moving image multiplexing unit 30 includes an adder 31, a subtractor 32, a ± sign assigning unit 33, and a selector 34. The adder 31 adds the image after 1/4 operation by the 2-bit shifter 22 to the image after 1/4 operation by the 2-bit shifter 21 in the bit number adjusting unit 20, and the subtracter 32 is 1 by the 2-bit shifter 21 in the bit number adjusting unit 20. The image after 1/4 calculation by the 2-bit shifter 22 is subtracted from the image after / 4 calculation. Since the calculation result may be negative in the subtracter 32, the ± sign assignment unit 33 assigns a ± sign. The selector 34 selects the addition data or the subtraction data according to the count value of the frame counter 11, thereby accommodating the addition data in the first frame and the subtraction data in the second frame of the multiplexed moving image paired with two frames. . Then, by selecting the output of the adder 31 or the output of the ± sign imparting unit 33, the addition data is accommodated in the first frame of the multiple moving image paired with two frames, and the subtraction data is accommodated in the second frame. And This multiple moving image is a transmission target.

  FIG. 2 is a block diagram showing the receiving side of the image processing apparatus of the present invention. The receiving side apparatus reproduces the original moving picture and the moving picture after receiving the multiplexed moving picture transmitted from the transmitting side apparatus described above. For this purpose, a moving image separation unit 40 is provided as shown in FIG. The moving image separation unit 40 separates and extracts the multiple moving images into the original moving image and the moving image by performing addition and subtraction of the multiple moving images.

  The moving image separation unit 40 includes a frame counter 41, a memory 42, a ± code extraction unit 43, an adder 44, a subtractor 45, and two selectors 46 and 47. The frame counter 41 detects a frame interval of multiple moving images and operates a 1-bit counter to generate a reception side timing for each frame. Note that the sequence number in the blanking of each frame is also referred to at the same time, and the count value of the frame counter 41 is set to “0” when the sequence number is an even number. Thus, when the frame count value is “0”, the first frame of the 2-frame pair image is indicated, and when the frame count is “1”, the second frame of the 2-frame pair image is indicated.

  The memory 42 delays the multiplexed moving image by one frame in synchronization with the reception side timing. The ± sign extraction unit 43 extracts the ± sign from a predetermined bit of the multiplexed moving image, and clears the data of the bit to “0”. When the ± sign indicates a negative number, the subsequent adder 44 functions as a subtracter and the subtractor 45 functions as an adder.

  The adder 44 adds the one-frame delayed image from the memory 42 to the multiplexed moving image when the ± sign extracted by the code extraction unit 43 is a positive number, and adds the one frame delay image from the memory 42 when the ± sign is a negative number. The multiple moving images are subtracted from the one frame delayed image. A moving image is obtained from the output of the adder 44.

  The subtracter 45 subtracts the one-frame delayed image from the memory 42 from the multiplex moving image when the ± sign extracted by the ± sign extraction unit 43 is a positive number, and multiplexes when the ± sign is a negative number. The one-frame delayed image from the memory 42 is added to the moving image. A moving image is obtained from the output of the subtracter 45.

  The selector 46 selects the output of the adder 44 every two counts of the reception side timing, and the selector 47 selects the output of the subtractor 45 every two counts of the reception side timing. In this way, an original moving image and a multiple moving image in which two frames are paired can be obtained.

[Description of operation]
Next, the operation of the image processing apparatus of the present invention will be described with reference to the transmission side processing image diagram shown in FIG. 3 and the reception side processing image diagram shown in FIG.

  In FIG. 3, the source images with the pixel number of 8 bits at the timings t 0, t 1, t 2, t 3, t 4... Are sequentially frame by frame, such as frames a, b, c, d, e. Input to the transmitting side device. The frame counter 11 of the averaging processing unit 10 detects the interval between the frames a and b of the original moving image, and generates a transmission side timing that alternately becomes “0” and “1” from the timing t1. The memory 12 delays the frame of the original moving image by one frame in synchronization with the transmission side timing. That is, the one-frame delayed image becomes a, b, c, d, e.

  The adder 14 adds a one-frame delayed image to the original moving image, and the added image is (a + b), (b + c), (c + d), (d + e). Become. As described above, since the number of bits of the pixel value of the original moving image is 8, the number of pixels of the added image is 9 bits. The selector 16 selects the output of the adder 14 when the transmission side timing is “0”, so that the post-selector image becomes (a + b) at timings t1 and t2, and (c + d) at timings t3 and t4. Is output. The 2-bit shifter 21 of the bit number adjusting unit 20 converts the post-selector images (a + b) and (c + d) into 2 bits in the LSB direction, and generates a 7-bit 1 / 4-computed image (a + b) of the original moving image. / 4 = A, (c + d) / 4 = B.

  The moving image to be multiplexed with the original moving image also has 8 pixel values, and is sequentially input to the transmission side device frame by frame at the timings t0, t1, t2, t3, t4. The same processing as that for the original moving image is performed. That is, the memory 13 operates in the same manner as the memory 12, the adder 15 operates in the same manner as the adder 14, the selector 17 operates in the selector 16, the 2-bit shifter 22 operates in the same manner as the 2-bit shifter 21, and the frame counter 11 operates in the original moving image processing system and the moving image processing. Works in common with the system. In FIG. 3, W1 and W2 are images after addition of moving images corresponding to A and B, which are images after 1/4 calculation of the original moving image.

  The adder 31 of the moving image multiplexing unit 30 adds the 1/4 post-computation image W1 to the 1/4 post-computation image A, and after the multiple data addition image (A + W1) at timing t1, t2, the timing t3, An image (B + W2) after addition of multiple data is obtained at t4. Also, the subtractor 32 of the moving image multiplexing unit 30 subtracts the 1/4 post-computation image W1 from the 1/4 post-computation image B, and after the multiple data subtraction image (A-W1) and timing An image (B-W2) after multiple data subtraction is obtained at t3 and t4. Since the eighth bit of the pixel value is always an unused area, if the calculation result is negative in the subtractor 32, the ± sign assignment unit 33 is “0” if the area of the eighth bit is a positive number. If it is a negative number, sign information of “1” is given.

  The selector 34 selects the multiplexed data added image (A + W1) = X1 at the timing t1 when the transmission side timing is “0”, and the multiplexed data added image (B + W2) = Y1 at the timing t3. Further, the image after multiple data subtraction (A-W1) = X2 is selected at timing t2 when the transmission side timing is “1”, and the image after multiple data subtraction (B-W2) = Y2 is selected at timing t4. Then, the post-selector image is transmitted as a multiplexed moving image.

  In FIG. 4, multiple moving images are sequentially input to the receiving apparatus frame by frame at timings t0, t1, t2, t3, t4, and so on, such as frames X1, X2, Y1, Y2, Z1,. Suppose you come. Z1 is an image after addition of multiple data to frames e and f following frame d of the original moving image.

  The frame counter 41 of the moving image separation unit 40 detects the interval between the frame X1 of the received multiplexed video of the multiplexed data and the image X2 of the multiplexed data after subtraction, and alternately “0” and “1” from the timing t1. The receiving side timing is generated. The memory 42 delays the frame of the received multiple moving image by one frame in synchronization with the reception side timing. That is, the one-frame delayed image is X1, X2, Y1, Y2, Z1,.

  The ± code extraction unit 43 extracts the ± code assigned to the 8th bit of the multiplexed moving image and clears the 8th bit data to “0”. The adder 44 adds the one-frame delayed images X1, X2, Y1, Y2... To the original moving images X2, Y1, Y2, Z1..., And the added image becomes (X1 + X2), ( X2 + Y1), (Y1 + Y2), (Y2 + Z1). Further, the subtracter 45 subtracts the one-frame delayed images X1, X2, Y1, Y2... From the source images X2, Y1, Y2, Z1..., And the subtracted image is (X1-X2) from timing t1. , (X2-Y1), (Y1-Y2), (Y2-Z1).

  The selector 46 adds the added image (X1 + X2) = 2A = (a + b) / 2 at the timing t1 when the receiving side timing is “0”, and the added image (Y1 + Y2) = 2B = (at the timing t3. c + d) / 2 is selected and output as a moving image. This original moving image is obtained by averaging two consecutive frames a and b, c and d, etc. of the original moving image input to the transmission side device as a pair. Further, the subtracted image (X1-X2) = 2W1 is selected at timing t2 at which the receiving side timing is “1”, and the subtracted image (Y1-Y2) = 2W2 is selected at timing t4 and output as a moving image. This moving image is obtained by averaging two consecutive frames of moving images input to the transmission side device as a pair.

  The number of bits of the pixel values of the original moving image and the moving image multiplexed in the received multiplexed moving image is 7-bit data, but passes through the adder 44 or the subtracter 45 in the moving image separation unit 40. Thus, the finally obtained original moving image and moving image are 8-bit data.

[Other Embodiments of the Invention]
As another embodiment of the present invention, the basic configuration is as described above. However, as described above, the number of bits of the pixel value of the moving image does not need to be limited to 8 bits. Multiplexing in a similar manner is possible.

  Further, it is not necessary to limit the number of moving images to be multiplexed. For example, when three moving images are multiplexed and combined with the original moving image, the number of moving images is four, and four frames are treated as one group. Multiplexing is possible by embedding four kinds of calculation results in each frame in the group. However, in this case, the amount of data that can be multiplexed decreases.

1 is a block diagram showing a transmission side of an image processing apparatus of the present invention. 1 is a block diagram showing a receiving side of an image processing apparatus according to the present invention. Transmission side processing image diagram of the image processing apparatus of the present invention Receiving side processing image diagram of image processing apparatus of the present invention

Explanation of symbols

10 Averaging processor
11, 41 frame counter
12, 13, 42 memory
14, 15, 31, 44 adder
16, 17, 34, 46, 47 selector
20-bit number adjustment section
21,22 2-bit shifter
30 Video multiplexing unit
32, 45 subtractor
33 ± Sign assignment section
40 Video separation part
43 ± Sign extraction unit

Claims (7)

In an image processing apparatus that multiplexes a moving image with respect to an original moving image,
An average processing unit for generating an image obtained by averaging two consecutive frames of the original moving image as a pair, and generating an image obtained by averaging two consecutive frames of the moving image as a pair;
The moving image frame of the pair is added to the moving image frame of the pair, the moving image frame of the pair is subtracted from the moving image frame of the pair, and the addition output and the subtraction output are alternately output as a multiplexed moving image. An image processing apparatus comprising a moving image multiplexing unit. The averaging processing unit
A frame counter that detects a frame interval of the input moving image and generates a transmission side timing for each frame by operating a 1-bit counter;
A first memory that delays the original moving image by one frame in synchronization with the transmission side timing;
A second memory for delaying the moving image input in synchronization with the transmission side timing by one frame;
A first adder for adding a one-frame delayed image from the first memory to the original moving image;
A first selector that selects an output of the first adder every two counts of the transmission side timing;
A second adder for adding a one-frame delayed image from the second memory to the moving image;
The image processing apparatus according to claim 1, further comprising a second selector that selects an output of the second adder every two counts of the transmission side timing. The moving image multiplexing unit includes:
An adder for adding the output of the second selector to the output of the first selector;
A subtractor for subtracting the output of the second selector from the output of the first selector;
The image processing apparatus according to claim 2, further comprising a selector that selects an output of the adder or the subtracter according to a count value of the frame counter.   A bit number adjusting unit that is between the averaging processing unit and the moving image multiplexing unit and adjusts the number of bits of each pixel value of the multiplexed moving image to be the number of bits of the original moving image and the moving image; The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. The bit number adjustment unit
A first bit shifter that shifts the output of the first selector in the LSB direction by the number of bits of a pixel value that is increased by addition and subtraction in the averaging processing unit and the moving image multiplexing unit;
5. The image according to claim 4, further comprising a second bit shifter that shifts the output of the second selector in the LSB direction by the number of bits of a pixel value that is increased by addition and subtraction in the averaging processing unit and the moving image multiplexing unit. Processing equipment. The image processing apparatus for reproducing the original moving image and the moving image from the multiple moving images according to claim 1,
Means for adding two consecutive frames of the multiple moving images;
Means for adding two consecutive frames of the multiple moving images;
Means for outputting the added output as a moving image;
An image processing apparatus comprising: a moving image separation unit configured to output the subtracted output as a moving image. The moving image separating unit
A frame counter that detects a frame interval of the multiple moving images and generates a reception side timing for each frame by operating a 1-bit counter; and
A memory that delays the multiple moving images by one frame in synchronization with the reception side timing;
An adder for adding a one-frame delayed image from the memory to the multiple moving image;
A subtractor for subtracting a one-frame delayed image from the memory from the multiple moving images;
A first selector that selects the output of the adder every two counts of the receiving side timing;
The image processing apparatus according to claim 6, further comprising a second selector that selects an output of the subtracter every two counts of the reception side timing.

Images