JP2007068209A - Moving image generating apparatus and moving image generating method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To perform photographing in darkness without losing smoothness of a moving image. <P>SOLUTION: A plurality of image signals (BI<SB>1</SB>-BI<SB>3</SB>) periodically outputted by an image output means are acquired and the plurality of acquired image signals are combined to produce one image signal (BO<SB>j</SB>). A value of luminance information or color difference information of a synthetic image signal (BO<SB>j</SB>) can be increased and photographing in darkness can be performed without changing a frame term. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a moving image generation apparatus and a moving image generation method, and more particularly to a moving image generation apparatus and a moving image generation method applicable to a television camera, a video camera, a moving image camera, and the like.

Appropriate exposure for a TV camera, video camera, video camera, etc. can be obtained by assuming that the brightness of the subject is B, the sensitivity of the photosensitive material is S, the brightness of the lens is 1 / F ² , and the exposure time is T.
BS · T / F ² = K or F ² / T = BS / K
For example, when B is small (hereinafter referred to as “when shooting in a dark place”), the countermeasure is as follows:
(1) Use a photosensitive material having a high sensitivity S. (2) Open the aperture to increase 1 / F ^2. (3) Increase the exposure time T.

Here, countermeasure (2) is the F number of the photographic lens (the reciprocal number f / D of the ratio D / f of the effective lens diameter D to the focal length f. This is called the F number in distinction from the focal length f.) In general, it is impossible to take a dark place with this measure alone, and it is often used in combination with other measures.
On the other hand, the measure (1) is a measure applied to a camera using a silver salt film. In a camera using a solid-state image pickup device or an image pickup tube, the sensitivity S is determined to be a specific value. It cannot be a countermeasure for TV cameras, video cameras, and video cameras equipped with devices.
It should be noted that the sensitivity S can be improved by amplifying the output signal of the imaging device with a high gain, but such high gain amplification increases the noise as well, so it is not an ideal measure.

  Therefore, when shooting in a dark place using a TV camera, video camera, video camera, etc. (hereinafter referred to as “imaging system”) equipped with a solid-state imaging device or imaging tube, that is, proper exposure is obtained even when the aperture is fully opened. If this is not possible, the only solution is to increase the exposure time T of the solid-state imaging device or imaging tube from the above measure (3).

  However, in the imaging system described above, light from an object is converted into an image signal using a solid-state imaging device such as a CCD (Charge Coupled Device) or MOS (Metal Oxide Semiconductor) or an imaging tube such as a Sachicon or a Harpicon, and a frame. For example, in the case of a solid-state imaging device or an imaging tube that outputs an image signal of the NTSC (National Television System Committee) standard, the output cycle (frame cycle) is Since the time is 1/30 second, the exposure time T cannot be set beyond this frame period. Therefore, there is a problem in that dark place photography cannot be performed even by the measure (3).

  If the frame period is lengthened, such a problem can be solved, but on the other hand, the number of frames of the image signal is reduced and a new problem that a smooth moving image cannot be obtained is brought about.

  From the above, the problem to be solved by the present invention is to enable dark place photography without losing the smoothness of a moving image.

According to a first aspect of the present invention, there is provided an image output means for periodically converting light from a subject into an electrical image signal, and a plurality of images from the image signal periodically output by the image output means. Image acquisition means for sequentially acquiring signals; and moving image generation means for periodically executing a process of generating a single image signal by synthesizing a plurality of image signals acquired by the image acquisition means. It is characterized by having.
According to a second aspect of the present invention, in the first aspect of the present invention, the moving image generation means generates one image signal by adding and synthesizing the plurality of image signals.
According to a third aspect of the present invention, in the second aspect of the present invention, the moving image generating means performs addition after weighting each of the plurality of image signals.
According to a fourth aspect of the present invention, in the first aspect of the invention, the moving image generating means performs a motion compensation prediction calculation between the plurality of image signals, and obtains a predicted image signal obtained by the calculation. One image signal is generated by performing addition processing on one of a plurality of image signals.
The invention according to claim 5 sequentially holds the image output means for periodically converting the light from the subject into an electrical image signal and outputting it periodically, and the image signal periodically outputted from the image output means. A moving image is obtained by executing a process of correcting the target pixel information in the image signal using adjacent pixel information in the vicinity of the target pixel for the image signal sequentially stored in the holding unit and the holding unit. And moving image generation means for generating.
According to a sixth aspect of the present invention, in the fifth aspect of the invention, the moving image generation unit weights one or both of the target pixel information and the adjacent pixel information and then adds the weights to the target pixel. It is characterized by correcting information.
A seventh aspect of the invention is characterized in that, in the sixth aspect of the invention, the weighting value is set corresponding to a positional relationship between the target pixel and the adjacent pixel.
According to an eighth aspect of the invention, in the fifth or sixth aspect of the invention, the moving image generating means examines the correlation of each of the neighboring pixels with respect to the target pixel and obtains information on neighboring pixels having high correlation. , And used as correction information for the target pixel information.
The invention according to claim 9 is an image acquisition step for sequentially acquiring a plurality of image signals from the image signals periodically output by the image output means, and the plurality of image signals acquired in the image acquisition step are synthesized. And a moving image generation step of periodically executing a process of generating one image signal by processing.
According to a tenth aspect of the present invention, a holding step for sequentially holding the image signals periodically output by the image output means, and the target pixel information in the image signal are adjacent to the image signals held in the holding step. And a correction step of generating a moving image by periodically executing a correction process using pixel information.

According to the first aspect of the present invention, the image output means for periodically converting the light from the subject into an electrical image signal and outputting it periodically, and a plurality of images from the image signal periodically output by the image output means Image acquisition means for sequentially acquiring the image signals, and moving image generation means for periodically executing a process of generating a single image signal by synthesizing a plurality of image signals acquired by the image acquisition means, Since the brightness information and the color difference information of the composite image signal can be increased, dark place photography can be performed without changing the frame period.
According to a second aspect of the present invention, in the first aspect of the invention, the moving image generating means adds and synthesizes the plurality of image signals to generate one image signal. Brightness information and color difference information values can be increased according to the number of additions, and dark place photography can be performed without changing the frame period.
According to a third aspect of the present invention, in the second aspect of the invention, the moving image generating means performs addition after weighting each of the plurality of image signals. A composite image signal can be obtained.
According to a fourth aspect of the present invention, in the first aspect of the present invention, the moving image generating means performs a motion compensation prediction calculation between the plurality of image signals, and obtains a predicted image signal obtained by the calculation. Since one image signal is generated by performing addition processing on one of the plurality of image signals, it can be suitable for photographing a moving subject in a dark place.
According to the fifth aspect of the present invention, the image output means for periodically converting the light from the subject into an electrical image signal and periodically outputting the image signal and the image signal periodically output from the image output means are sequentially provided. A moving image by executing a process for correcting the pixel information of interest in the image signal using the neighboring pixel information in the vicinity of the pixel of interest for the image signal sequentially held by the holding unit And a moving image generating means for generating an image. Therefore, correction processing can be performed on a pixel-by-pixel basis, and a composite image signal with good image quality can be obtained.
According to a sixth aspect of the present invention, in the fifth aspect of the present invention, the moving image generating means weights one or both of the target pixel information and the adjacent pixel information and then adds the two together. Since the pixel-of-interest information is corrected, a composite image signal with good image quality can be obtained by giving an appropriate weight value.
According to a seventh aspect of the invention, in the sixth aspect of the invention, the weighting value is set corresponding to the positional relationship between the target pixel and the adjacent pixel, so that the position is close and closely related. Thus, it is possible to perform correction using the adjacent pixels, and it is possible to obtain a composite image signal with good image quality.
According to an eighth aspect of the present invention, in the fifth or sixth aspect of the present invention, the moving image generation means examines the correlation of each of the adjacent pixels with respect to the pixel of interest, and determines an adjacent pixel having a high correlation. Since information is used as correction information for the pixel-of-interest information, correction can be performed using adjacent pixels with high correlation, and a composite image signal with good image quality can be obtained.
According to the invention of claim 9, an image acquisition step of sequentially acquiring a plurality of image signals from the image signals periodically output by the image output means, and the plurality of image signals acquired in the image acquisition step And a moving image generation step of periodically executing a process of generating one image signal by synthesizing the images, so that dark place shooting can be performed without changing the frame period.
According to the tenth aspect of the present invention, the holding step for sequentially holding the image signals periodically output by the image output means, and the target pixel information in the image signal with respect to the image signals held in the holding step A correction step of generating a moving image by periodically executing a process of correcting the image using adjacent pixel information, so that the correction process can be performed on a pixel-by-pixel basis, and a composite image signal with good image quality can be obtained. Obtainable.

Hereinafter, embodiments of the present invention will be described with reference to the drawings, taking as an example an electronic still camera capable of capturing moving images.
In FIG. 1, 10 is a photographic lens, 11 is an aperture mechanism provided on the optical axis of the photographic lens 10, 12 is a drive unit of the aperture mechanism 11, and 13 is an imaging signal of a subject that receives light passing through the aperture mechanism 11. 14 is a driver of the CCD 13, 14 is a timing signal such as a signal for controlling an exposure time of the CCD 13 (electronic shutter time: hereinafter, sometimes simply referred to as shutter time). A timing generator 16 generates a sample and hold circuit that samples the image signal from the CCD 13 and removes noise, 17 an analog-digital converter that converts the image signal after noise removal into a digital signal, and 18 an analog-digital converter. A color process circuit that generates a luminance / color difference composite signal (hereinafter referred to as a YUV signal) from 17 outputs There, CCD 13, sample hold circuit 16, analog-to-digital converter 17 and color process circuit 18 corresponds to the image output means of the subject matter of the invention.

Reference numeral 19 is a video transfer circuit, 20 is a buffer memory for holding sequentially generated YUV signals, and 21 is a compression / decompression process that compresses / decompresses with a predetermined encoding method (generally JPEG method) during recording and reproduction of YUV signals. The decompression circuit, 22 is a photometric sensor that measures the brightness of the subject, 23 is a fixed or removable flash memory that records the compressed YUV signal, and the buffer memory 20 includes the image acquisition means described in the gist of the invention and It corresponds to the holding means.
Reference numeral 24 denotes a CPU that loads a control program stored in the program ROM 24a into the work RAM 24b and executes it, and performs image recording control and reproduction control, and various control incidental to these controls. This corresponds to the described moving image generating means.
A key input unit 25 generates a key input signal in response to an operation of a shutter button or various buttons. A digital video encoder 26 converts a YUV signal held in the buffer memory 20 into a signal format suitable for display. , 27 is a liquid crystal display for an image monitor for displaying a signal from the digital video encoder 26, and 28 is a bus for connecting each part.

  The electronic still camera having such a configuration can be switched between an image recording mode and a reproduction mode by a predetermined key operation of the key input unit 25, and the recording mode is changed periodically from the CCD 13 (for convenience, the NTSC standard). A through mode in which an image pickup signal taken out in a frame period [1/30 second] is converted into a signal suitable for display and sequentially displayed on the liquid crystal display 27, and a desired image is operated by operating a shutter key. A capture mode in which a signal (still image) or a plurality of image signals (moving images) captured in a predetermined period is recorded in the flash memory 23, and a desired still image or moving image is read from the flash memory 23 and displayed on the liquid crystal display 27. Divided into playback mode.

  In the through mode, the CCD 13 disposed behind the photographic lens 10 is driven by a signal from the driver 14, and the subject images collected by the photographic lens 10 are photoelectrically converted at fixed intervals to output an imaging signal for one image. Is done. The imaging signal is sampled by the sample and hold circuit 16 and converted into a digital signal by the analog / digital converter 17, and then a YUV signal is generated by the color process circuit 18. This YUV signal is transferred to the image buffer of the buffer memory 20 via the video transfer circuit 19, and after the transfer to the buffer is completed, it is read out by the video transfer circuit 19 and sent to the liquid crystal display 27 via the digital video encoder 26. Sent and displayed as a through image.

If the orientation of the camera is changed in this state, the composition of the through image displayed on the liquid crystal display 27 changes, and exposure is performed by “half-pressing” the shutter key at an appropriate time (when the desired composition is obtained). After the focus is set and “full-pressed”, the mode switches to the capture mode. In the still image recording mode, the YUV signal stored in the image buffer of the buffer memory 20 is fixed at the YUV signal at that time, In addition, the through image displayed on the liquid crystal display 27 is also fixed at the same point image. The YUV signal stored in the image buffer of the buffer memory 20 at that time is sent to the compression / decompression circuit 21 via the video transfer circuit 19, and 8 × 8 pixels for each component of luminance information and color difference information. After being JPEG-encoded in units called basic blocks, they are recorded in the flash memory 23.
However, in the moving image recording mode, all YUV signals generated at a frame period of 1/30 seconds during a predetermined period after the shutter key is fully pressed are stored in the image buffer of the buffer memory 20 and Is read from the buffer memory 20 immediately after elapse of time, JPEG-encoded by the compression / decompression circuit 21 (MPEG encoding which is a moving image encoding standard may be used), and then recorded in the flash memory 23 .

  On the other hand, in the playback mode, the path from the CCD 13 to the buffer memory 20 is stopped and the latest captured image (still image or moving image) is read from the flash memory 23 and decompressed by the compression / decompression circuit 21. Then, it is sent to the image buffer of the buffer memory 20 via the video transfer circuit 19. The data in the image buffer is sent to the liquid crystal display 27 via the video transfer circuit 19 and the digital video encoder 26, and displayed as a reproduced image.

FIG. 2 is a structural diagram of the buffer memory 20, and schematically shows two types of image buffers (an input image buffer 30 and an output image buffer 31 for convenience) used particularly in a moving image shooting mode.
In FIG. 2, the input image buffer 30 is composed of a plurality (for convenience in the figure, three) buffer areas BI _{1 to} BI ₃ , and the output image buffer 31 is composed of m buffer areas BO _{1 to} BO _m. . The sizes of the buffer areas BI _{1 to} BI ₃ of the input image buffer 30 are large enough to store one image of YUV signals output from the color process circuit 18, and each buffer area BO of the output image buffer 31. The size of _{1 to} BO _m is about the same.

FIG. 3 is a flowchart showing a moving image shooting mode program using the input image buffer 30 and the output image buffer 31 of FIG. This program is stored in advance in the program ROM 24a of the CPU 24, and is loaded into the work RAM 24b of the CPU 24 and executed by a predetermined key operation of the key input unit 25.
When the program is executed, first, an initial value 1 is set to the loop variable j (S1), and whether j> m is satisfied, that is, whether the value of the loop variable j exceeds the number of areas m of the output image buffer 31. It is determined whether or not (S2). Here, the number m of areas of the output image buffer 31 is an element that determines the recording time of the moving image. For example, if the frame period is 1/30 seconds and m = 30, the moving image recording time is 1 second.

Next, the image signal is stored in the buffer area BI ₁ of the input image buffer 30 (S3). This image signal is a YUV signal output from the color process circuit 18.
Next, a synthesis process is executed (S4). As shown in FIG. 4, the combining process adds and saves information stored in the buffer areas BI _{1 to} BI ₃ of the input image buffer 30 and stores the combined result in the jth buffer area BO _j of the output image buffer 31. (S4_1).

When the composition processing is completed, the contents of the buffer areas BI _{1 to} BI ₃ of the input image buffer 30 are then transferred (S5). The replacement was transferred, first, transferred the contents of BI ₂ to BI _3, then an order of transferring the contents of the BI ₁ to BI _2. That is, the stored information is transferred in the direction of BI ₁ → BI ₂ → BI ₃ .
Finally, the loop variable j is incremented by 1 (S6), and the above processing is repeated until j> m, and the program ends when j> m is satisfied.

FIG. 5 is a conceptual diagram of the above-described addition synthesis. Reference numeral 32 schematically represents the addition process (S4_1) of FIG. This figure shows the situation when j = 5 for convenience. In FIG. 5, the first buffer area BI ₁ of the input image buffer 30 stores the _fifth YUV signal PI ₅ output from the color process circuit 18 during the moving image recording mode period. The buffer area BI ₂ stores the _fourth YUV signal PI ₄ output from the color process circuit 18 during the synchronization, and the third buffer area BI ₃ stores the color process circuit during the synchronization. The third YUV signal PI ₃ output from 18 is stored.

In addition, the image signal PO ₅ obtained by adding the contents of BI ₁ , BI _2, and BI ₃ is stored in the jth buffer area BO ₅ of the output image buffer 31 (j = 5 because it is 5). PO ₄ is the added image signal stored in the _fourth buffer area BO ₄ , PO ₃ is the added image signal stored in the _third buffer area BO ₃ , and PO ₂ is stored in the _second buffer area BO ₂ . The added image signal PO ₁ is an added image signal stored in the _first buffer area BO ₁ . Note that PO ₂ = PI ₁ + PI ₂ and PO ₁ = PI ₁ . When the added image signal is stored in all of the _m buffer areas BO _{1 to} BO m, the determination result of j> m becomes true (YES).

Now, for example, focusing on the PO _5, the PO ₅ are as described above, the contents stored in the BI ₁ ~BI _3, i.e., is an image signal obtained by adding the three image signals (PI _5, PI ₄ and PI ₃₎ . These image signals are two-dimensional images of the subject, and luminance information (Y) and color difference information (UV) of each part of the subject are arranged on a two-dimensional plane. Accordingly, among the image signals output from the color process circuit 18 during a series of moving image recording modes, those that are close in time are highly correlated, and the luminance information (Y) and color difference information (UV) in the two-dimensional plane are high. There are similarities in values.
For this reason, the image signal PO _j obtained by adding three temporally continuous image signals (PI _j−2 , PI _j−1 and PI _j ) is the luminance information (Y) and color difference information ( If the movement of the subject is zero, the luminance information (Y) and the color difference information (UV) are almost tripled by simple calculation. As a result, an effect equivalent to substantially increasing the sensitivity S of the photosensitive material can be obtained, and a special effect can be obtained in which dark place photography can be performed without changing the frame period of the CCD 13. .

  Incidentally, the above-described addition processing also adds random noise in the image signal at the same time. For example, when two image signals are added, the periodic signal component is doubled, Since it is known that random noise with no periodicity is only √2 times, there is no worry of deteriorating S / N.

In the addition process of FIG. 4, three image signals (PI _j-2 , PI _j-1 and PI _j ) are simply added, but the three image signals (PI _j-2 , PI _j-1 ) are added. And PI _j ) may be added after giving an appropriate weight value to each of them. That is, as shown in FIG. 6, after giving appropriate weight values A, B, and C to the contents of the buffer areas BI _{1 to} BI ₃ of the input image buffer 30, they are added and the output image buffer 31 The data may be stored in the _jth buffer area BO _j (S4_2). FIG. 7 is a conceptual diagram thereof, and reference numerals 33 to 35 in the figure schematically show weighting elements.
The weight value is given by replacing C having the maximum value in the j-th image signal PI _j among the three image signals (PI _j-2 , PI _j-1 and PI _j ) with the j-1th image signal. It is desirable to give B having the next value in PI _j-1 and A having the smallest value in the _j- 2th image signal PI _j-2 . This is because weighting according to the temporal distance from the j-th image signal can be performed, and a composite image signal in consideration of image motion can be obtained.

Alternatively, the combining process of FIG. 4 may be modified as shown in FIG. That is, the motion compensation prediction calculation between BI ₁ and BI ₂ is performed, the calculation result is stored in the variable Ma (S4_3), the motion compensation prediction calculation between BI ₂ and BI ₃ is performed, and the calculation result is calculated. After storing in the variable Mb (S4_4), BI ₁ and Ma and Mb may be added, and the addition result may be stored in BO _j (S4_5). FIG. 9 is a conceptual diagram thereof, and reference numeral 36 in the figure schematically shows a motion compensation prediction calculation element. The addition target of Ma and Mb is not limited to BI ₁ . Any of BI _{1 to} BI ₃ may be used.

  Motion compensation prediction calculation (also referred to as motion compensation inter-frame prediction calculation) is, for example, searching for which part of another frame that is temporally adjacent to the subject of a certain frame is searched, and its search direction and search The distance is calculated as motion information between adjacent frames, and is a technique for removing redundancy in the time domain of moving images. Therefore, according to this, a moving subject can be photographed in a dark place. Further, since this technique is a skeleton technique of MPEG coding, for example, when MPEG coding is performed by the compression / decompression circuit 21, the compression algorithm can be used.

As described above, according to the present embodiment, the image signal PI _j , the image signal PI _j−1 , and the image signal PI that are sequentially output from the color process circuit 18 at a predetermined frame period (for example, 1/30 second period). stores the _j-2 in three buffer area BI ₁ ~BI _3, by adding the process the contents of the three buffer area BI ₁ ~BI ₃ generates one image signal PO _j, the output image buffer this Since it is stored in the buffer area BO _{j of} 31 and used as a frame image of the moving image, the luminance value and the color difference value of each part of the moving image frame image are added to the addition number of the original image (PI _j , PI _j-1 and PI _j-2 ). Can be doubled accordingly.

  Therefore, for example, if the luminance value of the original image is n, the luminance value of each part of the frame image of the moving image can be simply calculated as “addition number × n”. Generally, the luminance value n becomes a white level as the value increases. Since they are close to each other, it is possible to obtain the same effects (effects of the countermeasures (1) to (3)) as when the exposure is increased. As a result, it is possible to perform dark place photography without changing the frame period of the CCD 13, and it is possible to obtain a particularly beneficial effect for a television camera, a video camera, or a moving image camera.

Further, when a weight value corresponding to the temporal distance of the original image is given to each of the original images (PI _j , PI _j-1 and PI _j-2 ) and then added, a composite image signal considering the motion of the image Is preferable.
Alternatively, a motion compensation prediction calculation between the original images (PI _j , PI _j-1 and PI _j-2 ) may be performed, and the calculation result may be added to one of the original images (for example, PI _j ). Since the calculation result represents the moving part of the subject, the calculation result can be particularly suitable for shooting in a dark place of a moving subject.

In each of the above embodiments, addition processing is performed in units of image signals (frame units), but the present invention is not limited to this. In particular, since an image signal output from a solid-state imaging device such as a CCD is easy to handle in units of pixels, addition processing may be performed in units of pixels instead of in units of frames.
FIG. 10 is a structural diagram of the buffer memory 20 when the addition processing is performed in units of pixels. In the illustrated example, an input image buffer 30 and an output image buffer 31 each including one buffer area BI and BO are shown. Yes.

  The size of the buffer area BI of the input image buffer 30 is large enough to store the YUV signal for one image output from the color process circuit 18, and the size of the buffer area BO of the output image buffer 31 is about the same. It is a size.

  FIG. 11 is a pixel arrangement diagram of image signals stored in the buffer areas BI and BO, and each cell is a pixel. The horizontal direction corresponds to the horizontal scanning direction of the CCD 13, and the vertical direction corresponds to the vertical scanning direction. Hereinafter, the pixel position is expressed by coordinates (i, j). Here, i is a horizontal coordinate value, and j is a vertical coordinate value. Now, assuming that the coordinates of the pixel of interest are (i, j), the coordinates of eight adjacent pixels vertically and horizontally are (i−1, j−1), (i, j) as shown in FIG. j-1), (i + 1, j-1), (i + 1, j), (i + 1, j + 1), (i, j + 1), (i-1, j + 1), (i-1, j). it can.

  FIG. 13 is a flowchart showing a moving image shooting mode program using the input image buffer 30 and the output image buffer 31 of FIG. This program is stored in advance in the program ROM 24a of the CPU 24, and is loaded into the work RAM 24b of the CPU 24 and executed by a predetermined key operation of the key input unit 25.

This program circulates the i × j pixels of the image signal stored in the buffer area BI of the input image buffer 30 as the target pixel, and the pixel values of the eight neighboring pixels around the pixel value of the target pixel. And the result of the addition is sequentially stored in the buffer area BO of the output image buffer 31.
That is, when the program is started, first, the initial value 1 is set to the loop variables i and j (S20), and then an addition process (S21) described later is executed, and then stored in the buffer area BI of the input image buffer 30. The same addition processing is executed for all of the i × j pixels of the image signal thus obtained (S22 to S26). S22 and S23 are pixel selection steps in the horizontal scanning direction, and S24 to S26 are pixel selection steps in the vertical direction. Further, the number of pixels in the horizontal direction and the vertical direction is set to 12 for convenience (see S22 and S24).

  FIG. 14 shows details of the addition processing in S21. In this figure, a target pixel (i, j) ′ with a dash (′) on the upper right is a pixel stored in the buffer area BO of the output image buffer 31, and this target pixel (i, j) ′ includes , The pixel value of the target pixel (i, j) of the image signal stored in the buffer area BI of the input image buffer 30, and the eight neighboring pixels (i-1, j-1), (i, j) -1), (i + 1, j-1), (i + 1, j), (i + 1, j + 1), (i, j + 1), (i-1, j + 1), and (i-1, j) pixel values An added value, that is, a value obtained by adding the nine pixel values in FIG. 12 is set (S21_1).

  If the nine pixel values in FIG. 12 are a for convenience, the value of (i, j) ′ is “a × 9”. This corresponds to multiplying the pixel value of (i, j) by 9 times and setting it to (i, j) ′. Therefore, since an increase in the pixel value means an approach to the white level, it is possible to take a dark place without changing the frame period of the CCD 12 in this embodiment.

In the addition process of FIG. 14, the pixel value of the pixel of interest and the pixel values of eight neighboring pixels around it are added. This can be shown as shown in FIG.
In FIG. 15, cross hatching is a pixel of interest, and left-down hatching is an adjacent pixel. Such a pixel arrangement is when the coordinates of the target pixel satisfy all the conditions of i> i _min , i <i _max , j> j _min and j <j _max . Here, i _min is the minimum value of coordinate i, i _max is the maximum value of coordinate i, j _min is the minimum value of coordinate j, and j _max is the maximum value of coordinate j.

FIG. 16 is a diagram showing each case that does not satisfy such a condition. That is, (a) does not satisfy i> i _min and j> j _min , (b) does not satisfy i <i _max and j> j _min , and (c) does i <i _max and j <J _max is not satisfied, (d) does not satisfy i> i _min and j <j _max , (e) does not satisfy j> j _min , and (f) is j <j _max (G) does not satisfy i> i _min and (h) does not satisfy i <i _max . In each of these cases, some of the adjacent pixels do not exist (the cells shown by the broken lines in the figure), so it is necessary to add the pixel values of the non-existing pixels as 0.

In the addition process of FIG. 14, the pixel value of the target pixel and the pixel values of eight neighboring pixels around it are simply added, but the present invention is not limited to this. For example, weight values W _NW , W _N , W _NE , W _E , W _SE , W _S , W _SW , W _W , and W _C as shown in FIG. 17 may be applied.
Here, W _NW is the weight value of the adjacent pixel (i−1, j−1) positioned at the upper left of the target pixel (i, j), and W _N is the adjacent pixel positioned above the target pixel (i, j). The weight value of (i, j−1), W _NE is the weight value of the adjacent pixel (i + 1, j−1) located at the upper right of the target pixel (i, j), and W _E is the target pixel (i, j). The weight value of the adjacent pixel (i + 1, j) located on the right, W _SE is the weight value of the adjacent pixel (i + 1, j + 1) located on the lower right of the target pixel (i, j), and W _S is the target pixel (i, j). j) is the weight value of the adjacent pixel (i, j + 1) located below, W _SW is the weight value of the adjacent pixel (i−1, j + 1) located at the lower left of the target pixel (i, j), and W _W is the target The weight value of the adjacent pixel (i−1, j) located to the left of the pixel (i, j) and W _C is the weight value of the target pixel (i, j).

Each weight value is set to a smaller value as the distance from the target pixel increases. For example, W _NW = W _NE = W _SE = W _SW = 1, W _N = W _E = W _S = W _W = 3, and W _C = 5 may be used. As the distance from the pixel of interest increases, the correlation between the pixel values decreases, so that it is possible to avoid deterioration in image quality due to addition.

14 may be improved as shown in FIG. In this improved example, the correlation between the target pixel and the neighboring pixels around the target pixel is examined, and the pixel value of the adjacent pixel having a high correlation is added to the pixel value of the target pixel. Degradation of image quality due to addition can be more actively avoided.
When the addition processing of FIG. 18 is started, first, the difference between the pixel value of the target pixel (i, j) and the pixel value of the adjacent pixel (i−1, j) on the left is calculated, and the absolute value of the calculation result is calculated. Is set to the variable V _L (S22_2), the difference between the pixel value of the target pixel (i, j) and the pixel value of the adjacent pixel (i + 1, j) on the right is calculated, and the absolute value of the calculation result is calculated. to set the variable V _R (S22_3). Next, V _L and V _R are compared to determine whether or not V _L > V _R (S22_4).

When the determination result is YES (when V _L > V _R ), the absolute difference between the pixel value of the target pixel (i, j) and the pixel value of the adjacent pixel (i−1, j) adjacent to the left The value is larger than the absolute value of the difference between the pixel value of the target pixel (i, j) and the pixel value of the adjacent pixel (i + 1, j) adjacent to the right side. In this case, for the target pixel (i, j) Since the right neighboring pixel (i + 1, j) has high correlation, as shown in FIG. 19A, five neighboring pixels (i, j−1) including the right neighboring pixel (i + 1, j). , (I + 1, j−1), (i + 1, j), (i + 1, j + 1), (i, j + 1) are added to the pixel value of the target pixel (i, j) (S22_5).

On the other hand, when the determination result is NO (when V _L > V _R is not satisfied), the difference between the pixel value of the target pixel (i, j) and the pixel value of the adjacent pixel (i−1, j) adjacent to the left is calculated. The absolute value is smaller than the absolute value of the difference between the pixel value of the target pixel (i, j) and the pixel value of the adjacent pixel (i + 1, j) adjacent to the right side. In this case, for the target pixel (i, j) Since the left neighboring pixel (i−1, j) is highly correlated, as shown in FIG. 19B, five neighboring pixels (i−1, j) including the left neighboring pixel (i−1, j) , J−1), (i−1, j−1), (i−1, j), (i−1, j + 1), (i, j + 1) pixel values of the pixel of interest (i, j). Add to the value (S22_6).

In this way, the pixel values of adjacent pixels having low correlation with the pixel of interest (i, j) can be omitted from the addition target, so that the image quality is improved by avoiding blurring of the contour of the subject. Can do.
In the addition process of FIG. 18, the correlation between the adjacent pixels on the right side and the left side with respect to the pixel of interest is examined, but it goes without saying that the upper and lower adjacent pixels may be used.

It is a block diagram of an electronic still camera. 1 is a structural diagram of a buffer memory according to an embodiment. FIG. It is a flowchart of the moving image recording program which concerns on one embodiment. It is a flowchart (example of simple addition) of a composition process. It is a conceptual diagram (example of simple addition) of composition processing. It is a flowchart (example of weighting addition) of a synthetic | combination process. It is a conceptual diagram (example of weighted addition) of composition processing. It is a flowchart (combined example of motion compensation prediction calculation) of a synthesis process. It is a conceptual diagram (combined example of motion compensation prediction calculation) of a synthesis process. It is a structure diagram of a buffer memory according to another embodiment. It is a pixel structure diagram of an image signal according to another embodiment. It is a figure which shows an attention pixel and the adjacent pixel of the circumference | surroundings. It is a flowchart of the moving image recording program which concerns on other embodiment. It is a flowchart (example of simple addition of pixel values) of composition processing. It is a figure which shows an addition object pixel. It is a pixel array figure which lacks some adjacent pixels. It is a figure which shows the weight value provided to an attention pixel and the adjacent pixel of the circumference | surroundings. It is a flowchart (example which uses correlation evaluation together) of a synthetic | combination process. It is a conceptual diagram (example which uses together evaluation of correlation) of a synthetic | combination process.

Explanation of symbols

13 CCD (image output means)
16 Sample hold circuit (image output means)
17 Analog-digital converter (image output means)
18 color process circuit (image output means)
20 Buffer memory (image acquisition means, holding means)
24 CPU (moving image generating means)

Claims (10)

Image output means for periodically converting the light from the subject into an electrical image signal and outputting it,
Image acquisition means for sequentially acquiring a plurality of image signals from image signals periodically output from the image output means;
A moving image generating means for periodically executing a process of generating a single image signal by synthesizing a plurality of image signals acquired by the image acquiring means;
A moving image generating apparatus comprising:   The moving image generating apparatus according to claim 1, wherein the moving image generating unit generates one image signal by adding and synthesizing the plurality of image signals.   3. The moving image generating apparatus according to claim 2, wherein the moving image generating means performs addition after weighting each of the plurality of image signals.   The moving image generating means performs a motion compensation prediction calculation between the plurality of image signals, and adds a predicted image signal obtained by the calculation to one of the plurality of image signals to generate one image. 2. The moving image generating apparatus according to claim 1, wherein the moving image generating apparatus generates a signal. Image output means for periodically converting the light from the subject into an electrical image signal and outputting it,
Holding means for sequentially holding the image signals periodically output from the image output means;
A moving image that generates a moving image by executing processing for correcting the target pixel information in the image signal using adjacent pixel information in the vicinity of the target pixel with respect to the image signals sequentially held in the holding unit Generating means;
A moving image generating apparatus comprising:   6. The moving image according to claim 5, wherein the moving image generating means corrects the attention pixel information by weighting one or both of the attention pixel information and the adjacent pixel information and then adding the both. Generator.   The moving image generating apparatus according to claim 6, wherein the weighting value is set corresponding to a positional relationship between the target pixel and the adjacent pixel.   6. The moving image generation means checks the correlation of each of the adjacent pixels with respect to the target pixel and uses information on the adjacent pixel having a high correlation as correction information for the target pixel information. Item 7. The moving image generating device according to Item 6. An image acquisition step of sequentially acquiring a plurality of image signals from the image signals periodically output by the image output means;
A moving image generation step of periodically executing a process of generating a single image signal by synthesizing a plurality of image signals acquired in the image acquisition step;
A moving image generating method comprising: A holding step for sequentially holding image signals periodically output by the image output means;
A correction step for generating a moving image by periodically executing processing for correcting the target pixel information in the image signal using the neighboring pixel information with respect to the image signal held in the holding step;
A moving image generating method comprising:

Images